2023-11-28T12:46:57.196 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T12:46:57.196 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972 --name neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_neesingh@teuthology --verbose -- /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/orig.config.yaml 2023-11-28T12:46:57.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T12:46:57.322 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469972' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: 3ae99413970476692dfab1fe38dadb056700a61d owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6754 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 3ae99413970476692dfab1fe38dadb056700a61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_12:38:30 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T12:46:57.322 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa; will attempt to use it 2023-11-28T12:46:57.323 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks 2023-11-28T12:46:57.323 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T12:46:57.324 INFO:teuthology.task.internal:Checking packages... 2023-11-28T12:46:57.347 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T12:46:57.348 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T12:46:57.348 INFO:teuthology.packaging:ref: None 2023-11-28T12:46:57.348 INFO:teuthology.packaging:tag: None 2023-11-28T12:46:57.348 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T12:46:57.348 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:46:57.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T12:46:57.497 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T12:46:57.497 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T12:46:57.504 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T12:46:57.504 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T12:46:57.527 INFO:teuthology.task.internal:Saving configuration 2023-11-28T12:46:57.543 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T12:46:57.549 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T12:46:57.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:36.653404', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T12:46:57.598 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:36.654995', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T12:46:57.599 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T12:46:57.605 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T12:46:57.605 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T12:46:57.605 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T12:46:57.690 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f228de8ca60>, signals=[15]) 2023-11-28T12:46:57.691 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T12:46:57.697 INFO:teuthology.task.internal:Opening connections... 2023-11-28T12:46:57.697 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T12:46:57.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:46:57.770 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T12:46:57.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:46:57.838 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T12:46:57.844 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2023-11-28T12:46:57.865 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2023-11-28T12:46:57.865 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2023-11-28T12:46:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T12:46:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T12:46:57.925 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.6" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T12:46:57.926 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T12:46:57.927 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:46:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T12:46:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T12:46:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T12:46:57.927 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T12:46:57.927 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2023-11-28T12:46:57.946 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2023-11-28T12:46:57.967 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2023-11-28T12:46:57.967 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-11-28T12:46:58.026 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T12:46:58.026 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.6" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T12:46:58.027 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:46:58.028 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T12:46:58.028 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T12:46:58.028 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T12:46:58.028 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T12:46:58.028 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-11-28T12:46:58.046 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T12:46:58.056 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T12:46:58.066 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T12:46:58.066 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2023-11-28T12:46:58.068 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2023-11-28T12:46:58.086 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T12:46:58.092 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T12:46:58.092 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2023-11-28T12:46:58.128 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2023-11-28T12:46:58.156 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T12:46:58.213 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T12:46:58.228 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T12:46:58.228 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-28T12:46:58.228 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-28T12:46:58.228 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T12:46:58.229 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T12:46:58.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:46:58.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:46:58.249 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2023-11-28T12:46:58.249 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2023-11-28T12:46:58.307 INFO:teuthology.orchestra.run.smithi177.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:58.308 INFO:teuthology.task.kernel:Running kernel on smithi177: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:58.308 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2023-11-28T12:46:58.309 INFO:teuthology.orchestra.run.smithi153.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:58.309 INFO:teuthology.task.kernel:Running kernel on smithi153: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:58.310 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum install -y kernel 2023-11-28T12:46:59.511 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:59.512 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-11-28T12:46:59.534 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:59.534 INFO:teuthology.orchestra.run.smithi153.stdout:Unable to read consumer identity 2023-11-28T12:47:00.929 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-11-28T12:47:00.990 INFO:teuthology.orchestra.run.smithi153.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-11-28T12:47:01.638 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-11-28T12:47:02.398 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-11-28T12:47:05.188 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 191 kB/s | 24 kB 00:00 2023-11-28T12:47:05.874 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T12:47:05.933 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-11-28T12:47:05.934 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2023-11-28T12:47:05.934 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-11-28T12:47:05.981 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 303 kB/s | 24 kB 00:00 2023-11-28T12:47:06.047 DEBUG:teuthology.orchestra.run.smithi177:> echo no | sudo yum reinstall kernel || true 2023-11-28T12:47:06.050 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:01 ago on Tue 28 Nov 2023 12:47:05 PM UTC. 2023-11-28T12:47:06.378 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T12:47:06.378 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-11-28T12:47:06.645 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:01 ago on Tue 28 Nov 2023 12:47:05 PM UTC. 2023-11-28T12:47:06.686 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:47:06.692 INFO:teuthology.orchestra.run.smithi153.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T12:47:06.756 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-11-28T12:47:06.757 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2023-11-28T12:47:06.757 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-11-28T12:47:06.779 INFO:teuthology.orchestra.run.smithi177.stderr:Error: No packages marked for reinstall. 2023-11-28T12:47:06.825 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum reinstall -y kernel || true 2023-11-28T12:47:06.889 DEBUG:teuthology.orchestra.run.smithi153:> echo no | sudo yum reinstall kernel || true 2023-11-28T12:47:07.155 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T12:47:07.155 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-11-28T12:47:07.221 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-11-28T12:47:07.221 INFO:teuthology.orchestra.run.smithi153.stdout:Unable to read consumer identity 2023-11-28T12:47:07.417 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 12:47:05 PM UTC. 2023-11-28T12:47:07.457 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:47:07.487 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 12:47:05 PM UTC. 2023-11-28T12:47:07.531 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:47:07.570 INFO:teuthology.orchestra.run.smithi153.stderr:Error: No packages marked for reinstall. 2023-11-28T12:47:07.577 INFO:teuthology.orchestra.run.smithi177.stderr:Error: No packages marked for reinstall. 2023-11-28T12:47:07.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum reinstall -y kernel || true 2023-11-28T12:47:07.629 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T12:47:07.709 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T12:47:07.710 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:47:07.710 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T12:47:07.710 DEBUG:teuthology.parallel:result is None 2023-11-28T12:47:07.949 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-11-28T12:47:07.949 INFO:teuthology.orchestra.run.smithi153.stdout:Unable to read consumer identity 2023-11-28T12:47:08.204 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:03 ago on Tue 28 Nov 2023 12:47:05 PM UTC. 2023-11-28T12:47:08.243 INFO:teuthology.orchestra.run.smithi153.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:47:08.264 INFO:teuthology.orchestra.run.smithi153.stderr:Error: No packages marked for reinstall. 2023-11-28T12:47:08.310 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T12:47:08.389 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T12:47:08.390 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:47:08.390 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T12:47:08.390 DEBUG:teuthology.parallel:result is None 2023-11-28T12:47:08.390 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T12:47:08.396 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T12:47:08.397 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T12:47:08.399 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T12:47:08.418 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T12:47:08.484 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T12:47:08.490 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T12:47:08.490 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T12:47:08.493 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T12:47:08.518 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T12:47:08.525 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T12:47:08.526 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T12:47:08.551 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T12:47:08.595 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:47:08.615 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:47:08.628 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:47:08.650 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:47:08.651 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T12:47:08.658 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T12:47:08.659 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T12:47:08.661 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T12:47:08.690 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T12:47:08.699 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T12:47:08.700 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T12:47:08.728 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T12:47:08.749 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:47:08.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:47:08.874 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:47:08.959 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:47:08.999 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:47:08.999 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T12:47:09.067 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:47:09.115 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:47:09.169 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:47:09.256 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:47:09.302 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:47:09.302 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T12:47:09.374 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2023-11-28T12:47:09.379 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2023-11-28T12:47:09.434 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:47:09.473 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:47:09.889 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T12:47:09.897 INFO:teuthology.task.internal:Starting timer... 2023-11-28T12:47:09.897 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T12:47:09.907 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T12:47:09.915 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T12:47:09.916 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2023-11-28T12:47:09.974 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2023-11-28T12:47:09.975 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2023-11-28T12:47:10.046 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2023-11-28T12:47:10.048 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T12:47:10.048 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2023-11-28T12:47:10.072 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2023-11-28T12:47:10.072 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2023-11-28T12:47:10.098 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2023-11-28T12:47:10.098 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi153.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2023-11-28T12:47:10.099 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:47:10.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:47:10.139 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:47:10.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:47:10.167 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T12:47:10.167 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2023-11-28T12:47:10.206 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2023-11-28T12:47:10.234 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T12:47:10.243 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T12:47:10.243 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T12:47:10.261 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T12:47:10.262 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi153.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T12:56:51.911 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi153.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2023-11-28T12:56:51.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-11-28T12:56:51.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:56:51.981 DEBUG:teuthology.orchestra.run.smithi153:> true 2023-11-28T12:56:52.052 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2023-11-28T12:56:52.052 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-11-28T12:56:52.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:56:52.121 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-11-28T12:56:52.189 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-11-28T12:56:52.190 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T12:56:52.200 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T12:56:52.200 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T12:56:52.201 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T12:56:52.203 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T12:56:52.203 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T12:56:52.233 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T12:56:52.249 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T12:56:52.261 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T12:56:52.275 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T12:56:52.323 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2023-11-28T12:56:52.335 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2023-11-28T12:56:52.348 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2023-11-28T12:56:52.349 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T12:56:52.361 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2023-11-28T12:56:52.364 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T12:56:52.379 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T12:56:52.390 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T12:56:52.436 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-11-28T12:56:52.440 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T12:56:52.440 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-11-28T12:56:52.440 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.440 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.440 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.440 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.461 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-11-28T12:56:52.465 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T12:56:52.465 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-11-28T12:56:52.465 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.466 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.466 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.466 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:52.466 INFO:teuthology.run_tasks:Running task install... 2023-11-28T12:56:52.505 DEBUG:teuthology.task.install:project ceph 2023-11-28T12:56:52.506 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T12:56:52.506 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T12:56:52.506 INFO:teuthology.task.install:Using flavor: default 2023-11-28T12:56:52.512 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T12:56:52.512 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T12:56:52.512 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T12:56:52.513 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:56:52.514 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T12:56:52.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:56:52.681 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T12:56:52.681 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T12:56:52.744 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T12:56:52.744 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T12:56:52.860 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T12:56:52.861 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:56:52.861 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T12:56:52.874 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T12:56:52.874 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:56:52.874 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T12:56:52.890 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T12:56:52.890 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T12:56:52.905 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T12:56:52.906 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T12:56:52.966 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T12:56:52.980 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T12:56:53.045 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T12:56:53.059 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T12:56:53.112 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2023-11-28T12:56:53.113 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2023-11-28T12:56:53.129 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2023-11-28T12:56:53.131 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2023-11-28T12:56:53.486 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:53.502 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:54.585 INFO:teuthology.orchestra.run.smithi153.stdout:44 files removed 2023-11-28T12:56:54.640 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T12:56:54.676 INFO:teuthology.orchestra.run.smithi177.stdout:44 files removed 2023-11-28T12:56:54.731 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T12:56:54.974 INFO:teuthology.orchestra.run.smithi153.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:55.063 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:56.484 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 228 kB/s | 80 kB 00:00 2023-11-28T12:56:56.567 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 227 kB/s | 80 kB 00:00 2023-11-28T12:56:56.825 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-11-28T12:56:56.926 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-11-28T12:56:57.202 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-11-28T12:56:57.277 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-11-28T12:56:57.336 INFO:teuthology.orchestra.run.smithi153.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-11-28T12:56:57.427 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-28T12:56:58.469 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-11-28T12:56:58.664 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-11-28T12:57:01.961 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 401 kB/s | 24 kB 00:00 2023-11-28T12:57:02.203 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-11-28T12:57:03.146 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-11-28T12:57:03.396 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-11-28T12:57:11.113 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 MB/s | 53 MB 00:00 2023-11-28T12:57:11.781 INFO:teuthology.orchestra.run.smithi153.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-11-28T12:57:22.246 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:22.249 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:22.262 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:22.263 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:22.264 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T12:57:22.265 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T12:57:22.268 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T12:57:22.271 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T12:57:22.272 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T12:57:22.410 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-11-28T12:57:22.422 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-28T12:57:22.422 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2023-11-28T12:57:22.422 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-28T12:57:22.422 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2023-11-28T12:57:22.422 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T12:57:22.422 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T12:57:22.423 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T12:57:22.424 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T12:57:22.425 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T12:57:22.426 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T12:57:22.427 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T12:57:22.428 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T12:57:22.429 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T12:57:22.430 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T12:57:22.431 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T12:57:22.432 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T12:57:22.433 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout:Enabling module streams: 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql 10 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-28T12:57:22.434 INFO:teuthology.orchestra.run.smithi177.stdout:Install 128 Packages 2023-11-28T12:57:22.435 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 3 Packages 2023-11-28T12:57:22.435 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:57:22.435 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 203 M 2023-11-28T12:57:22.435 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2023-11-28T12:57:22.645 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:22.646 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:22.660 INFO:teuthology.orchestra.run.smithi153.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:22.661 INFO:teuthology.orchestra.run.smithi153.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:22.662 INFO:teuthology.orchestra.run.smithi153.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T12:57:22.663 INFO:teuthology.orchestra.run.smithi153.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T12:57:22.665 INFO:teuthology.orchestra.run.smithi153.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T12:57:22.668 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T12:57:22.670 INFO:teuthology.orchestra.run.smithi153.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T12:57:22.763 INFO:teuthology.orchestra.run.smithi177.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 36 kB/s | 6.4 kB 00:00 2023-11-28T12:57:22.798 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2023-11-28T12:57:22.810 INFO:teuthology.orchestra.run.smithi153.stdout:=========================================================================================================================== 2023-11-28T12:57:22.810 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2023-11-28T12:57:22.810 INFO:teuthology.orchestra.run.smithi153.stdout:=========================================================================================================================== 2023-11-28T12:57:22.810 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2023-11-28T12:57:22.810 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T12:57:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T12:57:22.812 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T12:57:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T12:57:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T12:57:22.815 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T12:57:22.816 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T12:57:22.817 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T12:57:22.818 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T12:57:22.819 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:22.820 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T12:57:22.821 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling module streams: 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql 10 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout:=========================================================================================================================== 2023-11-28T12:57:22.822 INFO:teuthology.orchestra.run.smithi153.stdout:Install 128 Packages 2023-11-28T12:57:22.823 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 3 Packages 2023-11-28T12:57:22.823 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:57:22.823 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 203 M 2023-11-28T12:57:22.823 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2023-11-28T12:57:23.346 INFO:teuthology.orchestra.run.smithi177.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.4 MB/s | 845 kB 00:00 2023-11-28T12:57:23.400 INFO:teuthology.orchestra.run.smithi153.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 36 kB/s | 6.4 kB 00:00 2023-11-28T12:57:23.447 INFO:teuthology.orchestra.run.smithi177.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.6 MB/s | 165 kB 00:00 2023-11-28T12:57:23.948 INFO:teuthology.orchestra.run.smithi153.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.4 MB/s | 845 kB 00:00 2023-11-28T12:57:24.064 INFO:teuthology.orchestra.run.smithi153.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.4 MB/s | 165 kB 00:00 2023-11-28T12:57:24.130 INFO:teuthology.orchestra.run.smithi177.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 3.3 MB/s | 2.3 MB 00:00 2023-11-28T12:57:24.298 INFO:teuthology.orchestra.run.smithi153.stdout:(4/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 4.8 MB/s | 5.4 MB 00:01 2023-11-28T12:57:24.372 INFO:teuthology.orchestra.run.smithi177.stdout:(5/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 3.0 MB/s | 5.4 MB 00:01 2023-11-28T12:57:24.540 INFO:teuthology.orchestra.run.smithi153.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 6.4 MB/s | 1.5 MB 00:00 2023-11-28T12:57:24.580 INFO:teuthology.orchestra.run.smithi177.stdout:(6/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 3.4 MB/s | 1.5 MB 00:00 2023-11-28T12:57:25.240 INFO:teuthology.orchestra.run.smithi153.stdout:(6/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-11-28T12:57:25.349 INFO:teuthology.orchestra.run.smithi153.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 5.7 MB/s | 4.6 MB 00:00 2023-11-28T12:57:25.439 INFO:teuthology.orchestra.run.smithi177.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 4.3 MB/s | 4.6 MB 00:01 2023-11-28T12:57:26.706 INFO:teuthology.orchestra.run.smithi177.stdout:(8/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 5.2 MB/s | 22 MB 00:04 2023-11-28T12:57:26.798 INFO:teuthology.orchestra.run.smithi177.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 266 kB/s | 24 kB 00:00 2023-11-28T12:57:27.007 INFO:teuthology.orchestra.run.smithi153.stdout:(8/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 5.6 MB/s | 22 MB 00:03 2023-11-28T12:57:27.082 INFO:teuthology.orchestra.run.smithi153.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 325 kB/s | 24 kB 00:00 2023-11-28T12:57:28.458 INFO:teuthology.orchestra.run.smithi153.stdout:(10/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 4.2 MB/s | 13 MB 00:03 2023-11-28T12:57:28.641 INFO:teuthology.orchestra.run.smithi153.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.4 MB/s | 257 kB 00:00 2023-11-28T12:57:28.708 INFO:teuthology.orchestra.run.smithi153.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 458 kB/s | 30 kB 00:00 2023-11-28T12:57:28.798 INFO:teuthology.orchestra.run.smithi177.stdout:(10/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 3.9 MB/s | 13 MB 00:03 2023-11-28T12:57:28.900 INFO:teuthology.orchestra.run.smithi153.stdout:(13/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 4.8 MB/s | 18 MB 00:03 2023-11-28T12:57:28.940 INFO:teuthology.orchestra.run.smithi177.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.8 MB/s | 257 kB 00:00 2023-11-28T12:57:28.984 INFO:teuthology.orchestra.run.smithi153.stdout:(14/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 2.7 MB/s | 767 kB 00:00 2023-11-28T12:57:29.009 INFO:teuthology.orchestra.run.smithi153.stdout:(15/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.2 MB/s | 137 kB 00:00 2023-11-28T12:57:29.031 INFO:teuthology.orchestra.run.smithi177.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 333 kB/s | 30 kB 00:00 2023-11-28T12:57:29.060 INFO:teuthology.orchestra.run.smithi153.stdout:(16/131): librados-devel-18.0.0-2697.g2f5f6028. 1.8 MB/s | 139 kB 00:00 2023-11-28T12:57:29.110 INFO:teuthology.orchestra.run.smithi153.stdout:(17/131): libradosstriper1-18.0.0-2697.g2f5f602 4.6 MB/s | 474 kB 00:00 2023-11-28T12:57:29.210 INFO:teuthology.orchestra.run.smithi153.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 446 kB/s | 45 kB 00:00 2023-11-28T12:57:29.327 INFO:teuthology.orchestra.run.smithi153.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.0 MB/s | 117 kB 00:00 2023-11-28T12:57:29.365 INFO:teuthology.orchestra.run.smithi177.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 2.2 MB/s | 767 kB 00:00 2023-11-28T12:57:29.427 INFO:teuthology.orchestra.run.smithi153.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 1.9 MB/s | 190 kB 00:00 2023-11-28T12:57:29.474 INFO:teuthology.orchestra.run.smithi177.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.2 MB/s | 137 kB 00:00 2023-11-28T12:57:29.546 INFO:teuthology.orchestra.run.smithi153.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 3.1 MB/s | 371 kB 00:00 2023-11-28T12:57:29.599 INFO:teuthology.orchestra.run.smithi177.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 1.1 MB/s | 139 kB 00:00 2023-11-28T12:57:29.661 INFO:teuthology.orchestra.run.smithi153.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 3.1 MB/s | 364 kB 00:00 2023-11-28T12:57:29.761 INFO:teuthology.orchestra.run.smithi153.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.0 MB/s | 107 kB 00:00 2023-11-28T12:57:29.799 INFO:teuthology.orchestra.run.smithi177.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 2.3 MB/s | 474 kB 00:00 2023-11-28T12:57:29.870 INFO:teuthology.orchestra.run.smithi153.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 873 kB/s | 94 kB 00:00 2023-11-28T12:57:30.512 INFO:teuthology.orchestra.run.smithi153.stdout:(25/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 5.4 MB/s | 3.4 MB 00:00 2023-11-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi153.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.8 MB/s | 180 kB 00:00 2023-11-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi153.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 252 kB/s | 23 kB 00:00 2023-11-28T12:57:30.796 INFO:teuthology.orchestra.run.smithi153.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.3 MB/s | 123 kB 00:00 2023-11-28T12:57:31.105 INFO:teuthology.orchestra.run.smithi153.stdout:(29/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 5.6 MB/s | 1.7 MB 00:00 2023-11-28T12:57:31.855 INFO:teuthology.orchestra.run.smithi153.stdout:(30/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 2.3 MB/s | 6.5 MB 00:02 2023-11-28T12:57:31.963 INFO:teuthology.orchestra.run.smithi153.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 2.1 MB/s | 236 kB 00:00 2023-11-28T12:57:32.030 INFO:teuthology.orchestra.run.smithi153.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 710 kB/s | 47 kB 00:00 2023-11-28T12:57:32.089 INFO:teuthology.orchestra.run.smithi153.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 249 kB/s | 15 kB 00:00 2023-11-28T12:57:32.206 INFO:teuthology.orchestra.run.smithi153.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 6.7 MB/s | 7.4 MB 00:01 2023-11-28T12:57:32.208 INFO:teuthology.orchestra.run.smithi177.stdout:(17/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 8.6 MB/s | 47 MB 00:05 2023-11-28T12:57:32.231 INFO:teuthology.orchestra.run.smithi153.stdout:(35/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 1.8 MB/s | 263 kB 00:00 2023-11-28T12:57:32.273 INFO:teuthology.orchestra.run.smithi153.stdout:(36/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.2 MB/s | 83 kB 00:00 2023-11-28T12:57:32.293 INFO:teuthology.orchestra.run.smithi177.stdout:(18/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 2.6 MB/s | 6.5 MB 00:02 2023-11-28T12:57:32.299 INFO:teuthology.orchestra.run.smithi153.stdout:(37/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 270 kB/s | 18 kB 00:00 2023-11-28T12:57:32.317 INFO:teuthology.orchestra.run.smithi177.stdout:(19/131): python3-ceph-argparse-18.0.0-2697.g2f 412 kB/s | 45 kB 00:00 2023-11-28T12:57:32.407 INFO:teuthology.orchestra.run.smithi153.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 544 kB/s | 59 kB 00:00 2023-11-28T12:57:32.484 INFO:teuthology.orchestra.run.smithi177.stdout:(20/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 2.2 MB/s | 18 MB 00:07 2023-11-28T12:57:32.497 INFO:teuthology.orchestra.run.smithi153.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 842 kB/s | 75 kB 00:00 2023-11-28T12:57:32.517 INFO:teuthology.orchestra.run.smithi177.stdout:(21/131): python3-ceph-common-18.0.0-2697.g2f5f 520 kB/s | 117 kB 00:00 2023-11-28T12:57:32.525 INFO:teuthology.orchestra.run.smithi153.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-28T12:57:32.542 INFO:teuthology.orchestra.run.smithi177.stdout:(22/131): python3-cephfs-18.0.0-2697.g2f5f6028. 844 kB/s | 190 kB 00:00 2023-11-28T12:57:32.583 INFO:teuthology.orchestra.run.smithi153.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 750 kB/s | 44 kB 00:00 2023-11-28T12:57:32.618 INFO:teuthology.orchestra.run.smithi177.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.4 MB/s | 107 kB 00:00 2023-11-28T12:57:32.643 INFO:teuthology.orchestra.run.smithi177.stdout:(24/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.8 MB/s | 364 kB 00:00 2023-11-28T12:57:32.668 INFO:teuthology.orchestra.run.smithi177.stdout:(25/131): python3-rados-18.0.0-2697.g2f5f6028.e 2.0 MB/s | 371 kB 00:00 2023-11-28T12:57:32.694 INFO:teuthology.orchestra.run.smithi177.stdout:(26/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.2 MB/s | 94 kB 00:00 2023-11-28T12:57:32.752 INFO:teuthology.orchestra.run.smithi177.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 396 kB/s | 23 kB 00:00 2023-11-28T12:57:32.786 INFO:teuthology.orchestra.run.smithi177.stdout:(28/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.5 MB/s | 180 kB 00:00 2023-11-28T12:57:32.811 INFO:teuthology.orchestra.run.smithi177.stdout:(29/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 2.0 MB/s | 123 kB 00:00 2023-11-28T12:57:32.828 INFO:teuthology.orchestra.run.smithi153.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 756 kB/s | 185 kB 00:00 2023-11-28T12:57:32.884 INFO:teuthology.orchestra.run.smithi153.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-11-28T12:57:32.909 INFO:teuthology.orchestra.run.smithi153.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 410 kB/s | 33 kB 00:00 2023-11-28T12:57:32.935 INFO:teuthology.orchestra.run.smithi153.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2023-11-28T12:57:32.968 INFO:teuthology.orchestra.run.smithi153.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-11-28T12:57:33.010 INFO:teuthology.orchestra.run.smithi153.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-11-28T12:57:33.044 INFO:teuthology.orchestra.run.smithi153.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-11-28T12:57:33.069 INFO:teuthology.orchestra.run.smithi153.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-11-28T12:57:33.111 INFO:teuthology.orchestra.run.smithi153.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-11-28T12:57:33.153 INFO:teuthology.orchestra.run.smithi153.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-11-28T12:57:33.195 INFO:teuthology.orchestra.run.smithi153.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-11-28T12:57:33.262 INFO:teuthology.orchestra.run.smithi153.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2023-11-28T12:57:33.287 INFO:teuthology.orchestra.run.smithi153.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 657 kB/s | 60 kB 00:00 2023-11-28T12:57:33.329 INFO:teuthology.orchestra.run.smithi153.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-11-28T12:57:33.346 INFO:teuthology.orchestra.run.smithi153.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-11-28T12:57:33.371 INFO:teuthology.orchestra.run.smithi153.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 803 kB/s | 34 kB 00:00 2023-11-28T12:57:33.397 INFO:teuthology.orchestra.run.smithi153.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-11-28T12:57:33.422 INFO:teuthology.orchestra.run.smithi153.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-11-28T12:57:33.447 INFO:teuthology.orchestra.run.smithi153.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-11-28T12:57:33.481 INFO:teuthology.orchestra.run.smithi153.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-11-28T12:57:33.506 INFO:teuthology.orchestra.run.smithi153.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 499 kB/s | 29 kB 00:00 2023-11-28T12:57:33.528 INFO:teuthology.orchestra.run.smithi177.stdout:(30/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 3.9 MB/s | 3.4 MB 00:00 2023-11-28T12:57:33.531 INFO:teuthology.orchestra.run.smithi153.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-11-28T12:57:33.595 INFO:teuthology.orchestra.run.smithi177.stdout:(31/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 2.1 MB/s | 1.7 MB 00:00 2023-11-28T12:57:33.615 INFO:teuthology.orchestra.run.smithi153.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-11-28T12:57:33.620 INFO:teuthology.orchestra.run.smithi177.stdout:(32/131): ceph-mgr-modules-core-18.0.0-2697.g2f 2.5 MB/s | 236 kB 00:00 2023-11-28T12:57:33.640 INFO:teuthology.orchestra.run.smithi153.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-11-28T12:57:33.666 INFO:teuthology.orchestra.run.smithi153.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-28T12:57:33.699 INFO:teuthology.orchestra.run.smithi153.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-11-28T12:57:33.712 INFO:teuthology.orchestra.run.smithi177.stdout:(33/131): ceph-mgr-diskprediction-local-18.0.0- 8.2 MB/s | 7.4 MB 00:00 2023-11-28T12:57:33.725 INFO:teuthology.orchestra.run.smithi153.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 391 kB/s | 23 kB 00:00 2023-11-28T12:57:33.737 INFO:teuthology.orchestra.run.smithi177.stdout:(34/131): ceph-prometheus-alerts-18.0.0-2697.g2 124 kB/s | 15 kB 00:00 2023-11-28T12:57:33.750 INFO:teuthology.orchestra.run.smithi153.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-28T12:57:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:(35/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 283 kB/s | 47 kB 00:00 2023-11-28T12:57:33.796 INFO:teuthology.orchestra.run.smithi177.stdout:(36/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 3.1 MB/s | 263 kB 00:00 2023-11-28T12:57:33.821 INFO:teuthology.orchestra.run.smithi177.stdout:(37/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 990 kB/s | 83 kB 00:00 2023-11-28T12:57:33.847 INFO:teuthology.orchestra.run.smithi177.stdout:(38/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 215 kB/s | 18 kB 00:00 2023-11-28T12:57:33.917 INFO:teuthology.orchestra.run.smithi153.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 281 kB/s | 47 kB 00:00 2023-11-28T12:57:33.992 INFO:teuthology.orchestra.run.smithi153.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2023-11-28T12:57:34.005 INFO:teuthology.orchestra.run.smithi177.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 320 kB/s | 59 kB 00:00 2023-11-28T12:57:34.031 INFO:teuthology.orchestra.run.smithi177.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 430 kB/s | 75 kB 00:00 2023-11-28T12:57:34.064 INFO:teuthology.orchestra.run.smithi177.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-11-28T12:57:34.089 INFO:teuthology.orchestra.run.smithi177.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2023-11-28T12:57:34.100 INFO:teuthology.orchestra.run.smithi153.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.1 MB/s | 200 kB 00:00 2023-11-28T12:57:34.115 INFO:teuthology.orchestra.run.smithi177.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-11-28T12:57:34.126 INFO:teuthology.orchestra.run.smithi153.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 320 kB/s | 43 kB 00:00 2023-11-28T12:57:34.140 INFO:teuthology.orchestra.run.smithi177.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2023-11-28T12:57:34.165 INFO:teuthology.orchestra.run.smithi177.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2023-11-28T12:57:34.191 INFO:teuthology.orchestra.run.smithi177.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-11-28T12:57:34.232 INFO:teuthology.orchestra.run.smithi177.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-11-28T12:57:34.234 INFO:teuthology.orchestra.run.smithi153.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 444 kB/s | 48 kB 00:00 2023-11-28T12:57:34.258 INFO:teuthology.orchestra.run.smithi177.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2023-11-28T12:57:34.260 INFO:teuthology.orchestra.run.smithi153.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-11-28T12:57:34.275 INFO:teuthology.orchestra.run.smithi177.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 971 kB/s | 81 kB 00:00 2023-11-28T12:57:34.308 INFO:teuthology.orchestra.run.smithi177.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-11-28T12:57:34.334 INFO:teuthology.orchestra.run.smithi177.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 232 kB/s | 18 kB 00:00 2023-11-28T12:57:34.367 INFO:teuthology.orchestra.run.smithi177.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 833 kB/s | 49 kB 00:00 2023-11-28T12:57:34.392 INFO:teuthology.orchestra.run.smithi177.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-11-28T12:57:34.402 INFO:teuthology.orchestra.run.smithi153.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 301 kB/s | 43 kB 00:00 2023-11-28T12:57:34.418 INFO:teuthology.orchestra.run.smithi177.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-11-28T12:57:34.427 INFO:teuthology.orchestra.run.smithi153.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 453 kB/s | 87 kB 00:00 2023-11-28T12:57:34.451 INFO:teuthology.orchestra.run.smithi177.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 6.6 MB/s | 1.2 MB 00:00 2023-11-28T12:57:34.477 INFO:teuthology.orchestra.run.smithi177.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-11-28T12:57:34.502 INFO:teuthology.orchestra.run.smithi177.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-11-28T12:57:34.527 INFO:teuthology.orchestra.run.smithi177.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-11-28T12:57:34.528 INFO:teuthology.orchestra.run.smithi153.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 872 kB/s | 109 kB 00:00 2023-11-28T12:57:34.552 INFO:teuthology.orchestra.run.smithi177.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-11-28T12:57:34.561 INFO:teuthology.orchestra.run.smithi153.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2023-11-28T12:57:34.578 INFO:teuthology.orchestra.run.smithi177.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-11-28T12:57:34.611 INFO:teuthology.orchestra.run.smithi177.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2023-11-28T12:57:34.645 INFO:teuthology.orchestra.run.smithi177.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2023-11-28T12:57:34.670 INFO:teuthology.orchestra.run.smithi177.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-11-28T12:57:34.694 INFO:teuthology.orchestra.run.smithi153.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-11-28T12:57:34.695 INFO:teuthology.orchestra.run.smithi177.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-11-28T12:57:34.720 INFO:teuthology.orchestra.run.smithi153.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 118 kB/s | 19 kB 00:00 2023-11-28T12:57:34.720 INFO:teuthology.orchestra.run.smithi177.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-11-28T12:57:34.746 INFO:teuthology.orchestra.run.smithi177.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-28T12:57:34.771 INFO:teuthology.orchestra.run.smithi177.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-28T12:57:34.787 INFO:teuthology.orchestra.run.smithi153.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2023-11-28T12:57:34.796 INFO:teuthology.orchestra.run.smithi177.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-11-28T12:57:34.821 INFO:teuthology.orchestra.run.smithi177.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-11-28T12:57:34.839 INFO:teuthology.orchestra.run.smithi177.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-11-28T12:57:34.895 INFO:teuthology.orchestra.run.smithi153.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 3.1 MB/s | 559 kB 00:00 2023-11-28T12:57:34.914 INFO:teuthology.orchestra.run.smithi177.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-11-28T12:57:34.920 INFO:teuthology.orchestra.run.smithi153.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 134 kB/s | 18 kB 00:00 2023-11-28T12:57:34.996 INFO:teuthology.orchestra.run.smithi153.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2023-11-28T12:57:35.039 INFO:teuthology.orchestra.run.smithi177.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 566 kB/s | 110 kB 00:00 2023-11-28T12:57:35.046 INFO:teuthology.orchestra.run.smithi153.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2023-11-28T12:57:35.056 INFO:teuthology.orchestra.run.smithi177.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 382 kB/s | 54 kB 00:00 2023-11-28T12:57:35.147 INFO:teuthology.orchestra.run.smithi153.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-11-28T12:57:35.180 INFO:teuthology.orchestra.run.smithi153.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 542 kB/s | 72 kB 00:00 2023-11-28T12:57:35.299 INFO:teuthology.orchestra.run.smithi153.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 514 kB/s | 60 kB 00:00 2023-11-28T12:57:35.322 INFO:teuthology.orchestra.run.smithi153.stdout:(90/131): fuse-2.9.7-15.el8.x86_64.rpm 471 kB/s | 83 kB 00:00 2023-11-28T12:57:35.404 INFO:teuthology.orchestra.run.smithi177.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 710 kB/s | 259 kB 00:00 2023-11-28T12:57:35.414 INFO:teuthology.orchestra.run.smithi153.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 631 kB/s | 58 kB 00:00 2023-11-28T12:57:35.431 INFO:teuthology.orchestra.run.smithi177.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 860 kB/s | 323 kB 00:00 2023-11-28T12:57:35.440 INFO:teuthology.orchestra.run.smithi153.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 690 kB/s | 98 kB 00:00 2023-11-28T12:57:35.557 INFO:teuthology.orchestra.run.smithi153.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 272 kB/s | 39 kB 00:00 2023-11-28T12:57:35.590 INFO:teuthology.orchestra.run.smithi153.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.2 MB/s | 644 kB 00:00 2023-11-28T12:57:35.624 INFO:teuthology.orchestra.run.smithi177.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 17 MB/s | 14 MB 00:00 2023-11-28T12:57:35.641 INFO:teuthology.orchestra.run.smithi177.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 165 kB/s | 39 kB 00:00 2023-11-28T12:57:35.674 INFO:teuthology.orchestra.run.smithi177.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 2.8 MB/s | 688 kB 00:00 2023-11-28T12:57:35.716 INFO:teuthology.orchestra.run.smithi153.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.1 MB/s | 171 kB 00:00 2023-11-28T12:57:35.716 INFO:teuthology.orchestra.run.smithi177.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-11-28T12:57:35.741 INFO:teuthology.orchestra.run.smithi177.stdout:(80/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.5 MB/s | 538 kB 00:00 2023-11-28T12:57:35.766 INFO:teuthology.orchestra.run.smithi153.stdout:(96/131): NetworkManager-initscripts-updown-1.3 785 kB/s | 138 kB 00:00 2023-11-28T12:57:35.800 INFO:teuthology.orchestra.run.smithi177.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2023-11-28T12:57:35.859 INFO:teuthology.orchestra.run.smithi177.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.9 MB/s | 713 kB 00:00 2023-11-28T12:57:35.884 INFO:teuthology.orchestra.run.smithi177.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 317 kB/s | 27 kB 00:00 2023-11-28T12:57:35.991 INFO:teuthology.orchestra.run.smithi153.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 490 kB/s | 110 kB 00:00 2023-11-28T12:57:36.001 INFO:teuthology.orchestra.run.smithi177.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-11-28T12:57:36.026 INFO:teuthology.orchestra.run.smithi177.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 310 kB/s | 44 kB 00:00 2023-11-28T12:57:36.051 INFO:teuthology.orchestra.run.smithi177.stdout:(86/131): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-11-28T12:57:36.160 INFO:teuthology.orchestra.run.smithi177.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 24 MB/s | 3.7 MB 00:00 2023-11-28T12:57:36.183 INFO:teuthology.orchestra.run.smithi153.stdout:(98/131): python3-scipy-1.0.0-21.module+el8.5.0 29 MB/s | 14 MB 00:00 2023-11-28T12:57:36.185 INFO:teuthology.orchestra.run.smithi177.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.6 MB/s | 225 kB 00:00 2023-11-28T12:57:36.200 INFO:teuthology.orchestra.run.smithi153.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 260 kB/s | 54 kB 00:00 2023-11-28T12:57:36.210 INFO:teuthology.orchestra.run.smithi177.stdout:(89/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.4 MB/s | 457 kB 00:00 2023-11-28T12:57:36.269 INFO:teuthology.orchestra.run.smithi177.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 588 kB/s | 64 kB 00:00 2023-11-28T12:57:36.327 INFO:teuthology.orchestra.run.smithi177.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 890 kB/s | 126 kB 00:00 2023-11-28T12:57:36.334 INFO:teuthology.orchestra.run.smithi153.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.4 MB/s | 323 kB 00:00 2023-11-28T12:57:36.359 INFO:teuthology.orchestra.run.smithi153.stdout:(101/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2023-11-28T12:57:36.361 INFO:teuthology.orchestra.run.smithi177.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 133 kB/s | 20 kB 00:00 2023-11-28T12:57:36.386 INFO:teuthology.orchestra.run.smithi177.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 884 kB/s | 103 kB 00:00 2023-11-28T12:57:36.437 INFO:teuthology.orchestra.run.smithi177.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-11-28T12:57:36.484 INFO:teuthology.orchestra.run.smithi153.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 258 kB/s | 39 kB 00:00 2023-11-28T12:57:36.543 INFO:teuthology.orchestra.run.smithi153.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-11-28T12:57:36.562 INFO:teuthology.orchestra.run.smithi177.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 24 MB/s | 4.9 MB 00:00 2023-11-28T12:57:36.596 INFO:teuthology.orchestra.run.smithi177.stdout:(96/131): libpq-13.5-1.el8.x86_64.rpm 947 kB/s | 198 kB 00:00 2023-11-28T12:57:36.660 INFO:teuthology.orchestra.run.smithi153.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 3.0 MB/s | 538 kB 00:00 2023-11-28T12:57:36.721 INFO:teuthology.orchestra.run.smithi177.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-11-28T12:57:36.735 INFO:teuthology.orchestra.run.smithi153.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 818 kB/s | 157 kB 00:00 2023-11-28T12:57:36.746 INFO:teuthology.orchestra.run.smithi177.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2023-11-28T12:57:36.780 INFO:teuthology.orchestra.run.smithi177.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.9 MB/s | 354 kB 00:00 2023-11-28T12:57:36.835 INFO:teuthology.orchestra.run.smithi153.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-11-28T12:57:36.896 INFO:teuthology.orchestra.run.smithi177.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.3 MB/s | 808 kB 00:00 2023-11-28T12:57:36.986 INFO:teuthology.orchestra.run.smithi153.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 2.8 MB/s | 713 kB 00:00 2023-11-28T12:57:37.013 INFO:teuthology.orchestra.run.smithi177.stdout:(101/131): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-11-28T12:57:37.039 INFO:teuthology.orchestra.run.smithi177.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 330 kB/s | 47 kB 00:00 2023-11-28T12:57:37.081 INFO:teuthology.orchestra.run.smithi177.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 5.0 MB/s | 1.5 MB 00:00 2023-11-28T12:57:37.086 INFO:teuthology.orchestra.run.smithi153.stdout:(108/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-11-28T12:57:37.106 INFO:teuthology.orchestra.run.smithi177.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.1 MB/s | 200 kB 00:00 2023-11-28T12:57:37.111 INFO:teuthology.orchestra.run.smithi153.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 212 kB/s | 27 kB 00:00 2023-11-28T12:57:37.156 INFO:teuthology.orchestra.run.smithi177.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 363 kB/s | 43 kB 00:00 2023-11-28T12:57:37.206 INFO:teuthology.orchestra.run.smithi177.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-11-28T12:57:37.232 INFO:teuthology.orchestra.run.smithi177.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 382 kB/s | 48 kB 00:00 2023-11-28T12:57:37.259 INFO:teuthology.orchestra.run.smithi153.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 298 kB/s | 44 kB 00:00 2023-11-28T12:57:37.316 INFO:teuthology.orchestra.run.smithi177.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 547 kB/s | 87 kB 00:00 2023-11-28T12:57:37.332 INFO:teuthology.orchestra.run.smithi177.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 339 kB/s | 43 kB 00:00 2023-11-28T12:57:37.383 INFO:teuthology.orchestra.run.smithi177.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 724 kB/s | 109 kB 00:00 2023-11-28T12:57:37.425 INFO:teuthology.orchestra.run.smithi177.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.9 MB/s | 560 kB 00:00 2023-11-28T12:57:37.450 INFO:teuthology.orchestra.run.smithi177.stdout:(112/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2023-11-28T12:57:37.484 INFO:teuthology.orchestra.run.smithi177.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 186 kB/s | 19 kB 00:00 2023-11-28T12:57:37.559 INFO:teuthology.orchestra.run.smithi177.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-11-28T12:57:37.592 INFO:teuthology.orchestra.run.smithi177.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.8 MB/s | 559 kB 00:00 2023-11-28T12:57:37.609 INFO:teuthology.orchestra.run.smithi177.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 143 kB/s | 18 kB 00:00 2023-11-28T12:57:37.695 INFO:teuthology.orchestra.run.smithi153.stdout:(111/131): ceph-test-18.0.0-2697.g2f5f6028.el8. 4.4 MB/s | 47 MB 00:10 2023-11-28T12:57:37.709 INFO:teuthology.orchestra.run.smithi177.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-11-28T12:57:37.728 INFO:teuthology.orchestra.run.smithi153.stdout:(112/131): flex-2.6.1-9.el8.x86_64.rpm 498 kB/s | 320 kB 00:00 2023-11-28T12:57:37.776 INFO:teuthology.orchestra.run.smithi177.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2023-11-28T12:57:37.837 INFO:teuthology.orchestra.run.smithi153.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 6.5 MB/s | 3.7 MB 00:00 2023-11-28T12:57:37.843 INFO:teuthology.orchestra.run.smithi177.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.9 MB/s | 2.3 MB 00:00 2023-11-28T12:57:37.862 INFO:teuthology.orchestra.run.smithi153.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.7 MB/s | 225 kB 00:00 2023-11-28T12:57:37.868 INFO:teuthology.orchestra.run.smithi177.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 455 kB/s | 72 kB 00:00 2023-11-28T12:57:37.888 INFO:teuthology.orchestra.run.smithi153.stdout:(115/131): python3-werkzeug-0.12.2-4.el8.noarch 2.3 MB/s | 457 kB 00:00 2023-11-28T12:57:37.894 INFO:teuthology.orchestra.run.smithi177.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 705 kB/s | 83 kB 00:00 2023-11-28T12:57:37.996 INFO:teuthology.orchestra.run.smithi153.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 403 kB/s | 64 kB 00:00 2023-11-28T12:57:38.018 INFO:teuthology.orchestra.run.smithi177.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 344 kB/s | 60 kB 00:00 2023-11-28T12:57:38.038 INFO:teuthology.orchestra.run.smithi153.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 720 kB/s | 126 kB 00:00 2023-11-28T12:57:38.044 INFO:teuthology.orchestra.run.smithi177.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 559 kB/s | 98 kB 00:00 2023-11-28T12:57:38.062 INFO:teuthology.orchestra.run.smithi177.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 345 kB/s | 58 kB 00:00 2023-11-28T12:57:38.063 INFO:teuthology.orchestra.run.smithi153.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 114 kB/s | 20 kB 00:00 2023-11-28T12:57:38.139 INFO:teuthology.orchestra.run.smithi153.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 729 kB/s | 103 kB 00:00 2023-11-28T12:57:38.187 INFO:teuthology.orchestra.run.smithi177.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.4 MB/s | 644 kB 00:00 2023-11-28T12:57:38.203 INFO:teuthology.orchestra.run.smithi177.stdout:(126/131): libcom_err-devel-1.45.6-4.el8.x86_64 208 kB/s | 39 kB 00:00 2023-11-28T12:57:38.229 INFO:teuthology.orchestra.run.smithi177.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.0 MB/s | 171 kB 00:00 2023-11-28T12:57:38.312 INFO:teuthology.orchestra.run.smithi177.stdout:(128/131): NetworkManager-initscripts-updown-1. 1.1 MB/s | 138 kB 00:00 2023-11-28T12:57:38.314 INFO:teuthology.orchestra.run.smithi153.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-11-28T12:57:38.347 INFO:teuthology.orchestra.run.smithi153.stdout:(121/131): libtool-2.4.6-25.el8.x86_64.rpm 2.2 MB/s | 709 kB 00:00 2023-11-28T12:57:38.441 INFO:teuthology.orchestra.run.smithi153.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-11-28T12:57:38.454 INFO:teuthology.orchestra.run.smithi177.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.5 MB/s | 798 kB 00:00 2023-11-28T12:57:38.464 INFO:teuthology.orchestra.run.smithi153.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2023-11-28T12:57:38.573 INFO:teuthology.orchestra.run.smithi153.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-11-28T12:57:38.607 INFO:teuthology.orchestra.run.smithi153.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2023-11-28T12:57:38.674 INFO:teuthology.orchestra.run.smithi153.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-11-28T12:57:38.774 INFO:teuthology.orchestra.run.smithi153.stdout:(127/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-11-28T12:57:38.825 INFO:teuthology.orchestra.run.smithi153.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 6.9 MB/s | 1.5 MB 00:00 2023-11-28T12:57:38.991 INFO:teuthology.orchestra.run.smithi153.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.7 MB/s | 798 kB 00:00 2023-11-28T12:57:39.188 INFO:teuthology.orchestra.run.smithi177.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 3.7 MB/s | 3.6 MB 00:00 2023-11-28T12:57:39.525 INFO:teuthology.orchestra.run.smithi153.stdout:(130/131): librados2-18.0.0-2697.g2f5f6028.el8. 4.3 MB/s | 3.7 MB 00:00 2023-11-28T12:57:39.671 INFO:teuthology.orchestra.run.smithi177.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 2.5 MB/s | 3.7 MB 00:01 2023-11-28T12:57:39.675 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2023-11-28T12:57:39.676 INFO:teuthology.orchestra.run.smithi177.stdout:Total 12 MB/s | 203 MB 00:17 2023-11-28T12:57:40.077 INFO:teuthology.orchestra.run.smithi153.stdout:(131/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 2.7 MB/s | 3.6 MB 00:01 2023-11-28T12:57:40.078 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2023-11-28T12:57:40.078 INFO:teuthology.orchestra.run.smithi153.stdout:Total 12 MB/s | 203 MB 00:17 2023-11-28T12:57:40.508 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2023-11-28T12:57:40.628 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2023-11-28T12:57:40.628 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2023-11-28T12:57:40.921 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2023-11-28T12:57:41.041 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2023-11-28T12:57:41.042 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2023-11-28T12:57:41.728 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2023-11-28T12:57:41.732 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2023-11-28T12:57:42.146 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2023-11-28T12:57:42.150 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2023-11-28T12:57:46.509 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2023-11-28T12:57:46.699 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T12:57:46.867 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T12:57:47.066 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-28T12:57:47.226 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T12:57:47.336 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T12:57:47.453 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2023-11-28T12:57:47.535 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T12:57:47.651 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T12:57:47.847 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T12:57:47.934 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T12:57:48.012 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T12:57:48.134 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T12:57:48.167 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:48.353 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T12:57:48.562 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:48.756 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T12:57:48.759 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:48.892 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:49.076 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:57:49.159 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:57:49.211 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T12:57:49.344 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T12:57:49.579 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T12:57:49.610 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T12:57:49.787 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T12:57:49.845 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:49.980 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:50.186 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:50.195 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T12:57:50.279 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:50.438 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T12:57:50.492 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:57:50.659 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:57:50.670 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T12:57:50.766 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:57:50.942 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:57:50.949 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T12:57:51.032 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:57:51.139 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T12:57:51.150 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T12:57:51.290 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T12:57:51.417 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T12:57:51.576 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T12:57:51.807 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T12:57:52.032 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T12:57:52.036 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:57:52.145 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:57:52.176 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T12:57:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:57:52.541 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:57:52.753 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T12:57:52.756 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T12:57:52.942 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T12:57:52.950 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T12:57:53.151 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T12:57:53.277 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T12:57:53.419 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T12:57:53.577 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T12:57:53.681 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T12:57:53.932 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-28T12:57:54.151 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-28T12:57:54.199 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T12:57:54.354 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-28T12:57:54.411 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T12:57:54.514 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-28T12:57:54.697 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-28T12:57:54.909 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T12:57:54.997 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T12:57:55.065 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T12:57:55.202 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T12:57:55.222 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T12:57:55.940 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T12:57:56.090 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T12:57:56.192 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T12:57:56.396 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-28T12:57:56.417 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T12:57:56.644 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-28T12:57:56.645 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T12:57:56.858 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T12:57:56.905 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-28T12:57:56.996 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T12:57:57.099 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T12:57:57.277 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T12:57:57.502 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T12:57:58.949 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T12:57:59.250 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-28T12:57:59.283 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T12:57:59.460 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T12:57:59.652 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T12:57:59.822 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T12:57:59.969 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T12:58:00.137 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-28T12:58:00.285 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T12:58:00.300 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-28T12:58:00.497 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-28T12:58:00.626 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T12:58:00.660 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-28T12:58:00.846 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-28T12:58:00.938 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T12:58:01.053 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-28T12:58:01.174 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T12:58:01.389 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T12:58:01.416 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-28T12:58:01.612 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T12:58:01.694 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-28T12:58:01.826 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T12:58:01.900 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-28T12:58:01.991 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T12:58:02.127 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-28T12:58:02.192 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T12:58:02.336 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-28T12:58:02.353 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T12:58:02.559 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T12:58:02.622 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-28T12:58:02.637 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T12:58:02.854 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T12:58:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-28T12:58:03.058 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-28T12:58:03.096 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T12:58:03.276 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T12:58:03.279 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-28T12:58:03.356 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T12:58:03.446 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-28T12:58:03.583 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T12:58:03.610 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-28T12:58:03.789 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T12:58:03.912 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-28T12:58:03.950 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T12:58:04.163 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T12:58:04.170 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-28T12:58:04.362 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T12:58:04.394 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-28T12:58:04.521 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T12:58:04.613 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-28T12:58:04.734 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T12:58:04.797 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-28T12:58:04.933 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T12:58:04.957 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-28T12:58:05.131 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T12:58:05.185 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T12:58:05.321 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T12:58:05.411 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-28T12:58:05.610 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T12:58:05.635 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-28T12:58:05.790 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T12:58:05.854 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T12:58:05.862 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T12:58:06.060 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-28T12:58:06.133 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T12:58:06.232 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-28T12:58:06.372 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T12:58:06.397 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T12:58:06.575 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T12:58:06.703 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T12:58:06.841 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T12:58:06.950 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T12:58:07.071 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T12:58:07.143 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T12:58:07.235 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T12:58:07.382 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T12:58:07.390 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T12:58:07.558 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T12:58:07.572 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T12:58:07.767 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T12:58:07.770 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T12:58:07.949 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T12:58:08.123 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T12:58:08.133 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T12:58:08.295 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T12:58:08.416 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T12:58:08.517 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T12:58:08.622 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T12:58:08.726 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T12:58:08.868 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T12:58:08.911 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T12:58:09.016 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T12:58:09.176 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T12:58:09.219 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T12:58:09.414 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T12:58:09.535 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T12:58:09.588 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T12:58:09.796 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T12:58:09.806 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T12:58:09.970 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T12:58:10.168 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T12:58:10.374 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T12:58:10.627 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T12:58:10.834 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T12:58:10.868 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T12:58:11.028 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T12:58:11.117 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T12:58:11.189 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T12:58:11.338 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T12:58:11.439 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T12:58:11.637 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T12:58:11.662 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T12:58:11.855 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T12:58:11.920 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T12:58:12.100 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T12:58:12.147 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T12:58:12.415 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T12:58:12.599 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T12:58:12.860 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T12:58:12.880 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T12:58:13.063 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:13.140 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T12:58:14.000 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T12:58:14.286 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T12:58:14.532 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T12:58:14.886 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T12:58:15.173 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T12:58:15.418 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T12:58:16.200 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T12:58:16.407 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:16.478 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:16.641 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T12:58:16.819 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:17.046 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:17.629 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:18.174 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:18.259 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:18.412 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:20.242 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:20.640 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:21.241 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:21.553 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:21.637 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:21.772 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:37.023 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:37.023 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2023-11-28T12:58:37.023 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2023-11-28T12:58:37.023 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-11-28T12:58:37.023 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /run 2023-11-28T12:58:37.024 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2023-11-28T12:58:37.024 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2023-11-28T12:58:37.024 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2023-11-28T12:58:37.024 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2023-11-28T12:58:37.024 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-11-28T12:58:37.024 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:58:37.351 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:38.926 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:40.753 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:40.753 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2023-11-28T12:58:40.753 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2023-11-28T12:58:40.753 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-11-28T12:58:40.753 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /run 2023-11-28T12:58:40.753 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2023-11-28T12:58:40.754 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2023-11-28T12:58:40.754 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2023-11-28T12:58:40.754 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2023-11-28T12:58:40.754 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /dev 2023-11-28T12:58:40.754 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:58:40.755 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:40.756 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:40.756 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:40.756 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:40.756 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:40.756 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:41.003 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:41.841 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:42.006 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:42.131 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T12:58:42.314 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T12:58:42.483 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:42.587 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:42.700 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:42.782 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:43.000 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:43.018 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:43.178 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:43.637 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:43.894 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:43.912 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:43.981 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:44.133 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:44.223 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:44.605 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T12:58:44.802 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:45.008 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:45.019 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:45.585 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:45.768 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:45.908 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T12:58:46.116 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T12:58:46.272 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T12:58:46.352 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:46.496 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:46.529 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:46.570 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:46.683 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:46.724 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:46.883 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:46.906 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:47.228 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:47.383 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:58:47.874 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:48.043 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:48.134 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:48.517 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T12:58:48.698 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:48.734 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:48.943 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:48.955 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:50.210 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T12:58:50.472 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:50.507 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:50.660 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:50.702 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:50.860 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:51.207 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:51.393 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:58:51.806 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:58:52.101 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T12:58:52.256 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T12:58:52.448 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T12:58:53.341 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-28T12:58:53.687 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-28T12:58:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 119/134 2023-11-28T12:58:54.068 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T12:58:54.252 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T12:58:54.407 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T12:58:55.833 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:58:56.561 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:58:56.717 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T12:58:56.800 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:58:56.818 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:58:56.986 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T12:58:57.082 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T12:58:57.264 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T12:58:57.279 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T12:58:57.529 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T12:58:57.556 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T12:58:57.700 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T12:58:57.849 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T12:58:57.893 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T12:58:58.014 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T12:58:58.022 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T12:58:58.129 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T12:58:58.188 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-28T12:58:58.334 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T12:58:58.338 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T12:58:58.418 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T12:58:58.476 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T12:58:58.622 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T12:58:58.634 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T12:58:58.634 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:58:58.825 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:58:58.825 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:58:59.060 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:58:59.229 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:58:59.744 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T12:58:59.745 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T12:58:59.746 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T12:58:59.755 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T12:58:59.755 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T12:58:59.755 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T12:58:59.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T12:58:59.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T12:58:59.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-28T12:58:59.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-28T12:58:59.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-28T12:58:59.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-28T12:58:59.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-28T12:58:59.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:58:59.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T12:59:00.662 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:00.917 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:00.934 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:01.103 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T12:59:01.433 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T12:59:01.727 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T12:59:02.092 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T12:59:02.266 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T12:59:02.367 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T12:59:02.368 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:02.368 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:52 ago on Tue 28 Nov 2023 12:57:10 PM UTC. 2023-11-28T12:59:02.566 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T12:59:02.579 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T12:59:02.750 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T12:59:02.757 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T12:59:02.876 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T12:59:02.877 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:03.102 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:03.103 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:59:03.321 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:59:03.540 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:04.043 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:04.043 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T12:59:04.043 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T12:59:04.043 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T12:59:04.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T12:59:04.045 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T12:59:04.046 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T12:59:04.046 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T12:59:04.046 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T12:59:04.046 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T12:59:04.046 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T12:59:04.055 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T12:59:04.055 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T12:59:04.055 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T12:59:04.055 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T12:59:04.055 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T12:59:04.055 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T12:59:04.055 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T12:59:04.056 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T12:59:04.057 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T12:59:04.058 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T12:59:04.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T12:59:04.060 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T12:59:04.061 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T12:59:04.062 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:04.063 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T12:59:06.847 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:06.847 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:55 ago on Tue 28 Nov 2023 12:57:11 PM UTC. 2023-11-28T12:59:07.318 INFO:teuthology.orchestra.run.smithi177.stdout:Installed products updated. 2023-11-28T12:59:07.544 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:07.544 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2023-11-28T12:59:07.544 INFO:teuthology.orchestra.run.smithi177.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T12:59:07.544 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.545 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.546 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-3.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T12:59:07.547 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.548 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T12:59:07.549 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T12:59:07.550 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T12:59:07.551 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T12:59:07.551 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T12:59:07.551 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.551 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.551 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.551 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T12:59:07.551 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T12:59:07.552 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T12:59:07.553 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T12:59:07.554 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.555 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:07.556 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T12:59:07.556 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T12:59:07.556 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T12:59:07.556 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T12:59:07.556 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T12:59:07.556 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T12:59:07.556 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:07.557 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-11-28T12:59:07.725 DEBUG:teuthology.parallel:result is None 2023-11-28T12:59:13.745 INFO:teuthology.orchestra.run.smithi153.stdout:Installed products updated. 2023-11-28T12:59:13.962 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:13.962 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2023-11-28T12:59:13.962 INFO:teuthology.orchestra.run.smithi153.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T12:59:13.962 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.962 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.963 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.964 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T12:59:13.965 INFO:teuthology.orchestra.run.smithi153.stdout: jq-1.6-3.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T12:59:13.966 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T12:59:13.967 INFO:teuthology.orchestra.run.smithi153.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:13.968 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T12:59:13.969 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T12:59:13.970 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T12:59:13.971 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.972 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T12:59:13.973 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T12:59:13.974 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T12:59:13.975 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T12:59:13.975 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:13.975 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2023-11-28T12:59:14.160 DEBUG:teuthology.parallel:result is None 2023-11-28T12:59:14.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:59:14.304 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T12:59:14.327 INFO:teuthology.orchestra.run.smithi153.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:14.328 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:14.328 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T12:59:14.329 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:59:14.552 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T12:59:14.581 INFO:teuthology.orchestra.run.smithi177.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:14.582 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:14.582 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T12:59:14.582 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T12:59:14.583 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:14.583 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T12:59:14.624 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:14.624 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T12:59:14.671 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T12:59:14.671 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:14.671 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T12:59:14.698 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T12:59:14.764 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:14.764 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T12:59:14.793 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T12:59:14.862 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T12:59:14.862 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:14.862 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T12:59:14.888 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T12:59:14.954 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:14.954 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T12:59:14.982 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T12:59:15.049 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T12:59:15.049 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:15.049 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T12:59:15.076 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T12:59:15.142 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:15.143 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T12:59:15.170 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T12:59:15.237 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T12:59:15.304 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T12:59:15.305 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /var/log/ceph 2023-11-28T12:59:15.307 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2023-11-28T12:59:15.332 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T12:59:15.333 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T12:59:15.374 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T12:59:15.399 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T12:59:15.399 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T12:59:15.441 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T12:59:15.475 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T12:59:15.475 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T12:59:15.475 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T12:59:15.476 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:15.476 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T12:59:15.522 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:15.665 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:15.692 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:15.692 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T12:59:15.708 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:15.839 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:15.874 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T12:59:15.874 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T12:59:15.874 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469972', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': '3ae99413970476692dfab1fe38dadb056700a61d'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6754, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '3ae99413970476692dfab1fe38dadb056700a61d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi153.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_12:38:30', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T12:59:15.875 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T12:59:15.877 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T12:59:15.922 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T12:59:15.936 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T12:59:15.987 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:15.987 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T12:59:16.017 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T12:59:16.017 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2023-11-28T12:59:16.074 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T12:59:16.075 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:16.075 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 389802 Links: 1 2023-11-28T12:59:16.075 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:16.075 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:16.075 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-11-28 12:59:03.746299209 +0000 2023-11-28T12:59:16.076 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-11-28 12:56:22.790700589 +0000 2023-11-28T12:59:16.076 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-11-28 12:56:22.790700589 +0000 2023-11-28T12:59:16.076 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-11-28T12:59:16.076 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T12:59:16.139 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-11-28T12:59:16.140 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-11-28T12:59:16.140 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000237388 s, 2.2 MB/s 2023-11-28T12:59:16.141 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T12:59:16.198 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2023-11-28T12:59:16.255 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T12:59:16.255 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:16.255 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 394256 Links: 1 2023-11-28T12:59:16.255 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:16.255 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:16.256 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-11-28 12:59:03.746299209 +0000 2023-11-28T12:59:16.256 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-11-28 12:56:23.318691279 +0000 2023-11-28T12:59:16.256 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-11-28 12:56:23.318691279 +0000 2023-11-28T12:59:16.256 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-11-28T12:59:16.256 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T12:59:16.321 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-11-28T12:59:16.321 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-11-28T12:59:16.321 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000200257 s, 2.6 MB/s 2023-11-28T12:59:16.322 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T12:59:16.380 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 386809 Links: 1 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-11-28 12:59:03.746299209 +0000 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-11-28 12:56:23.835682163 +0000 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-11-28 12:56:23.835682163 +0000 2023-11-28T12:59:16.437 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-11-28T12:59:16.438 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T12:59:16.502 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-11-28T12:59:16.502 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-11-28T12:59:16.502 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000208611 s, 2.5 MB/s 2023-11-28T12:59:16.503 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T12:59:16.561 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2023-11-28T12:59:16.618 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T12:59:16.618 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:16.618 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 6h/6d Inode: 391678 Links: 1 2023-11-28T12:59:16.618 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:16.618 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:16.618 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2023-11-28 12:59:03.746299209 +0000 2023-11-28T12:59:16.619 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2023-11-28 12:56:24.337673313 +0000 2023-11-28T12:59:16.619 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2023-11-28 12:56:24.337673313 +0000 2023-11-28T12:59:16.619 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2023-11-28T12:59:16.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T12:59:16.685 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2023-11-28T12:59:16.686 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2023-11-28T12:59:16.686 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.00022654 s, 2.3 MB/s 2023-11-28T12:59:16.686 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T12:59:16.743 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:16.743 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:16.743 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T12:59:16.762 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T12:59:16.762 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2023-11-28T12:59:16.821 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T12:59:16.821 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:16.821 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 382704 Links: 1 2023-11-28T12:59:16.821 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:16.821 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:16.821 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 12:58:59.432644780 +0000 2023-11-28T12:59:16.822 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 12:56:20.016191077 +0000 2023-11-28T12:59:16.822 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 12:56:20.016191077 +0000 2023-11-28T12:59:16.822 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T12:59:16.822 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T12:59:16.888 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T12:59:16.889 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T12:59:16.889 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000273153 s, 1.9 MB/s 2023-11-28T12:59:16.890 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T12:59:16.949 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2023-11-28T12:59:17.007 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 389353 Links: 1 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 12:58:59.432644780 +0000 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 12:56:20.533176206 +0000 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 12:56:20.533176206 +0000 2023-11-28T12:59:17.008 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T12:59:17.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T12:59:17.072 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T12:59:17.073 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T12:59:17.073 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000197914 s, 2.6 MB/s 2023-11-28T12:59:17.076 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T12:59:17.132 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2023-11-28T12:59:17.189 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T12:59:17.189 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:17.189 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 391501 Links: 1 2023-11-28T12:59:17.190 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:17.190 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:17.190 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 12:58:59.432644780 +0000 2023-11-28T12:59:17.190 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 12:56:21.038161680 +0000 2023-11-28T12:59:17.190 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 12:56:21.038161680 +0000 2023-11-28T12:59:17.190 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T12:59:17.190 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T12:59:17.254 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T12:59:17.254 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T12:59:17.254 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000240567 s, 2.1 MB/s 2023-11-28T12:59:17.255 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T12:59:17.312 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2023-11-28T12:59:17.369 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T12:59:17.369 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:17.369 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 391556 Links: 1 2023-11-28T12:59:17.369 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:17.370 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:17.370 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 12:58:59.432644780 +0000 2023-11-28T12:59:17.370 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 12:56:21.552146895 +0000 2023-11-28T12:59:17.370 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 12:56:21.552146895 +0000 2023-11-28T12:59:17.370 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T12:59:17.370 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T12:59:17.434 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T12:59:17.434 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T12:59:17.434 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000197271 s, 2.6 MB/s 2023-11-28T12:59:17.435 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T12:59:17.493 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:17.493 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T12:59:17.493 INFO:tasks.ceph:Generating config... 2023-11-28T12:59:17.494 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T12:59:17.494 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T12:59:17.494 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T12:59:17.495 INFO:tasks.ceph:[client] pid file = 2023-11-28T12:59:17.495 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T12:59:17.495 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T12:59:17.495 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T12:59:17.495 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T12:59:17.495 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T12:59:17.495 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T12:59:17.495 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T12:59:17.495 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T12:59:17.495 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T12:59:17.496 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T12:59:17.496 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-28T12:59:17.496 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-28T12:59:17.496 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T12:59:17.497 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T12:59:17.498 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T12:59:17.498 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T12:59:17.578 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T12:59:17.581 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T12:59:17.630 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:59:17.658 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.153'), ('mon.b', '172.21.15.177'), ('mon.c', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]')] 2023-11-28T12:59:17.659 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.153,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T12:59:17.659 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:17.660 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T12:59:17.715 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.153 --add b 172.21.15.177 --addv c '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:17.805 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:17.805 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool: generated fsid 47bfaa00-a151-4537-aa4c-32b0866eb248 2023-11-28T12:59:17.805 INFO:teuthology.orchestra.run.smithi153.stdout:setting min_mon_release = pacific 2023-11-28T12:59:17.805 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 0 2023-11-28T12:59:17.805 INFO:teuthology.orchestra.run.smithi153.stdout:fsid 47bfaa00-a151-4537-aa4c-32b0866eb248 2023-11-28T12:59:17.805 INFO:teuthology.orchestra.run.smithi153.stdout:last_changed 2023-11-28T12:59:17.805196+0000 2023-11-28T12:59:17.805 INFO:teuthology.orchestra.run.smithi153.stdout:created 2023-11-28T12:59:17.805196+0000 2023-11-28T12:59:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:min_mon_release 16 (pacific) 2023-11-28T12:59:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:election_strategy: 1 2023-11-28T12:59:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:0: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.a 2023-11-28T12:59:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2023-11-28T12:59:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:2: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.c 2023-11-28T12:59:17.806 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T12:59:17.807 DEBUG:teuthology.orchestra.run.smithi153:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T12:59:17.824 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 47bfaa00-a151-4537-aa4c-32b0866eb248... 2023-11-28T12:59:17.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T12:59:17.867 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout: chdir = "" 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout: auth supported = cephx 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout: filestore xattr use omap = true 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.910 INFO:teuthology.orchestra.run.smithi153.stdout: mon clock drift allowed = 1.000 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: osd crush chooseleaf type = 0 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: auth debug = true 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on old message = true 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on bug = true 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: mon pg warn max object skew = 0 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T12:59:17.911 INFO:teuthology.orchestra.run.smithi153.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default size = 2 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow primary affinity = true 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd allow pg remap = true 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on legacy crush tunables = false 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on crush straw calc version zero = false 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on no sortbitwise = false 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on osd down out interval zero = false 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon warn on too few osds = false 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T12:59:17.912 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: mon_allow_pool_size_one = true 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: osd default data pool replay window = 5 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: mon allow pool delete = true 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: mon cluster log file level = debug 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: debug asserts on shutdown = true 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: mon health detail to clog = false 2023-11-28T12:59:17.913 INFO:teuthology.orchestra.run.smithi153.stdout: mon host = "172.21.15.153,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: ms die on skipped message = False 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = 47bfaa00-a151-4537-aa4c-32b0866eb248 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout:[osd] 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: osd journal size = 100 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub load threshold = 5.0 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: osd scrub max interval = 600 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: osd recover clone overlap = true 2023-11-28T12:59:17.914 INFO:teuthology.orchestra.run.smithi153.stdout: osd recovery max chunk = 1048576 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug shutdown = true 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug op order = true 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug verify stray on activate = true 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: osd open classes on start = true 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug pg log writeout = true 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: osd deep scrub update digest min age = 30 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: osd map max advance = 10 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.915 INFO:teuthology.orchestra.run.smithi153.stdout: journal zero on create = true 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: filestore ondisk finisher threads = 3 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: filestore apply finisher threads = 3 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: bdev debug aio = true 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: osd debug misdirected ops = true 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: bdev async discard = True 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: bdev enable discard = True 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore allocator = bitmap 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore block size = 96636764160 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: bluestore fsck on mount = True 2023-11-28T12:59:17.916 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluefs = 1/20 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: debug bluestore = 1/20 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: debug osd = 20 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: debug rocksdb = 4/10 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd full ratio = 0.9 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd nearfull ratio = 0.8 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: osd failsafe full ratio = 0.95 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: osd objectstore = bluestore 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.917 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr] 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: debug mgr = 20 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: mgr/telemetry/nag = false 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout:[mon] 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: debug mon = 20 2023-11-28T12:59:17.918 INFO:teuthology.orchestra.run.smithi153.stdout: debug paxos = 20 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: debug auth = 20 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: mon data avail warn = 5 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: mon mgr mkfs grace = 240 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd reporter subtree level = osd 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: mon osd prime pg temp = true 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T12:59:17.919 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: # 1m isn't quite enough 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: mon_down_mkfs_grace = 2m 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: mon_warn_on_filestore_osds = false 2023-11-28T12:59:17.920 INFO:teuthology.orchestra.run.smithi153.stdout: mon op complaint time = 120 2023-11-28T12:59:17.921 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:17.921 INFO:teuthology.orchestra.run.smithi153.stdout:[client] 2023-11-28T12:59:17.921 INFO:teuthology.orchestra.run.smithi153.stdout: rgw cache enabled = true 2023-11-28T12:59:17.921 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable ops log = true 2023-11-28T12:59:17.921 INFO:teuthology.orchestra.run.smithi153.stdout: rgw enable usage log = true 2023-11-28T12:59:17.922 INFO:teuthology.orchestra.run.smithi153.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T12:59:17.922 INFO:teuthology.orchestra.run.smithi153.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T12:59:17.922 INFO:teuthology.orchestra.run.smithi153.stdout: client mount timeout = 600 2023-11-28T12:59:17.922 INFO:teuthology.orchestra.run.smithi153.stdout: debug client = 20 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout: pid file = "" 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.a] 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.b] 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout:[mon.c] 2023-11-28T12:59:17.923 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout:[mds] 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds = 20 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: debug mds balancer = 20 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: debug ms = 1 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug frag = True 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: mds debug scatterstat = True 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: mds op complaint time = 180 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: mds verify scatter = True 2023-11-28T12:59:17.924 INFO:teuthology.orchestra.run.smithi153.stdout: osd op complaint time = 180 2023-11-28T12:59:17.925 INFO:teuthology.orchestra.run.smithi153.stdout: rados mon op timeout = 900 2023-11-28T12:59:17.925 INFO:teuthology.orchestra.run.smithi153.stdout: rados osd op timeout = 900 2023-11-28T12:59:17.926 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2023-11-28T12:59:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2023-11-28T12:59:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2023-11-28T12:59:17.927 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2023-11-28T12:59:17.928 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.929 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.153,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 47bfaa00-a151-4537-aa4c-32b0866eb248 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.930 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.931 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2023-11-28T12:59:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2023-11-28T12:59:17.933 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2023-11-28T12:59:17.934 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2023-11-28T12:59:17.935 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T12:59:17.936 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2023-11-28T12:59:17.937 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-11-28T12:59:17.938 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T12:59:17.939 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2023-11-28T12:59:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2023-11-28T12:59:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2023-11-28T12:59:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2023-11-28T12:59:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-11-28T12:59:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-11-28T12:59:17.940 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-11-28T12:59:17.940 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T12:59:17.940 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T12:59:18.025 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T12:59:18.025 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:18.025 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T12:59:18.041 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:18.041 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T12:59:18.098 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T12:59:18.098 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:18.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T12:59:18.098 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:59:18.174 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:18.174 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:18.231 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T12:59:18.231 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:18.231 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T12:59:18.231 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:59:18.270 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:18.270 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:18.327 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T12:59:18.327 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T12:59:18.328 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T12:59:18.381 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T12:59:18.383 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T12:59:18.481 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T12:59:18.483 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T12:59:18.484 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T12:59:18.537 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T12:59:18.539 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T12:59:18.566 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T12:59:18.659 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T12:59:18.662 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T12:59:18.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T12:59:18.744 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T12:59:18.746 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T12:59:18.774 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T12:59:18.867 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T12:59:18.870 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T12:59:18.895 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T12:59:18.896 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T12:59:18.939 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T12:59:18.952 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T12:59:18.995 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T12:59:19.008 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T12:59:19.051 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T12:59:19.063 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T12:59:19.146 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T12:59:19.159 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T12:59:19.159 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T12:59:19.159 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:19.186 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:19.187 INFO:tasks.ceph:role: osd.0 2023-11-28T12:59:19.187 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T12:59:19.187 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T12:59:19.260 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:19.261 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:19.261 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:19.261 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:19.261 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:19.261 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:19.261 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:19.262 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:19.262 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:19.262 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:19.708 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-11-28T12:59:19.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-11-28T12:59:19.710 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:19.875 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:19.908 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:19.975 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:19.975 INFO:tasks.ceph:role: osd.1 2023-11-28T12:59:19.975 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T12:59:19.976 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T12:59:20.049 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:20.049 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:20.049 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:20.050 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:20.050 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:20.050 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:20.050 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:20.050 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:20.050 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:20.050 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:20.485 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-11-28T12:59:20.487 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-11-28T12:59:20.487 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:20.603 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:20.634 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:20.701 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:20.701 INFO:tasks.ceph:role: osd.2 2023-11-28T12:59:20.701 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T12:59:20.701 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T12:59:20.775 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:20.775 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:20.775 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:20.775 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:20.775 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:20.775 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:20.775 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:20.776 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:20.776 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:20.776 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:21.228 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-11-28T12:59:21.230 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-11-28T12:59:21.230 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:21.391 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:21.424 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:21.492 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:21.492 INFO:tasks.ceph:role: osd.3 2023-11-28T12:59:21.492 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T12:59:21.492 DEBUG:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T12:59:21.566 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:21.567 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:22.010 INFO:teuthology.orchestra.run.smithi153.stdout:Discarding blocks...Done. 2023-11-28T12:59:22.012 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2023-11-28T12:59:22.012 DEBUG:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:22.200 DEBUG:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:22.237 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:22.641+0000 7f0a1b73f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T12:59:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:22.642+0000 7f0a1b73f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T12:59:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:22.642+0000 7f0a1b73f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:59:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:22.642+0000 7f0a1b73f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:59:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:22.642+0000 7f0a1b73f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:59:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:22.642+0000 7f0a1b73f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T12:59:25.511 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:25.543 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:25.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:25.631+0000 7fd172c36640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T12:59:25.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:25.632+0000 7fd172c36640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T12:59:25.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:25.632+0000 7fd172c36640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:59:25.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:25.632+0000 7fd172c36640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:59:25.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:25.632+0000 7fd172c36640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:59:25.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:25.632+0000 7fd172c36640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T12:59:28.505 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:28.536 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:28.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:28.626+0000 7f62b8c8a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T12:59:28.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:28.626+0000 7f62b8c8a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T12:59:28.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:28.626+0000 7f62b8c8a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:59:28.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:28.626+0000 7f62b8c8a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:59:28.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:28.626+0000 7f62b8c8a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:59:28.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:28.626+0000 7f62b8c8a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T12:59:31.496 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:31.527 DEBUG:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:31.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:31.619+0000 7fbead396640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T12:59:31.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:31.619+0000 7fbead396640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T12:59:31.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:31.619+0000 7fbead396640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:59:31.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:31.619+0000 7fbead396640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:59:31.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:31.619+0000 7fbead396640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:59:31.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:31.620+0000 7fbead396640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T12:59:34.481 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:34.511 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:34.545 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:34.545 INFO:tasks.ceph:role: osd.4 2023-11-28T12:59:34.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T12:59:34.546 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T12:59:34.623 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:34.623 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:34.624 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:35.050 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T12:59:35.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T12:59:35.052 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:35.208 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:35.244 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:35.314 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:35.314 INFO:tasks.ceph:role: osd.5 2023-11-28T12:59:35.314 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T12:59:35.314 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T12:59:35.390 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:35.390 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:35.390 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:35.391 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:35.391 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:35.391 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:35.391 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:35.391 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:35.391 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:35.391 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:35.811 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T12:59:35.813 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T12:59:35.813 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:35.922 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:35.956 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:36.025 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:36.025 INFO:tasks.ceph:role: osd.6 2023-11-28T12:59:36.026 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T12:59:36.026 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T12:59:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:36.102 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:36.102 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:36.103 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:36.103 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:36.546 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T12:59:36.550 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T12:59:36.550 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:36.695 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:36.731 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T12:59:36.802 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:36.802 INFO:tasks.ceph:role: osd.7 2023-11-28T12:59:36.802 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T12:59:36.802 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T12:59:36.879 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:36.881 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:36.882 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:36.882 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:36.882 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:36.883 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:36.883 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:36.883 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:36.883 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:36.883 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:37.314 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T12:59:37.316 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T12:59:37.316 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T12:59:37.486 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T12:59:37.524 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:37.628 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:37.625+0000 7f69b7b4b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T12:59:37.628 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:37.625+0000 7f69b7b4b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T12:59:37.629 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:37.625+0000 7f69b7b4b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:59:37.629 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:37.625+0000 7f69b7b4b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:59:37.629 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:37.625+0000 7f69b7b4b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:59:37.629 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:37.626+0000 7f69b7b4b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T12:59:40.496 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:40.527 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:40.618+0000 7fdd76ceb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T12:59:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:40.618+0000 7fdd76ceb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T12:59:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:40.618+0000 7fdd76ceb640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:59:40.619 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:40.618+0000 7fdd76ceb640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:59:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:40.618+0000 7fdd76ceb640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:59:40.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:40.619+0000 7fdd76ceb640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T12:59:43.488 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:43.519 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:43.612 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:43.611+0000 7f47887ef640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T12:59:43.612 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:43.611+0000 7f47887ef640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T12:59:43.612 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:43.611+0000 7f47887ef640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:59:43.612 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:43.611+0000 7f47887ef640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:59:43.613 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:43.611+0000 7f47887ef640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:59:43.613 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:43.612+0000 7f47887ef640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T12:59:45.877 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:45.880 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:46.491 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:46.523 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:46.616 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:46.614+0000 7f3ce2fb8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T12:59:46.617 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:46.614+0000 7f3ce2fb8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T12:59:46.617 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:46.614+0000 7f3ce2fb8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:59:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:46.614+0000 7f3ce2fb8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:59:46.618 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:46.614+0000 7f3ce2fb8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:59:46.619 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:46.615+0000 7f3ce2fb8640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T12:59:49.473 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T12:59:49.503 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T12:59:49.503 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:49.504 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T12:59:49.535 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:49.535 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T12:59:49.606 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:49.606 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T12:59:49.675 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:49.675 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T12:59:49.743 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:49.743 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T12:59:49.811 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:49.812 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T12:59:49.879 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:49.879 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T12:59:49.946 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:49.946 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T12:59:49.977 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:49.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T12:59:50.047 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:50.047 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T12:59:50.119 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:50.119 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T12:59:50.186 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:50.187 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T12:59:50.254 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:50.255 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T12:59:50.322 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:50.322 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T12:59:50.389 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:50.389 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T12:59:50.409 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:50.409 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T12:59:50.468 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:50.468 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T12:59:50.485 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:50.485 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T12:59:50.542 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T12:59:50.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T12:59:50.544 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T12:59:50.585 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.y] 2023-11-28T12:59:50.585 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVld7eyFhAAafZalFXR/4jRd5fUct7T2Q== 2023-11-28T12:59:50.585 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.a] 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVlYBAHIBAA4jaMav4CAatojxXY6AkGPw== 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.c] 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVlG5hJJxAAtFV9HSiLJClNn3nmB7ORSg== 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.0] 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAq5GVlatZbJhAA2RQenckwVd7P0kBpzW08Lg== 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.1] 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAt5GVlf73BJRAA51B4aDKKKVO+yo0+jQ20Yw== 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.2] 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAw5GVl3M9qJRAAa2Jhr7Cp6vN/fRKJpMzCOg== 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.3] 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAz5GVlWVgHJRAAlP6Gc1QZkzufchXRrkLkbA== 2023-11-28T12:59:50.586 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.x] 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVlKLmuHBAANErUDgd4g3Yysqk/sPAbrA== 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.b] 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVl9vBaLBAAD84m+iSSScY6ETZRBpBUEg== 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.d] 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVlvAC6MxAAX0N4lN+NKADdjjjROw+Zbw== 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.4] 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQA55GVlLb5hJRAAMMymIqHNw8oeVW8s3BrtIg== 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.5] 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQA85GVlwsLjJBAAb/1bWARS8ILpFdBGemlcoA== 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.6] 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQA/5GVlwXx5JBAAXOS8JqPaXyQ7sj+2t9tsPQ== 2023-11-28T12:59:50.587 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.7] 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQBC5GVl5q+qJBAAMc7ZUInNCMJmf0lC22s1QQ== 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVlR2IBOBAAJBEAQ3PnTN6KaU44zoHW7Q== 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout:[client.2] 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAm5GVlsT5YOxAAZ6ep21AlkI2znaE9wyskxA== 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout:[client.1] 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAn5GVlTvYJAxAA68QZff0NcXXeMV7aGZoBfQ== 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout:[client.3] 2023-11-28T12:59:50.588 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQAn5GVlxW63CBAApiPmiN4tN3S2dZH/bCKS/Q== 2023-11-28T12:59:50.607 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2023-11-28T12:59:50.607 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVld7eyFhAAafZalFXR/4jRd5fUct7T2Q== 2023-11-28T12:59:50.607 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-11-28T12:59:50.607 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVlYBAHIBAA4jaMav4CAatojxXY6AkGPw== 2023-11-28T12:59:50.607 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-11-28T12:59:50.607 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVlG5hJJxAAtFV9HSiLJClNn3nmB7ORSg== 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAq5GVlatZbJhAA2RQenckwVd7P0kBpzW08Lg== 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAt5GVlf73BJRAA51B4aDKKKVO+yo0+jQ20Yw== 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAw5GVl3M9qJRAAa2Jhr7Cp6vN/fRKJpMzCOg== 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAz5GVlWVgHJRAAlP6Gc1QZkzufchXRrkLkbA== 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVlKLmuHBAANErUDgd4g3Yysqk/sPAbrA== 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-11-28T12:59:50.608 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVl9vBaLBAAD84m+iSSScY6ETZRBpBUEg== 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVlvAC6MxAAX0N4lN+NKADdjjjROw+Zbw== 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQA55GVlLb5hJRAAMMymIqHNw8oeVW8s3BrtIg== 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQA85GVlwsLjJBAAb/1bWARS8ILpFdBGemlcoA== 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQA/5GVlwXx5JBAAXOS8JqPaXyQ7sj+2t9tsPQ== 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBC5GVl5q+qJBAAMc7ZUInNCMJmf0lC22s1QQ== 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2023-11-28T12:59:50.609 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVlR2IBOBAAJBEAQ3PnTN6KaU44zoHW7Q== 2023-11-28T12:59:50.610 INFO:teuthology.orchestra.run.smithi177.stdout:[client.2] 2023-11-28T12:59:50.610 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAm5GVlsT5YOxAAZ6ep21AlkI2znaE9wyskxA== 2023-11-28T12:59:50.610 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2023-11-28T12:59:50.610 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAn5GVlTvYJAxAA68QZff0NcXXeMV7aGZoBfQ== 2023-11-28T12:59:50.610 INFO:teuthology.orchestra.run.smithi177.stdout:[client.3] 2023-11-28T12:59:50.610 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAn5GVlxW63CBAApiPmiN4tN3S2dZH/bCKS/Q== 2023-11-28T12:59:50.611 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:59:50.632 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:59:50.704 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:50.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:50.762 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:50.764 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:50.819 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:50.822 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:50.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:50.877 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:50.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:50.931 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:50.984 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:50.986 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:59:51.041 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T12:59:51.094 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:51.096 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:51.151 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:51.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T12:59:51.212 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.214 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.269 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.323 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.325 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.379 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T12:59:51.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.434 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.490 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.493 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.548 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.550 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.606 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.608 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T12:59:51.664 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T12:59:51.664 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T12:59:51.691 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T12:59:51.788 DEBUG:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T12:59:51.815 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T12:59:51.841 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T12:59:51.939 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T12:59:51.966 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T12:59:52.033 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T12:59:52.132 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T12:59:52.158 DEBUG:teuthology.orchestra.run.smithi153:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:52.160 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:52.215 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T12:59:52.215 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T12:59:52.215 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T12:59:52.217 INFO:tasks.ceph.mon.a:Started 2023-11-28T12:59:52.217 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T12:59:52.218 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T12:59:52.258 INFO:tasks.ceph.mon.b:Started 2023-11-28T12:59:52.258 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T12:59:52.258 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T12:59:52.260 INFO:tasks.ceph.mon.c:Started 2023-11-28T12:59:52.260 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T12:59:52.260 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T12:59:52.260 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T12:59:52.262 INFO:tasks.ceph.mgr.y:Started 2023-11-28T12:59:52.262 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T12:59:52.263 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T12:59:52.264 INFO:tasks.ceph.mgr.x:Started 2023-11-28T12:59:52.264 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T12:59:52.265 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T12:59:52.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.386+0000 7f3a713af700 1 Processor -- start 2023-11-28T12:59:52.538 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a713af700 1 -- start start 2023-11-28T12:59:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a713af700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a6c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a713af700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 0x7f3a6c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a713af700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c06b420 con 0x7f3a6c0ce040 2023-11-28T12:59:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a713af700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c0b32f0 con 0x7f3a6c0b2930 2023-11-28T12:59:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a713af700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c06ff60 con 0x7f3a6c0d0a00 2023-11-28T12:59:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a6b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 0x7f3a6c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.546 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a6affd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3a6c0ce040 0x7f3a6c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:55752/0 (socket says 172.21.15.153:55752) 2023-11-28T12:59:52.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.387+0000 7f3a6affd700 1 -- 172.21.15.153:0/597592118 learned_addr learned my addr 172.21.15.153:0/597592118 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T12:59:52.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6b7fe700 1 -- 172.21.15.153:0/597592118 >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 msgr2=0x7f3a6c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T12:59:52.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6b7fe700 1 -- 172.21.15.153:0/597592118 >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 msgr2=0x7f3a6c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T12:59:52.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6b7fe700 1 --2- 172.21.15.153:0/597592118 >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 0x7f3a6c0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T12:59:52.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a6c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6b7fe700 1 --2- 172.21.15.153:0/597592118 >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 0x7f3a6c0b2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T12:59:52.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6a7fc700 1 -- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 msgr2=0x7f3a6c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T12:59:52.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6a7fc700 1 -- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 msgr2=0x7f3a6c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T12:59:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a6c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T12:59:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.388+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a6c15df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T12:59:52.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.584+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455523522 0 0) 0x7f3a6c06b420 con 0x7f3a6c0ce040 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.584+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a540035b0 con 0x7f3a6c0ce040 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358522506 0 0) 0x7f3a540035b0 con 0x7f3a6c0ce040 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 msgr2=0x7f3a6c15df10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a6c15df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 msgr2=0x7f3a6c0b2db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 --2- 172.21.15.153:0/597592118 >> v2:172.21.15.153:3300/0 conn(0x7f3a6c0b2930 0x7f3a6c0b2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c06b700 con 0x7f3a6c0ce040 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f3a640017f0 con 0x7f3a6c0ce040 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 >> v1:172.21.15.153:6789/0 conn(0x7f3a6c0ce040 legacy=0x7f3a6c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a54003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a54004080 0x7f3a54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a54004d90 0x7f3a54005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0b32f0 con 0x7f3a54004080 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c06ff60 con 0x7f3a54004d90 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5400bb70 con 0x7f3a6c0d0a00 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a54004080 0x7f3a54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a6affd700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a54003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a6b7fe700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a54004d90 0x7f3a54005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.585+0000 7f3a6a7fc700 1 -- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 msgr2=0x7f3a54003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0d0a00 0x7f3a54003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a6a7fc700 1 -- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a54004d90 msgr2=0x7f3a54005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a54004d90 0x7f3a54005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a6a7fc700 1 -- 172.21.15.153:0/597592118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a540035b0 con 0x7f3a54004080 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a6b7fe700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a54004d90 0x7f3a54005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a54004080 0x7f3a54004850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009860 tx=0x7f3a54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d96057bd22c69d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000c070 con 0x7f3a54004080 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60004020 con 0x7f3a54004080 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.586+0000 7f3a69ffb700 1 -- 172.21.15.153:0/597592118 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600043d0 con 0x7f3a54004080 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.587+0000 7f3a713af700 1 -- 172.21.15.153:0/597592118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a54004080 msgr2=0x7f3a54004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.587+0000 7f3a713af700 1 --2- 172.21.15.153:0/597592118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a54004080 0x7f3a54004850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009860 tx=0x7f3a54005880 comp rx=0 tx=0).stop 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.587+0000 7f3a6b7fe700 1 -- 172.21.15.153:0/597592118 reap_dead start 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.587+0000 7f3a713af700 1 -- 172.21.15.153:0/597592118 shutdown_connections 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.587+0000 7f3a713af700 1 -- 172.21.15.153:0/597592118 >> 172.21.15.153:0/597592118 conn(0x7f3a6c06aed0 msgr2=0x7f3a6c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.587+0000 7f3a713af700 1 -- 172.21.15.153:0/597592118 shutdown_connections 2023-11-28T12:59:52.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.587+0000 7f3a713af700 1 -- 172.21.15.153:0/597592118 wait complete. 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.588+0000 7f3a713af700 1 Processor -- start 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.588+0000 7f3a713af700 1 -- start start 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a713af700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a54004080 0x7f3a6c14ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a713af700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c14d580 0x7f3a6c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a713af700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c14ee40 0x7f3a6c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a713af700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c14d350 con 0x7f3a6c14d580 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a713af700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c14db50 con 0x7f3a6c14ee40 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a713af700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c14e9b0 con 0x7f3a54004080 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c14d580 0x7f3a6c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a54004080 0x7f3a6c14ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c14d580 0x7f3a6c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37616/0 (socket says 172.21.15.153:37616) 2023-11-28T12:59:52.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6a7fc700 1 -- 172.21.15.153:0/2853942177 learned_addr learned my addr 172.21.15.153:0/2853942177 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6b7fe700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c14ee40 0x7f3a6c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6affd700 1 -- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c14ee40 msgr2=0x7f3a6c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6affd700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c14ee40 0x7f3a6c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6affd700 1 -- 172.21.15.153:0/2853942177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c14d580 msgr2=0x7f3a6c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6affd700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c14d580 0x7f3a6c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6affd700 1 -- 172.21.15.153:0/2853942177 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c06bcc0 con 0x7f3a54004080 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.589+0000 7f3a6a7fc700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c14d580 0x7f3a6c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:52.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.590+0000 7f3a6affd700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a54004080 0x7f3a6c14ce10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a640011d0 tx=0x7f3a6400fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:52.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.594+0000 7f3a537fe700 1 -- 172.21.15.153:0/2853942177 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640072e0 con 0x7f3a54004080 2023-11-28T12:59:52.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.594+0000 7f3a537fe700 1 -- 172.21.15.153:0/2853942177 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64007440 con 0x7f3a54004080 2023-11-28T12:59:52.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.594+0000 7f3a537fe700 1 -- 172.21.15.153:0/2853942177 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64005830 con 0x7f3a54004080 2023-11-28T12:59:52.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.594+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c06b9e0 con 0x7f3a54004080 2023-11-28T12:59:52.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.594+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c06bfa0 con 0x7f3a54004080 2023-11-28T12:59:52.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.594+0000 7f3a537fe700 1 -- 172.21.15.153:0/2853942177 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3a64005c80 con 0x7f3a54004080 2023-11-28T12:59:52.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.594+0000 7f3a537fe700 1 -- 172.21.15.153:0/2853942177 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3a6401de40 con 0x7f3a54004080 2023-11-28T12:59:52.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.596+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c0cd680 con 0x7f3a54004080 2023-11-28T12:59:52.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.602+0000 7f3a537fe700 1 -- 172.21.15.153:0/2853942177 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3a6401d500 con 0x7f3a54004080 2023-11-28T12:59:52.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.690+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3a6c068390 con 0x7f3a54004080 2023-11-28T12:59:52.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.701+0000 7f3a537fe700 1 -- 172.21.15.153:0/2853942177 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3a6401d6a0 con 0x7f3a54004080 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a54004080 msgr2=0x7f3a6c14ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a54004080 0x7f3a6c14ce10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a640011d0 tx=0x7f3a6400fb20 comp rx=0 tx=0).stop 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 shutdown_connections 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c14ee40 0x7f3a6c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a54004080 0x7f3a6c14ce10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 --2- 172.21.15.153:0/2853942177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c14d580 0x7f3a6c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 >> 172.21.15.153:0/2853942177 conn(0x7f3a6c06aed0 msgr2=0x7f3a6c15c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 shutdown_connections 2023-11-28T12:59:52.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:52.702+0000 7f3a713af700 1 -- 172.21.15.153:0/2853942177 wait complete. 2023-11-28T12:59:52.704 INFO:teuthology.orchestra.run.smithi153.stderr:adjusted tunables profile to default 2023-11-28T12:59:52.712 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:52.711+0000 7fdc580d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T12:59:52.714 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T12:59:52.714 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T12:59:52.715 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:52.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T12:59:52.715 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:52.714+0000 7ff7c362e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T12:59:52.741 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:52.741 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T12:59:52.777 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:52.776+0000 7fdc580d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T12:59:52.784 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:52.783+0000 7ff7c362e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T12:59:52.808 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:52.808 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T12:59:52.850 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:52.850+0000 7fdc580d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T12:59:52.863 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:52.863+0000 7ff7c362e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T12:59:52.874 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T12:59:52.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T12:59:52.929 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:52.928+0000 7fdc580d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T12:59:52.941 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:52.941 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T12:59:52.949 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:52.948+0000 7ff7c362e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T12:59:52.967 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:52.968 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T12:59:53.034 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:53.034 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T12:59:53.077 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:53.077+0000 7fdc580d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T12:59:53.091 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:53.091+0000 7ff7c362e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T12:59:53.099 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T12:59:53.099 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T12:59:53.166 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 63fec646-657d-4957-bcc2-76a5db288946 0 2023-11-28T12:59:53.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.324+0000 7fc6976de700 1 Processor -- start 2023-11-28T12:59:53.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.324+0000 7fc6976de700 1 -- start start 2023-11-28T12:59:53.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc6976de700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cd8c0 0x7fc690140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc6976de700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69013ff00 con 0x7fc6900cdc90 2023-11-28T12:59:53.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc6976de700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6901404c0 con 0x7fc69006f850 2023-11-28T12:59:53.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc6976de700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69015aa80 con 0x7fc6900cd8c0 2023-11-28T12:59:53.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc69547a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cd8c0 0x7fc690140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc69547a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cd8c0 0x7fc690140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58066/0 (socket says 172.21.15.177:58066) 2023-11-28T12:59:53.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc695c7b700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc69006f850 0x7fc69006fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58242/0 (socket says 172.21.15.177:58242) 2023-11-28T12:59:53.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.325+0000 7fc69547a700 1 -- 172.21.15.177:0/2464740719 learned_addr learned my addr 172.21.15.177:0/2464740719 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:53.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.326+0000 7fc69547a700 1 -- 172.21.15.177:0/2464740719 >> v1:172.21.15.177:6789/0 conn(0x7fc69006f850 legacy=0x7fc69006fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.326+0000 7fc69547a700 1 -- 172.21.15.177:0/2464740719 >> v1:172.21.15.153:6789/0 conn(0x7fc6900cdc90 legacy=0x7fc6900b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.326+0000 7fc69547a700 1 -- 172.21.15.177:0/2464740719 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6900b3040 con 0x7fc6900cd8c0 2023-11-28T12:59:53.328 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.326+0000 7fc69547a700 1 --2- 172.21.15.177:0/2464740719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cd8c0 0x7fc690140950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098f0 tx=0x7fc68c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd8c57041b46596 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.326+0000 7fc687fff700 1 -- 172.21.15.177:0/2464740719 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c005e30 con 0x7fc6900cd8c0 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.326+0000 7fc687fff700 1 -- 172.21.15.177:0/2464740719 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c004030 con 0x7fc6900cd8c0 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.326+0000 7fc687fff700 1 -- 172.21.15.177:0/2464740719 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0043e0 con 0x7fc6900cd8c0 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.327+0000 7fc6976de700 1 -- 172.21.15.177:0/2464740719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cd8c0 msgr2=0x7fc690140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.327+0000 7fc6976de700 1 --2- 172.21.15.177:0/2464740719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cd8c0 0x7fc690140950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098f0 tx=0x7fc68c009c00 comp rx=0 tx=0).stop 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.327+0000 7fc6976de700 1 -- 172.21.15.177:0/2464740719 shutdown_connections 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.327+0000 7fc6976de700 1 --2- 172.21.15.177:0/2464740719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cd8c0 0x7fc690140950 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.327+0000 7fc6976de700 1 -- 172.21.15.177:0/2464740719 >> 172.21.15.177:0/2464740719 conn(0x7fc69006aed0 msgr2=0x7fc69006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.327+0000 7fc6976de700 1 -- 172.21.15.177:0/2464740719 shutdown_connections 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.327+0000 7fc6976de700 1 -- 172.21.15.177:0/2464740719 wait complete. 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.328+0000 7fc6976de700 1 Processor -- start 2023-11-28T12:59:53.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.328+0000 7fc6976de700 1 -- start start 2023-11-28T12:59:53.334 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc6976de700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc69006f850 0x7fc6901588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.334 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc6976de700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6900cd8c0 0x7fc6900b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.334 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc6976de700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cdc90 0x7fc690143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.334 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc6976de700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69006d3a0 con 0x7fc6900cd8c0 2023-11-28T12:59:53.334 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc6976de700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6901f2090 con 0x7fc69006f850 2023-11-28T12:59:53.334 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc6976de700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6901f66e0 con 0x7fc6900cdc90 2023-11-28T12:59:53.334 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cdc90 0x7fc690143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc69547a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc69006f850 0x7fc6901588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cdc90 0x7fc690143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58076/0 (socket says 172.21.15.177:58076) 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc69547a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc69006f850 0x7fc6901588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45314/0 (socket says 172.21.15.177:45314) 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 -- 172.21.15.177:0/3159466621 learned_addr learned my addr 172.21.15.177:0/3159466621 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc694c79700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6900cd8c0 0x7fc6900b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 -- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc69006f850 msgr2=0x7fc6901588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc69006f850 0x7fc6901588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 -- 172.21.15.177:0/3159466621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6900cd8c0 msgr2=0x7fc6900b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.335 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6900cd8c0 0x7fc6900b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 -- 172.21.15.177:0/3159466621 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c009500 con 0x7fc6900cdc90 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc69547a700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc69006f850 0x7fc6901588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc694c79700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6900cd8c0 0x7fc6900b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc695c7b700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cdc90 0x7fc690143510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc68800cbd0 tx=0x7fc68800cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc685ffb700 1 -- 172.21.15.177:0/3159466621 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68800aaa0 con 0x7fc6900cdc90 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.329+0000 7fc685ffb700 1 -- 172.21.15.177:0/3159466621 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68800ac00 con 0x7fc6900cdc90 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.330+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6900b35f0 con 0x7fc6900cdc90 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.330+0000 7fc685ffb700 1 -- 172.21.15.177:0/3159466621 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688007070 con 0x7fc6900cdc90 2023-11-28T12:59:53.336 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.330+0000 7fc685ffb700 1 -- 172.21.15.177:0/3159466621 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc688019030 con 0x7fc6900cdc90 2023-11-28T12:59:53.337 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.330+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6900b3310 con 0x7fc6900cdc90 2023-11-28T12:59:53.337 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.330+0000 7fc685ffb700 1 -- 172.21.15.177:0/3159466621 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc6880075b0 con 0x7fc6900cdc90 2023-11-28T12:59:53.337 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.331+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc690140150 con 0x7fc6900cdc90 2023-11-28T12:59:53.337 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.333+0000 7fc685ffb700 1 -- 172.21.15.177:0/3159466621 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc688017070 con 0x7fc6900cdc90 2023-11-28T12:59:53.416 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.415+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "63fec646-657d-4957-bcc2-76a5db288946", "id": 0} v 0) v1 -- 0x7fc690068690 con 0x7fc6900cdc90 2023-11-28T12:59:53.422 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.421+0000 7fc685ffb700 1 -- 172.21.15.177:0/3159466621 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63fec646-657d-4957-bcc2-76a5db288946", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc688007840 con 0x7fc6900cdc90 2023-11-28T12:59:53.422 INFO:teuthology.orchestra.run.smithi177.stdout:0 2023-11-28T12:59:53.423 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cdc90 msgr2=0x7fc690143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.423 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cdc90 0x7fc690143510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc68800cbd0 tx=0x7fc68800cee0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.423 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 shutdown_connections 2023-11-28T12:59:53.423 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6900cdc90 0x7fc690143510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.424 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc69006f850 0x7fc6901588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.424 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 --2- 172.21.15.177:0/3159466621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6900cd8c0 0x7fc6900b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.424 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 >> 172.21.15.177:0/3159466621 conn(0x7fc69006aed0 msgr2=0x7fc6900b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:53.424 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 shutdown_connections 2023-11-28T12:59:53.424 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.422+0000 7fc6976de700 1 -- 172.21.15.177:0/3159466621 wait complete. 2023-11-28T12:59:53.436 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new c9553e4f-e498-4be9-bda7-52af39c42b26 1 2023-11-28T12:59:53.556 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.555+0000 7f9b31f71700 1 Processor -- start 2023-11-28T12:59:53.556 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.555+0000 7f9b31f71700 1 -- start start 2023-11-28T12:59:53.559 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b31f71700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9b2c15cef0 0x7f9b2c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b31f71700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c06ba90 0x7f9b2c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b31f71700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c0cc5f0 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b31f71700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c15d470 con 0x7f9b2c15cef0 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b31f71700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c15d750 con 0x7f9b2c06ba90 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b2b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9b2c15cef0 0x7f9b2c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b2b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9b2c15cef0 0x7f9b2c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45320/0 (socket says 172.21.15.177:45320) 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b2b7fe700 1 -- 172.21.15.177:0/1156757459 learned_addr learned my addr 172.21.15.177:0/1156757459 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.556+0000 7f9b2bfff700 1 --2- 172.21.15.177:0/1156757459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c06ba90 0x7f9b2c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2bfff700 1 -- 172.21.15.177:0/1156757459 >> v2:172.21.15.177:3300/0 conn(0x7f9b2c15cef0 msgr2=0x7f9b2c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:53.560 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2bfff700 1 --2- 172.21.15.177:0/1156757459 >> v2:172.21.15.177:3300/0 conn(0x7f9b2c15cef0 0x7f9b2c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2bfff700 1 -- 172.21.15.177:0/1156757459 >> v1:172.21.15.153:6789/0 conn(0x7f9b2c0cbdf0 legacy=0x7f9b2c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2a7fc700 1 -- 172.21.15.177:0/1156757459 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666246971 0 0) 0x7f9b2c0cc5f0 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2bfff700 1 -- 172.21.15.177:0/1156757459 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c15da30 con 0x7f9b2c06ba90 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2b7fe700 1 --2- 172.21.15.177:0/1156757459 >> v2:172.21.15.177:3300/0 conn(0x7f9b2c15cef0 0x7f9b2c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2bfff700 1 --2- 172.21.15.177:0/1156757459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c06ba90 0x7f9b2c0ca650 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009dc0 tx=0x7f9b2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea9099d9ccb726f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2a7fc700 1 -- 172.21.15.177:0/1156757459 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400e070 con 0x7f9b2c06ba90 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2a7fc700 1 -- 172.21.15.177:0/1156757459 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2400bae0 con 0x7f9b2c06ba90 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.557+0000 7f9b2a7fc700 1 -- 172.21.15.177:0/1156757459 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400be90 con 0x7f9b2c06ba90 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 -- 172.21.15.177:0/1156757459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c06ba90 msgr2=0x7f9b2c0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 --2- 172.21.15.177:0/1156757459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c06ba90 0x7f9b2c0ca650 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009dc0 tx=0x7f9b2400b330 comp rx=0 tx=0).stop 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 -- 172.21.15.177:0/1156757459 shutdown_connections 2023-11-28T12:59:53.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 --2- 172.21.15.177:0/1156757459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c06ba90 0x7f9b2c0ca650 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 --2- 172.21.15.177:0/1156757459 >> v2:172.21.15.177:3300/0 conn(0x7f9b2c15cef0 0x7f9b2c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 -- 172.21.15.177:0/1156757459 >> 172.21.15.177:0/1156757459 conn(0x7f9b2c0d0000 msgr2=0x7f9b2c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:53.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 -- 172.21.15.177:0/1156757459 shutdown_connections 2023-11-28T12:59:53.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.558+0000 7f9b31f71700 1 -- 172.21.15.177:0/1156757459 wait complete. 2023-11-28T12:59:53.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.559+0000 7f9b31f71700 1 Processor -- start 2023-11-28T12:59:53.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.559+0000 7f9b31f71700 1 -- start start 2023-11-28T12:59:53.564 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b31f71700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c06ba90 0x7f9b2c0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b31f71700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0cbdf0 0x7f9b2c156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b31f71700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b2c15cef0 0x7f9b2c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b31f71700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c158d60 con 0x7f9b2c15cef0 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b31f71700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c158430 con 0x7f9b2c06ba90 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b31f71700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c158710 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0cbdf0 0x7f9b2c156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0cbdf0 0x7f9b2c156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58084/0 (socket says 172.21.15.177:58084) 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 -- 172.21.15.177:0/2095022786 learned_addr learned my addr 172.21.15.177:0/2095022786 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2b7fe700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c06ba90 0x7f9b2c0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2bfff700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b2c15cef0 0x7f9b2c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 -- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c06ba90 msgr2=0x7f9b2c0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.565 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c06ba90 0x7f9b2c0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 -- 172.21.15.177:0/2095022786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b2c15cef0 msgr2=0x7f9b2c1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b2c15cef0 0x7f9b2c1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 -- 172.21.15.177:0/2095022786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c009550 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2b7fe700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c06ba90 0x7f9b2c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b2affd700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0cbdf0 0x7f9b2c156060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b200011e0 tx=0x7f9b2000ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b13fff700 1 -- 172.21.15.177:0/2095022786 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b200054e0 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b13fff700 1 -- 172.21.15.177:0/2095022786 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b20003680 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b24009a70 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.560+0000 7f9b13fff700 1 -- 172.21.15.177:0/2095022786 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20003a30 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.561+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c15dff0 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.561+0000 7f9b13fff700 1 -- 172.21.15.177:0/2095022786 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9b20019030 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.566 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.561+0000 7f9b13fff700 1 -- 172.21.15.177:0/2095022786 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f9b2001ce00 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.567 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.562+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c15dd10 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.567 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.564+0000 7f9b13fff700 1 -- 172.21.15.177:0/2095022786 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9b20017070 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.647 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.647+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c9553e4f-e498-4be9-bda7-52af39c42b26", "id": 1} v 0) v1 -- 0x7f9b2c0cab90 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.656 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.655+0000 7f9b13fff700 1 -- 172.21.15.177:0/2095022786 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c9553e4f-e498-4be9-bda7-52af39c42b26", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9b2001c510 con 0x7f9b2c0cbdf0 2023-11-28T12:59:53.656 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-11-28T12:59:53.657 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0cbdf0 msgr2=0x7f9b2c156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.657 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0cbdf0 0x7f9b2c156060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b200011e0 tx=0x7f9b2000ca30 comp rx=0 tx=0).stop 2023-11-28T12:59:53.657 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 shutdown_connections 2023-11-28T12:59:53.658 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b2c15cef0 0x7f9b2c1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.658 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c06ba90 0x7f9b2c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.658 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 --2- 172.21.15.177:0/2095022786 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0cbdf0 0x7f9b2c156060 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.658 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 >> 172.21.15.177:0/2095022786 conn(0x7f9b2c0d0000 msgr2=0x7f9b2c06a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:53.659 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 shutdown_connections 2023-11-28T12:59:53.659 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.656+0000 7f9b31f71700 1 -- 172.21.15.177:0/2095022786 wait complete. 2023-11-28T12:59:53.669 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 66975359-a324-4e64-a069-173dbe9e0aad 2 2023-11-28T12:59:53.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.787+0000 7f0002e58700 1 Processor -- start 2023-11-28T12:59:53.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.787+0000 7f0002e58700 1 -- start start 2023-11-28T12:59:53.791 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f0002e58700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efffc0d1330 0x7efffc0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.791 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f0002e58700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efffc0b20b0 0x7efffc0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.791 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f0002e58700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc0b0230 con 0x7efffc0d01b0 2023-11-28T12:59:53.791 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f0002e58700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc15cec0 con 0x7efffc0d1330 2023-11-28T12:59:53.791 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f0002e58700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efffc0b4570 con 0x7efffc0b20b0 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f00013f5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efffc0b20b0 0x7efffc0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f00013f5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efffc0b20b0 0x7efffc0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45334/0 (socket says 172.21.15.177:45334) 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7f00013f5700 1 -- 172.21.15.177:0/2220750021 learned_addr learned my addr 172.21.15.177:0/2220750021 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.788+0000 7efffbfff700 1 --2- 172.21.15.177:0/2220750021 >> v2:172.21.15.153:3300/0 conn(0x7efffc0d1330 0x7efffc0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7f00013f5700 1 -- 172.21.15.177:0/2220750021 >> v2:172.21.15.153:3300/0 conn(0x7efffc0d1330 msgr2=0x7efffc0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7f00013f5700 1 --2- 172.21.15.177:0/2220750021 >> v2:172.21.15.153:3300/0 conn(0x7efffc0d1330 0x7efffc0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7f00013f5700 1 -- 172.21.15.177:0/2220750021 >> v1:172.21.15.153:6789/0 conn(0x7efffc0d01b0 legacy=0x7efffc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7f00013f5700 1 -- 172.21.15.177:0/2220750021 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0b4850 con 0x7efffc0b20b0 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7f00013f5700 1 --2- 172.21.15.177:0/2220750021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0b20b0 0x7efffc0b0c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efffc066e80 tx=0x7efff400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724ea00d46a2e9e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:53.792 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7efffb7fe700 1 -- 172.21.15.177:0/2220750021 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400e070 con 0x7efffc0b20b0 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7efffb7fe700 1 -- 172.21.15.177:0/2220750021 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400b910 con 0x7efffc0b20b0 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.789+0000 7efffb7fe700 1 -- 172.21.15.177:0/2220750021 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400bcc0 con 0x7efffc0b20b0 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 -- 172.21.15.177:0/2220750021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0b20b0 msgr2=0x7efffc0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 --2- 172.21.15.177:0/2220750021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0b20b0 0x7efffc0b0c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efffc066e80 tx=0x7efff400b330 comp rx=0 tx=0).stop 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 -- 172.21.15.177:0/2220750021 shutdown_connections 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 --2- 172.21.15.177:0/2220750021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0b20b0 0x7efffc0b0c00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 --2- 172.21.15.177:0/2220750021 >> v2:172.21.15.153:3300/0 conn(0x7efffc0d1330 0x7efffc0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 -- 172.21.15.177:0/2220750021 >> 172.21.15.177:0/2220750021 conn(0x7efffc09c3f0 msgr2=0x7efffc0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 -- 172.21.15.177:0/2220750021 shutdown_connections 2023-11-28T12:59:53.793 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.790+0000 7f0002e58700 1 -- 172.21.15.177:0/2220750021 wait complete. 2023-11-28T12:59:53.794 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:53.791+0000 7fdc580d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T12:59:53.795 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:53.794+0000 7ff7c362e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T12:59:53.796 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.791+0000 7f0002e58700 1 Processor -- start 2023-11-28T12:59:53.796 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.791+0000 7f0002e58700 1 -- start start 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f0002e58700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efffc0b20b0 0x7efffc155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f0002e58700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0d01b0 0x7efffc15f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f0002e58700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efffc0d1330 0x7efffc158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f0002e58700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc15d9c0 con 0x7efffc0d1330 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f0002e58700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc15dc60 con 0x7efffc0d01b0 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f0002e58700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc15df40 con 0x7efffc0b20b0 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7efffbfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0d01b0 0x7efffc15f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7efffbfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0d01b0 0x7efffc15f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45338/0 (socket says 172.21.15.177:45338) 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7efffbfff700 1 -- 172.21.15.177:0/1714233063 learned_addr learned my addr 172.21.15.177:0/1714233063 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f00013f5700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efffc0d1330 0x7efffc158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f0000bf4700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efffc0b20b0 0x7efffc155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f00013f5700 1 -- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efffc0b20b0 msgr2=0x7efffc155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f00013f5700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efffc0b20b0 0x7efffc155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f00013f5700 1 -- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0d01b0 msgr2=0x7efffc15f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f00013f5700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0d01b0 0x7efffc15f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7f00013f5700 1 -- 172.21.15.177:0/1714233063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4009b30 con 0x7efffc0d1330 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.792+0000 7efffbfff700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0d01b0 0x7efffc15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7f00013f5700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efffc0d1330 0x7efffc158e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efff4015000 tx=0x7efff4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7efff97fa700 1 -- 172.21.15.177:0/1714233063 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4004880 con 0x7efffc0d1330 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7efff97fa700 1 -- 172.21.15.177:0/1714233063 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff40099b0 con 0x7efffc0d1330 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc0afec0 con 0x7efffc0d1330 2023-11-28T12:59:53.798 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7efff97fa700 1 -- 172.21.15.177:0/1714233063 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff401c680 con 0x7efffc0d1330 2023-11-28T12:59:53.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0afbe0 con 0x7efffc0d1330 2023-11-28T12:59:53.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7efff97fa700 1 -- 172.21.15.177:0/1714233063 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7efff4022030 con 0x7efffc0d1330 2023-11-28T12:59:53.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.793+0000 7efff97fa700 1 -- 172.21.15.177:0/1714233063 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7efff401ce20 con 0x7efffc0d1330 2023-11-28T12:59:53.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.794+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc0b2600 con 0x7efffc0d1330 2023-11-28T12:59:53.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.796+0000 7efff97fa700 1 -- 172.21.15.177:0/1714233063 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7efff4020070 con 0x7efffc0d1330 2023-11-28T12:59:53.870 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:53.869+0000 7fdc580d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T12:59:53.870 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:53.870+0000 7ff7c362e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T12:59:53.880 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.879+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd new", "uuid": "66975359-a324-4e64-a069-173dbe9e0aad", "id": 2} v 0) v1 -- 0x7efffc068690 con 0x7efffc0d1330 2023-11-28T12:59:53.887 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.887+0000 7efff97fa700 1 -- 172.21.15.177:0/1714233063 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "66975359-a324-4e64-a069-173dbe9e0aad", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7efff401c7e0 con 0x7efffc0d1330 2023-11-28T12:59:53.888 INFO:teuthology.orchestra.run.smithi177.stdout:2 2023-11-28T12:59:53.889 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efffc0d1330 msgr2=0x7efffc158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efffc0d1330 0x7efffc158e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efff4015000 tx=0x7efff4004ad0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 shutdown_connections 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efffc0d1330 0x7efffc158e40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efffc0b20b0 0x7efffc155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 --2- 172.21.15.177:0/1714233063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efffc0d01b0 0x7efffc15f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 >> 172.21.15.177:0/1714233063 conn(0x7efffc09c3f0 msgr2=0x7efffc15c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 shutdown_connections 2023-11-28T12:59:53.890 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:53.888+0000 7f0002e58700 1 -- 172.21.15.177:0/1714233063 wait complete. 2023-11-28T12:59:53.901 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new bd9de07f-3315-48c1-9d94-69f8f7a164fa 3 2023-11-28T12:59:53.958 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:53.958+0000 7ff7c362e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T12:59:53.965 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:53.964+0000 7fdc580d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T12:59:54.024 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.023+0000 7f4009575700 1 Processor -- start 2023-11-28T12:59:54.024 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.023+0000 7f4009575700 1 -- start start 2023-11-28T12:59:54.026 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f4009575700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40040cb620 0x7f40040ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.026 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f4009575700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40040cf070 0x7f40040ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f4009575700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40040cf4d0 con 0x7f40040b3090 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f4009575700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40040ccbb0 con 0x7f40040cb620 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f4009575700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400415ad60 con 0x7f40040cf070 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40040cf070 0x7f40040ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40040cf070 0x7f40040ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45346/0 (socket says 172.21.15.177:45346) 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 -- 172.21.15.177:0/768621150 learned_addr learned my addr 172.21.15.177:0/768621150 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40027fc700 1 --2- 172.21.15.177:0/768621150 >> v2:172.21.15.153:3300/0 conn(0x7f40040cb620 0x7f40040ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 -- 172.21.15.177:0/768621150 >> v2:172.21.15.153:3300/0 conn(0x7f40040cb620 msgr2=0x7f40040ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 --2- 172.21.15.177:0/768621150 >> v2:172.21.15.153:3300/0 conn(0x7f40040cb620 0x7f40040ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.027 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 -- 172.21.15.177:0/768621150 >> v1:172.21.15.177:6789/0 conn(0x7f40040b3090 legacy=0x7f40040cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 -- 172.21.15.177:0/768621150 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40040adb90 con 0x7f40040cf070 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40027fc700 1 --2- 172.21.15.177:0/768621150 >> v2:172.21.15.153:3300/0 conn(0x7f40040cb620 0x7f40040ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f4001ffb700 1 -- 172.21.15.177:0/768621150 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152842217 0 0) 0x7f40040cf4d0 con 0x7f40040b3090 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f40037fe700 1 --2- 172.21.15.177:0/768621150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040cf070 0x7f40040ae300 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4004071dc0 tx=0x7f3ffc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57ed0c45b7fada9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.024+0000 7f4001ffb700 1 -- 172.21.15.177:0/768621150 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00e070 con 0x7f40040cf070 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4001ffb700 1 -- 172.21.15.177:0/768621150 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc00b9c0 con 0x7f40040cf070 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4001ffb700 1 -- 172.21.15.177:0/768621150 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00bd70 con 0x7f40040cf070 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 -- 172.21.15.177:0/768621150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040cf070 msgr2=0x7f40040ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 --2- 172.21.15.177:0/768621150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040cf070 0x7f40040ae300 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4004071dc0 tx=0x7f3ffc00b330 comp rx=0 tx=0).stop 2023-11-28T12:59:54.028 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 -- 172.21.15.177:0/768621150 shutdown_connections 2023-11-28T12:59:54.029 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 --2- 172.21.15.177:0/768621150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040cf070 0x7f40040ae300 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.029 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 --2- 172.21.15.177:0/768621150 >> v2:172.21.15.153:3300/0 conn(0x7f40040cb620 0x7f40040ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.029 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 -- 172.21.15.177:0/768621150 >> 172.21.15.177:0/768621150 conn(0x7f400409a230 msgr2=0x7f400408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.029 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 -- 172.21.15.177:0/768621150 shutdown_connections 2023-11-28T12:59:54.029 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.025+0000 7f4009575700 1 -- 172.21.15.177:0/768621150 wait complete. 2023-11-28T12:59:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 Processor -- start 2023-11-28T12:59:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 -- start start 2023-11-28T12:59:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b3090 0x7f400415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040cb620 0x7f40041f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.032 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40040cf070 0x7f400406e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40040c99e0 con 0x7f40040cf070 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400415e070 con 0x7f40040b3090 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.027+0000 7f4009575700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400414d7a0 con 0x7f40040cb620 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b3090 0x7f400415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f40027fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040cb620 0x7f40041f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b3090 0x7f400415db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45348/0 (socket says 172.21.15.177:45348) 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 -- 172.21.15.177:0/2486861297 learned_addr learned my addr 172.21.15.177:0/2486861297 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.033 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f40037fe700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40040cf070 0x7f400406e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 -- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040cb620 msgr2=0x7f40041f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040cb620 0x7f40041f31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 -- 172.21.15.177:0/2486861297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40040cf070 msgr2=0x7f400406e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40040cf070 0x7f400406e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 -- 172.21.15.177:0/2486861297 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc009b30 con 0x7f40040b3090 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f40027fc700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040cb620 0x7f40041f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f40037fe700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40040cf070 0x7f400406e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4002ffd700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b3090 0x7f400415db30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c750 tx=0x7f3ff800ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.034 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f3feb7fe700 1 -- 172.21.15.177:0/2486861297 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80054e0 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f3feb7fe700 1 -- 172.21.15.177:0/2486861297 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff8003680 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f3feb7fe700 1 -- 172.21.15.177:0/2486861297 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8003a30 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40040cd0b0 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.028+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40040ccdd0 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.029+0000 7f3feb7fe700 1 -- 172.21.15.177:0/2486861297 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3ff8019030 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.029+0000 7f3feb7fe700 1 -- 172.21.15.177:0/2486861297 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f3ff801c440 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.029+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40040cd390 con 0x7f40040b3090 2023-11-28T12:59:54.035 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.031+0000 7f3feb7fe700 1 -- 172.21.15.177:0/2486861297 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3ff8017070 con 0x7f40040b3090 2023-11-28T12:59:54.095 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:54.095+0000 7ff7c362e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T12:59:54.108 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:54.107+0000 7fdc580d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T12:59:54.115 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.115+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bd9de07f-3315-48c1-9d94-69f8f7a164fa", "id": 3} v 0) v1 -- 0x7f40041f3770 con 0x7f40040b3090 2023-11-28T12:59:54.123 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.122+0000 7f3feb7fe700 1 -- 172.21.15.177:0/2486861297 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd9de07f-3315-48c1-9d94-69f8f7a164fa", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3ff8003b90 con 0x7f40040b3090 2023-11-28T12:59:54.123 INFO:teuthology.orchestra.run.smithi177.stdout:3 2023-11-28T12:59:54.124 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.123+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b3090 msgr2=0x7f400415db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.124 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.123+0000 7f4009575700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b3090 0x7f400415db30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c750 tx=0x7f3ff800ca60 comp rx=0 tx=0).stop 2023-11-28T12:59:54.124 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.123+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 shutdown_connections 2023-11-28T12:59:54.125 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.123+0000 7f4009575700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40040cf070 0x7f400406e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.125 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.123+0000 7f4009575700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40040b3090 0x7f400415db30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.125 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.123+0000 7f4009575700 1 --2- 172.21.15.177:0/2486861297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40040cb620 0x7f40041f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.125 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.123+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 >> 172.21.15.177:0/2486861297 conn(0x7f400409a230 msgr2=0x7f400406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.125 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.124+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 shutdown_connections 2023-11-28T12:59:54.125 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.124+0000 7f4009575700 1 -- 172.21.15.177:0/2486861297 wait complete. 2023-11-28T12:59:54.136 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new b595bb37-b598-404f-a1a7-80b1f1c28b11 4 2023-11-28T12:59:54.257 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.256+0000 7f287c168700 1 Processor -- start 2023-11-28T12:59:54.257 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.256+0000 7f287c168700 1 -- start start 2023-11-28T12:59:54.260 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287c168700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f287415cef0 0x7f28740af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.260 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287c168700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f28740cbdf0 0x7f287406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287c168700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f287406b8c0 0x7f2874071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287c168700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287406bd40 con 0x7f28740cbdf0 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287c168700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28740cc5f0 con 0x7f287406b8c0 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287c168700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287415d470 con 0x7f287415cef0 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f287406b8c0 0x7f2874071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f2879f04700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f287415cef0 0x7f28740af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f2879f04700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f287415cef0 0x7f28740af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58112/0 (socket says 172.21.15.177:58112) 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f287406b8c0 0x7f2874071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45358/0 (socket says 172.21.15.177:45358) 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f2879f04700 1 -- 172.21.15.177:0/2882836062 learned_addr learned my addr 172.21.15.177:0/2882836062 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f2879703700 1 --2- 172.21.15.177:0/2882836062 >> v2:172.21.15.153:3300/0 conn(0x7f28740cbdf0 0x7f287406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 -- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f287415cef0 msgr2=0x7f28740af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 --2- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f287415cef0 0x7f28740af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 -- 172.21.15.177:0/2882836062 >> v2:172.21.15.153:3300/0 conn(0x7f28740cbdf0 msgr2=0x7f287406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 --2- 172.21.15.177:0/2882836062 >> v2:172.21.15.153:3300/0 conn(0x7f28740cbdf0 0x7f287406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 -- 172.21.15.177:0/2882836062 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287415d750 con 0x7f287406b8c0 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f2879f04700 1 --2- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f287415cef0 0x7f28740af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f287a705700 1 --2- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287406b8c0 0x7f2874071040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2874073a70 tx=0x7f286c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeef2aa473f84846 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.258+0000 7f2878f02700 1 -- 172.21.15.177:0/2882836062 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c00e070 con 0x7f287406b8c0 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f2878f02700 1 -- 172.21.15.177:0/2882836062 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c00bb20 con 0x7f287406b8c0 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f2878f02700 1 -- 172.21.15.177:0/2882836062 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c012430 con 0x7f287406b8c0 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 -- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287406b8c0 msgr2=0x7f2874071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 --2- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287406b8c0 0x7f2874071040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2874073a70 tx=0x7f286c00b330 comp rx=0 tx=0).stop 2023-11-28T12:59:54.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 -- 172.21.15.177:0/2882836062 shutdown_connections 2023-11-28T12:59:54.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 --2- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f287415cef0 0x7f28740af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 --2- 172.21.15.177:0/2882836062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287406b8c0 0x7f2874071040 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 --2- 172.21.15.177:0/2882836062 >> v2:172.21.15.153:3300/0 conn(0x7f28740cbdf0 0x7f287406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 -- 172.21.15.177:0/2882836062 >> 172.21.15.177:0/2882836062 conn(0x7f28740d0000 msgr2=0x7f28740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 -- 172.21.15.177:0/2882836062 shutdown_connections 2023-11-28T12:59:54.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.259+0000 7f287c168700 1 -- 172.21.15.177:0/2882836062 wait complete. 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.261+0000 7f287c168700 1 Processor -- start 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.261+0000 7f287c168700 1 -- start start 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287406b8c0 0x7f28741551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28740cbdf0 0x7f28740add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287415cef0 0x7f2874148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740ae7d0 con 0x7f287406b8c0 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740b36e0 con 0x7f287415cef0 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28740b39c0 con 0x7f28740cbdf0 2023-11-28T12:59:54.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28740cbdf0 0x7f28740add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287a705700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287415cef0 0x7f2874148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28740cbdf0 0x7f28740add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58120/0 (socket says 172.21.15.177:58120) 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287a705700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287415cef0 0x7f2874148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45364/0 (socket says 172.21.15.177:45364) 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 -- 172.21.15.177:0/421239267 learned_addr learned my addr 172.21.15.177:0/421239267 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879f04700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287406b8c0 0x7f28741551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 -- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287415cef0 msgr2=0x7f2874148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287415cef0 0x7f2874148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 -- 172.21.15.177:0/421239267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287406b8c0 msgr2=0x7f28741551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287406b8c0 0x7f28741551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 -- 172.21.15.177:0/421239267 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287000b7f0 con 0x7f28740cbdf0 2023-11-28T12:59:54.267 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287a705700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287415cef0 0x7f2874148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f2879703700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28740cbdf0 0x7f28740add10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f286400e930 tx=0x7f286400ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f286a7fc700 1 -- 172.21.15.177:0/421239267 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864013680 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f286a7fc700 1 -- 172.21.15.177:0/421239267 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28640137e0 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c009a70 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f286a7fc700 1 -- 172.21.15.177:0/421239267 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864013b70 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.262+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287415dd10 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.263+0000 7f286a7fc700 1 -- 172.21.15.177:0/421239267 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f2864019030 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.263+0000 7f286a7fc700 1 -- 172.21.15.177:0/421239267 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2864006670 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.263+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287415da30 con 0x7f28740cbdf0 2023-11-28T12:59:54.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.265+0000 7f286a7fc700 1 -- 172.21.15.177:0/421239267 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2864021060 con 0x7f28740cbdf0 2023-11-28T12:59:54.350 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.349+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b595bb37-b598-404f-a1a7-80b1f1c28b11", "id": 4} v 0) v1 -- 0x7f287415dff0 con 0x7f28740cbdf0 2023-11-28T12:59:54.358 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.357+0000 7f286a7fc700 1 -- 172.21.15.177:0/421239267 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b595bb37-b598-404f-a1a7-80b1f1c28b11", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2864017070 con 0x7f28740cbdf0 2023-11-28T12:59:54.358 INFO:teuthology.orchestra.run.smithi177.stdout:4 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28740cbdf0 msgr2=0x7f28740add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28740cbdf0 0x7f28740add10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f286400e930 tx=0x7f286400ec40 comp rx=0 tx=0).stop 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 shutdown_connections 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287406b8c0 0x7f28741551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f287415cef0 0x7f2874148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 --2- 172.21.15.177:0/421239267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28740cbdf0 0x7f28740add10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 >> 172.21.15.177:0/421239267 conn(0x7f28740d0000 msgr2=0x7f287406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 shutdown_connections 2023-11-28T12:59:54.360 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.358+0000 7f287c168700 1 -- 172.21.15.177:0/421239267 wait complete. 2023-11-28T12:59:54.371 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 7eaefb1e-4587-46f9-9e26-614f22833df4 5 2023-11-28T12:59:54.487 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:54.486+0000 7ff7c362e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T12:59:54.492 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.492+0000 7fd2448f0700 1 Processor -- start 2023-11-28T12:59:54.492 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.492+0000 7fd2448f0700 1 -- start start 2023-11-28T12:59:54.495 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd2448f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400d0a00 0x7fd24015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd2448f0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24006b420 con 0x7fd2400b2930 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd2448f0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24006b700 con 0x7fd2400ce040 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd2448f0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24006ff60 con 0x7fd2400d0a00 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23dd9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400d0a00 0x7fd24015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23dd9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400d0a00 0x7fd24015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58126/0 (socket says 172.21.15.177:58126) 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23e59c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd2400ce040 0x7fd24006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:58296/0 (socket says 172.21.15.177:58296) 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23dd9b700 1 -- 172.21.15.177:0/2492132983 learned_addr learned my addr 172.21.15.177:0/2492132983 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23d59a700 1 -- 172.21.15.177:0/2492132983 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736628785 0 0) 0x7fd24006b700 con 0x7fd2400ce040 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23d59a700 1 -- 172.21.15.177:0/2492132983 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2280035b0 con 0x7fd2400ce040 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23dd9b700 1 -- 172.21.15.177:0/2492132983 >> v1:172.21.15.177:6789/0 conn(0x7fd2400ce040 legacy=0x7fd24006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23dd9b700 1 -- 172.21.15.177:0/2492132983 >> v1:172.21.15.153:6789/0 conn(0x7fd2400b2930 legacy=0x7fd24006c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.496 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23dd9b700 1 -- 172.21.15.177:0/2492132983 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24006b9e0 con 0x7fd2400d0a00 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23dd9b700 1 --2- 172.21.15.177:0/2492132983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400d0a00 0x7fd24015df10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd234009920 tx=0x7fd234009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eef3ea9be3a310f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23d59a700 1 -- 172.21.15.177:0/2492132983 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234011830 con 0x7fd2400d0a00 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23d59a700 1 -- 172.21.15.177:0/2492132983 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234011990 con 0x7fd2400d0a00 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.493+0000 7fd23d59a700 1 -- 172.21.15.177:0/2492132983 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234011d20 con 0x7fd2400d0a00 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.494+0000 7fd2448f0700 1 -- 172.21.15.177:0/2492132983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400d0a00 msgr2=0x7fd24015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.494+0000 7fd2448f0700 1 --2- 172.21.15.177:0/2492132983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400d0a00 0x7fd24015df10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd234009920 tx=0x7fd234009c30 comp rx=0 tx=0).stop 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.494+0000 7fd2448f0700 1 -- 172.21.15.177:0/2492132983 shutdown_connections 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.494+0000 7fd2448f0700 1 --2- 172.21.15.177:0/2492132983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400d0a00 0x7fd24015df10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.494+0000 7fd2448f0700 1 -- 172.21.15.177:0/2492132983 >> 172.21.15.177:0/2492132983 conn(0x7fd24006aed0 msgr2=0x7fd24006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.494+0000 7fd2448f0700 1 -- 172.21.15.177:0/2492132983 shutdown_connections 2023-11-28T12:59:54.497 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.494+0000 7fd2448f0700 1 -- 172.21.15.177:0/2492132983 wait complete. 2023-11-28T12:59:54.500 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.495+0000 7fd2448f0700 1 Processor -- start 2023-11-28T12:59:54.500 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd2448f0700 1 -- start start 2023-11-28T12:59:54.500 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd2448f0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2400b2930 0x7fd240152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.500 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd2448f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400ce040 0x7fd2400b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.500 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd2448f0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2400d0a00 0x7fd2401491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.500 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd2448f0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400b4470 con 0x7fd2400b2930 2023-11-28T12:59:54.501 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd2448f0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400b6370 con 0x7fd2400d0a00 2023-11-28T12:59:54.501 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd2448f0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400b4240 con 0x7fd2400ce040 2023-11-28T12:59:54.501 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd23ed9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2400d0a00 0x7fd2401491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.501 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd23dd9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400ce040 0x7fd2400b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.501 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd23dd9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400ce040 0x7fd2400b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58132/0 (socket says 172.21.15.177:58132) 2023-11-28T12:59:54.501 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd23ed9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2400d0a00 0x7fd2401491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45376/0 (socket says 172.21.15.177:45376) 2023-11-28T12:59:54.501 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd23dd9b700 1 -- 172.21.15.177:0/2182983792 learned_addr learned my addr 172.21.15.177:0/2182983792 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.502 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.496+0000 7fd23e59c700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2400b2930 0x7fd240152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.502 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd23ed9d700 1 -- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400ce040 msgr2=0x7fd2400b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.502 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd23ed9d700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400ce040 0x7fd2400b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.502 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd23ed9d700 1 -- 172.21.15.177:0/2182983792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2400b2930 msgr2=0x7fd240152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.502 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd23ed9d700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2400b2930 0x7fd240152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd23ed9d700 1 -- 172.21.15.177:0/2182983792 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd234009550 con 0x7fd2400d0a00 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd23dd9b700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400ce040 0x7fd2400b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd23ed9d700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2400d0a00 0x7fd2401491a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd23800cb90 tx=0x7fd23800cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd22effd700 1 -- 172.21.15.177:0/2182983792 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238003c00 con 0x7fd2400d0a00 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd22effd700 1 -- 172.21.15.177:0/2182983792 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238003d60 con 0x7fd2400d0a00 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24006bfa0 con 0x7fd2400d0a00 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd22effd700 1 -- 172.21.15.177:0/2182983792 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800aa80 con 0x7fd2400d0a00 2023-11-28T12:59:54.503 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24006bcc0 con 0x7fd2400d0a00 2023-11-28T12:59:54.504 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd22effd700 1 -- 172.21.15.177:0/2182983792 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd238010050 con 0x7fd2400d0a00 2023-11-28T12:59:54.504 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.497+0000 7fd22effd700 1 -- 172.21.15.177:0/2182983792 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fd238012420 con 0x7fd2400d0a00 2023-11-28T12:59:54.504 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.498+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2400b2db0 con 0x7fd2400d0a00 2023-11-28T12:59:54.504 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.500+0000 7fd22effd700 1 -- 172.21.15.177:0/2182983792 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd2380050d0 con 0x7fd2400d0a00 2023-11-28T12:59:54.523 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:54.523+0000 7fdc580d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T12:59:54.586 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.585+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7eaefb1e-4587-46f9-9e26-614f22833df4", "id": 5} v 0) v1 -- 0x7fd240068690 con 0x7fd2400d0a00 2023-11-28T12:59:54.595 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.594+0000 7fd22effd700 1 -- 172.21.15.177:0/2182983792 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7eaefb1e-4587-46f9-9e26-614f22833df4", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd238005270 con 0x7fd2400d0a00 2023-11-28T12:59:54.595 INFO:teuthology.orchestra.run.smithi177.stdout:5 2023-11-28T12:59:54.596 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2400d0a00 msgr2=0x7fd2401491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.596 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2400d0a00 0x7fd2401491a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd23800cb90 tx=0x7fd23800cea0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.596 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 shutdown_connections 2023-11-28T12:59:54.596 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2400b2930 0x7fd240152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.596 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2400d0a00 0x7fd2401491a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.596 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 --2- 172.21.15.177:0/2182983792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2400ce040 0x7fd2400b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.596 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 >> 172.21.15.177:0/2182983792 conn(0x7fd24006aed0 msgr2=0x7fd24006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.597 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 shutdown_connections 2023-11-28T12:59:54.597 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.595+0000 7fd2448f0700 1 -- 172.21.15.177:0/2182983792 wait complete. 2023-11-28T12:59:54.608 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 891d9b75-4a1a-49b0-ac1f-393a9271171f 6 2023-11-28T12:59:54.728 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.727+0000 7f33ab2d0700 1 Processor -- start 2023-11-28T12:59:54.729 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.728+0000 7f33ab2d0700 1 -- start start 2023-11-28T12:59:54.731 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33ab2d0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f33a4160b10 0x7f33a40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33ab2d0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a406b060 0x7f33a406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33ab2d0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40c9e10 con 0x7f33a4160740 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33ab2d0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a40ca720 con 0x7f33a4160b10 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33ab2d0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a406efc0 con 0x7f33a406b060 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33a986d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a406b060 0x7f33a406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33a886b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f33a4160b10 0x7f33a40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33a886b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f33a4160b10 0x7f33a40cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45380/0 (socket says 172.21.15.177:45380) 2023-11-28T12:59:54.732 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33a986d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a406b060 0x7f33a406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58140/0 (socket says 172.21.15.177:58140) 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.729+0000 7f33a886b700 1 -- 172.21.15.177:0/3420703281 learned_addr learned my addr 172.21.15.177:0/3420703281 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f33a986d700 1 -- 172.21.15.177:0/3420703281 >> v2:172.21.15.177:3300/0 conn(0x7f33a4160b10 msgr2=0x7f33a40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f33a986d700 1 --2- 172.21.15.177:0/3420703281 >> v2:172.21.15.177:3300/0 conn(0x7f33a4160b10 0x7f33a40cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f33a986d700 1 -- 172.21.15.177:0/3420703281 >> v1:172.21.15.177:6789/0 conn(0x7f33a4160740 legacy=0x7f33a40cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f33a986d700 1 -- 172.21.15.177:0/3420703281 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40ca0d0 con 0x7f33a406b060 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f33a986d700 1 --2- 172.21.15.177:0/3420703281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a406b060 0x7f33a406b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f339c009d90 tx=0x7f339c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953ded515459b2b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f339bfff700 1 -- 172.21.15.177:0/3420703281 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c00e070 con 0x7f33a406b060 2023-11-28T12:59:54.733 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f339bfff700 1 -- 172.21.15.177:0/3420703281 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c00bb20 con 0x7f33a406b060 2023-11-28T12:59:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f339bfff700 1 -- 172.21.15.177:0/3420703281 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c012430 con 0x7f33a406b060 2023-11-28T12:59:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f33ab2d0700 1 -- 172.21.15.177:0/3420703281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a406b060 msgr2=0x7f33a406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.730+0000 7f33ab2d0700 1 --2- 172.21.15.177:0/3420703281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a406b060 0x7f33a406b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f339c009d90 tx=0x7f339c00b330 comp rx=0 tx=0).stop 2023-11-28T12:59:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.731+0000 7f33ab2d0700 1 -- 172.21.15.177:0/3420703281 shutdown_connections 2023-11-28T12:59:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.731+0000 7f33ab2d0700 1 --2- 172.21.15.177:0/3420703281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a406b060 0x7f33a406b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.731+0000 7f33ab2d0700 1 --2- 172.21.15.177:0/3420703281 >> v2:172.21.15.177:3300/0 conn(0x7f33a4160b10 0x7f33a40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.734 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.731+0000 7f33ab2d0700 1 -- 172.21.15.177:0/3420703281 >> 172.21.15.177:0/3420703281 conn(0x7f33a40b12b0 msgr2=0x7f33a40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.735 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.731+0000 7f33ab2d0700 1 -- 172.21.15.177:0/3420703281 shutdown_connections 2023-11-28T12:59:54.735 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.731+0000 7f33ab2d0700 1 -- 172.21.15.177:0/3420703281 wait complete. 2023-11-28T12:59:54.737 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.732+0000 7f33ab2d0700 1 Processor -- start 2023-11-28T12:59:54.737 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.732+0000 7f33ab2d0700 1 -- start start 2023-11-28T12:59:54.737 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33ab2d0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33a406b060 0x7f33a41f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.737 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33ab2d0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a4160740 0x7f33a4159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.737 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33ab2d0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4160b10 0x7f33a4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33ab2d0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40c8900 con 0x7f33a4160b10 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33ab2d0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40c8470 con 0x7f33a406b060 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33ab2d0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41f3550 con 0x7f33a4160740 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33a886b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a4160740 0x7f33a4159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33a906c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33a406b060 0x7f33a41f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33a886b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a4160740 0x7f33a4159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58144/0 (socket says 172.21.15.177:58144) 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.733+0000 7f33a886b700 1 -- 172.21.15.177:0/2710986881 learned_addr learned my addr 172.21.15.177:0/2710986881 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.738 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a986d700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4160b10 0x7f33a4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a886b700 1 -- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33a406b060 msgr2=0x7f33a41f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a886b700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33a406b060 0x7f33a41f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a886b700 1 -- 172.21.15.177:0/2710986881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4160b10 msgr2=0x7f33a4159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a886b700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4160b10 0x7f33a4159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a886b700 1 -- 172.21.15.177:0/2710986881 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c009a70 con 0x7f33a4160740 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a906c700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33a406b060 0x7f33a41f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33a886b700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a4160740 0x7f33a4159710 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f339400e8b0 tx=0x7f339400ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.739 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f3399ffb700 1 -- 172.21.15.177:0/2710986881 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394009600 con 0x7f33a4160740 2023-11-28T12:59:54.740 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f3399ffb700 1 -- 172.21.15.177:0/2710986881 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394013680 con 0x7f33a4160740 2023-11-28T12:59:54.740 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f3399ffb700 1 -- 172.21.15.177:0/2710986881 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394013a30 con 0x7f33a4160740 2023-11-28T12:59:54.740 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a40cd8d0 con 0x7f33a4160740 2023-11-28T12:59:54.740 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a40ca3b0 con 0x7f33a4160740 2023-11-28T12:59:54.740 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f3399ffb700 1 -- 172.21.15.177:0/2710986881 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3394019030 con 0x7f33a4160740 2023-11-28T12:59:54.740 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.734+0000 7f3399ffb700 1 -- 172.21.15.177:0/2710986881 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f33940066b0 con 0x7f33a4160740 2023-11-28T12:59:54.740 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.735+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4160f60 con 0x7f33a4160740 2023-11-28T12:59:54.741 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.739+0000 7f3399ffb700 1 -- 172.21.15.177:0/2710986881 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3394021060 con 0x7f33a4160740 2023-11-28T12:59:54.822 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.821+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "891d9b75-4a1a-49b0-ac1f-393a9271171f", "id": 6} v 0) v1 -- 0x7f33a40c85e0 con 0x7f33a4160740 2023-11-28T12:59:54.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.830+0000 7f3399ffb700 1 -- 172.21.15.177:0/2710986881 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "891d9b75-4a1a-49b0-ac1f-393a9271171f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3394017070 con 0x7f33a4160740 2023-11-28T12:59:54.831 INFO:teuthology.orchestra.run.smithi177.stdout:6 2023-11-28T12:59:54.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.830+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a4160740 msgr2=0x7f33a4159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.830+0000 7f33ab2d0700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a4160740 0x7f33a4159710 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f339400e8b0 tx=0x7f339400ebc0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.831+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 shutdown_connections 2023-11-28T12:59:54.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.831+0000 7f33ab2d0700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33a4160b10 0x7f33a4159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.831+0000 7f33ab2d0700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33a406b060 0x7f33a41f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.831+0000 7f33ab2d0700 1 --2- 172.21.15.177:0/2710986881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33a4160740 0x7f33a4159710 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.831+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 >> 172.21.15.177:0/2710986881 conn(0x7f33a40b12b0 msgr2=0x7f33a40b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.831+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 shutdown_connections 2023-11-28T12:59:54.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.831+0000 7f33ab2d0700 1 -- 172.21.15.177:0/2710986881 wait complete. 2023-11-28T12:59:54.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new b34ca128-3e2e-4ff0-8d55-8372fd8df439 7 2023-11-28T12:59:54.982 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.982+0000 7ff64f62f700 1 Processor -- start 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.982+0000 7ff64f62f700 1 -- start start 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.983+0000 7ff64f62f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff648158890 0x7ff6480ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.983+0000 7ff64f62f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff6480b0d80 0x7ff64806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.983+0000 7ff64f62f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64815c490 con 0x7ff64806d4f0 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.983+0000 7ff64f62f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6480ce650 con 0x7ff648158890 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.983+0000 7ff64f62f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64815c730 con 0x7ff6480b0d80 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64cbca700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff64806d4f0 0x7ff6480b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.177:51044/0 (socket says 172.21.15.177:51044) 2023-11-28T12:59:54.985 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64d3cb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff648158890 0x7ff6480ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64cbca700 1 -- 172.21.15.177:0/3515404846 learned_addr learned my addr 172.21.15.177:0/3515404846 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff63ffff700 1 -- 172.21.15.177:0/3515404846 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226531246 0 0) 0x7ff64815c490 con 0x7ff64806d4f0 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff63ffff700 1 -- 172.21.15.177:0/3515404846 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6340035b0 con 0x7ff64806d4f0 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64d3cb700 1 -- 172.21.15.177:0/3515404846 >> v2:172.21.15.177:3300/0 conn(0x7ff6480b0d80 msgr2=0x7ff64806b820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff63ffff700 1 -- 172.21.15.177:0/3515404846 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2957143998 0 0) 0x7ff6340035b0 con 0x7ff64806d4f0 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64d3cb700 1 --2- 172.21.15.177:0/3515404846 >> v2:172.21.15.177:3300/0 conn(0x7ff6480b0d80 0x7ff64806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64d3cb700 1 -- 172.21.15.177:0/3515404846 >> v1:172.21.15.153:6789/0 conn(0x7ff64806d4f0 legacy=0x7ff6480b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64d3cb700 1 -- 172.21.15.177:0/3515404846 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64815ca10 con 0x7ff648158890 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff64d3cb700 1 --2- 172.21.15.177:0/3515404846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff648158890 0x7ff6480ce110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff6480ca0d0 tx=0x7ff64000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc7ed5396406aaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff63ffff700 1 -- 172.21.15.177:0/3515404846 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638001670 con 0x7ff648158890 2023-11-28T12:59:54.987 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff63ffff700 1 -- 172.21.15.177:0/3515404846 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff640009e50 con 0x7ff648158890 2023-11-28T12:59:54.987 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.984+0000 7ff63ffff700 1 -- 172.21.15.177:0/3515404846 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640003c40 con 0x7ff648158890 2023-11-28T12:59:54.987 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 -- 172.21.15.177:0/3515404846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff648158890 msgr2=0x7ff6480ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.987 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 --2- 172.21.15.177:0/3515404846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff648158890 0x7ff6480ce110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff6480ca0d0 tx=0x7ff64000b330 comp rx=0 tx=0).stop 2023-11-28T12:59:54.987 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 -- 172.21.15.177:0/3515404846 shutdown_connections 2023-11-28T12:59:54.987 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 --2- 172.21.15.177:0/3515404846 >> v2:172.21.15.177:3300/0 conn(0x7ff6480b0d80 0x7ff64806b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.987 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 --2- 172.21.15.177:0/3515404846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff648158890 0x7ff6480ce110 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff6480ca0d0 tx=0x7ff64000b330 comp rx=0 tx=0).stop 2023-11-28T12:59:54.988 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 -- 172.21.15.177:0/3515404846 >> 172.21.15.177:0/3515404846 conn(0x7ff6480cb670 msgr2=0x7ff6480cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:54.988 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 -- 172.21.15.177:0/3515404846 shutdown_connections 2023-11-28T12:59:54.989 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.985+0000 7ff64f62f700 1 -- 172.21.15.177:0/3515404846 wait complete. 2023-11-28T12:59:54.989 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.987+0000 7ff64f62f700 1 Processor -- start 2023-11-28T12:59:54.989 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.987+0000 7ff64f62f700 1 -- start start 2023-11-28T12:59:54.989 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64f62f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff64806d4f0 0x7ff6481592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.989 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64f62f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6480b0d80 0x7ff6480c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.989 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64f62f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6480c46f0 0x7ff6480b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:54.989 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64f62f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480c5c30 con 0x7ff64806d4f0 2023-11-28T12:59:54.990 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64f62f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480c8d70 con 0x7ff6480c46f0 2023-11-28T12:59:54.990 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64f62f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480b28d0 con 0x7ff6480b0d80 2023-11-28T12:59:54.990 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64cbca700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6480b0d80 0x7ff6480c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.990 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64dbcc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6480c46f0 0x7ff6480b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.990 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64cbca700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6480b0d80 0x7ff6480c3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:58154/0 (socket says 172.21.15.177:58154) 2023-11-28T12:59:54.990 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64dbcc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6480c46f0 0x7ff6480b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:45398/0 (socket says 172.21.15.177:45398) 2023-11-28T12:59:54.990 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.988+0000 7ff64cbca700 1 -- 172.21.15.177:0/468422050 learned_addr learned my addr 172.21.15.177:0/468422050 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T12:59:54.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64d3cb700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff64806d4f0 0x7ff6481592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:54.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64dbcc700 1 -- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6480b0d80 msgr2=0x7ff6480c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64dbcc700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6480b0d80 0x7ff6480c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64dbcc700 1 -- 172.21.15.177:0/468422050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff64806d4f0 msgr2=0x7ff6481592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64dbcc700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff64806d4f0 0x7ff6481592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64dbcc700 1 -- 172.21.15.177:0/468422050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff640009a70 con 0x7ff6480c46f0 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64cbca700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6480b0d80 0x7ff6480c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64dbcc700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6480c46f0 0x7ff6480b2d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff64400b3e0 tx=0x7ff64400b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff63dffb700 1 -- 172.21.15.177:0/468422050 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644011620 con 0x7ff6480c46f0 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff63dffb700 1 -- 172.21.15.177:0/468422050 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644011780 con 0x7ff6480c46f0 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff63dffb700 1 -- 172.21.15.177:0/468422050 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644011b30 con 0x7ff6480c46f0 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.989+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6480c9370 con 0x7ff6480c46f0 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.990+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64815ccf0 con 0x7ff6480c46f0 2023-11-28T12:59:54.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.990+0000 7ff63dffb700 1 -- 172.21.15.177:0/468422050 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff644021070 con 0x7ff6480c46f0 2023-11-28T12:59:54.993 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.990+0000 7ff63dffb700 1 -- 172.21.15.177:0/468422050 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff64401aad0 con 0x7ff6480c46f0 2023-11-28T12:59:54.993 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.991+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648158c60 con 0x7ff6480c46f0 2023-11-28T12:59:54.995 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:54.995+0000 7ff63dffb700 1 -- 172.21.15.177:0/468422050 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff644039b00 con 0x7ff6480c46f0 2023-11-28T12:59:55.053 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:55.052+0000 7ff7c362e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T12:59:55.084 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.083+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b34ca128-3e2e-4ff0-8d55-8372fd8df439", "id": 7} v 0) v1 -- 0x7ff6480b2a80 con 0x7ff6480c46f0 2023-11-28T12:59:55.092 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.091+0000 7ff63dffb700 1 -- 172.21.15.177:0/468422050 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b34ca128-3e2e-4ff0-8d55-8372fd8df439", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff6480b2a80 con 0x7ff6480c46f0 2023-11-28T12:59:55.093 INFO:teuthology.orchestra.run.smithi177.stdout:7 2023-11-28T12:59:55.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6480c46f0 msgr2=0x7ff6480b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6480c46f0 0x7ff6480b2d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff64400b3e0 tx=0x7ff64400b6f0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 shutdown_connections 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff64806d4f0 0x7ff6481592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6480c46f0 0x7ff6480b2d60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 --2- 172.21.15.177:0/468422050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6480b0d80 0x7ff6480c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 >> 172.21.15.177:0/468422050 conn(0x7ff6480cb670 msgr2=0x7ff6480cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 shutdown_connections 2023-11-28T12:59:55.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T12:59:55.092+0000 7ff64f62f700 1 -- 172.21.15.177:0/468422050 wait complete. 2023-11-28T12:59:55.105 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T12:59:55.105 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T12:59:55.107 INFO:tasks.ceph.osd.0:Started 2023-11-28T12:59:55.108 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T12:59:55.108 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T12:59:55.109 INFO:tasks.ceph.osd.1:Started 2023-11-28T12:59:55.110 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T12:59:55.110 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T12:59:55.112 INFO:tasks.ceph.osd.2:Started 2023-11-28T12:59:55.112 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T12:59:55.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T12:59:55.114 INFO:tasks.ceph.osd.3:Started 2023-11-28T12:59:55.114 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T12:59:55.115 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T12:59:55.116 INFO:tasks.ceph.osd.4:Started 2023-11-28T12:59:55.117 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T12:59:55.117 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T12:59:55.118 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:55.118+0000 7fdc580d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T12:59:55.119 INFO:tasks.ceph.osd.5:Started 2023-11-28T12:59:55.119 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T12:59:55.119 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T12:59:55.121 INFO:tasks.ceph.osd.6:Started 2023-11-28T12:59:55.121 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T12:59:55.121 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T12:59:55.125 INFO:tasks.ceph.osd.7:Started 2023-11-28T12:59:55.126 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T12:59:55.130 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:55.129+0000 7ff7c362e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T12:59:55.204 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:55.203+0000 7fdc580d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T12:59:55.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.281+0000 7fee63702700 1 Processor -- start 2023-11-28T12:59:55.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee63702700 1 -- start start 2023-11-28T12:59:55.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee63702700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c06dee0 0x7fee5c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:55.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee63702700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fee5c160540 0x7fee5c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:55.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee63702700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5c15d060 con 0x7fee5c071040 2023-11-28T12:59:55.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee63702700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5c15d340 con 0x7fee5c160540 2023-11-28T12:59:55.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee63702700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c15d620 con 0x7fee5c06dee0 2023-11-28T12:59:55.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee6149e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fee5c071040 0x7fee5c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:55814/0 (socket says 172.21.15.153:55814) 2023-11-28T12:59:55.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.282+0000 7fee6149e700 1 -- 172.21.15.153:0/3706910778 learned_addr learned my addr 172.21.15.153:0/3706910778 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee61c9f700 1 --2- 172.21.15.153:0/3706910778 >> v2:172.21.15.177:3300/0 conn(0x7fee5c160540 0x7fee5c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee53fff700 1 -- 172.21.15.153:0/3706910778 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668485915 0 0) 0x7fee5c15d060 con 0x7fee5c071040 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee53fff700 1 -- 172.21.15.153:0/3706910778 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee480035b0 con 0x7fee5c071040 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee61c9f700 1 -- 172.21.15.153:0/3706910778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c06dee0 msgr2=0x7fee5c06e330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee61c9f700 1 --2- 172.21.15.153:0/3706910778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c06dee0 0x7fee5c06e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee61c9f700 1 -- 172.21.15.153:0/3706910778 >> v1:172.21.15.153:6789/0 conn(0x7fee5c071040 legacy=0x7fee5c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee53fff700 1 -- 172.21.15.153:0/3706910778 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990978608 0 0) 0x7fee480035b0 con 0x7fee5c071040 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee61c9f700 1 -- 172.21.15.153:0/3706910778 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c15d900 con 0x7fee5c160540 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee61c9f700 1 --2- 172.21.15.153:0/3706910778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c160540 0x7fee5c1609c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d90 tx=0x7fee5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6fe9839ed811bc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:55.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.283+0000 7fee53fff700 1 -- 172.21.15.153:0/3706910778 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5400e070 con 0x7fee5c160540 2023-11-28T12:59:55.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee53fff700 1 -- 172.21.15.153:0/3706910778 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee54003a20 con 0x7fee5c160540 2023-11-28T12:59:55.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee53fff700 1 -- 172.21.15.153:0/3706910778 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c001670 con 0x7fee5c160540 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 -- 172.21.15.153:0/3706910778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c160540 msgr2=0x7fee5c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 --2- 172.21.15.153:0/3706910778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c160540 0x7fee5c1609c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d90 tx=0x7fee5400b330 comp rx=0 tx=0).stop 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 -- 172.21.15.153:0/3706910778 shutdown_connections 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 --2- 172.21.15.153:0/3706910778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c160540 0x7fee5c1609c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 --2- 172.21.15.153:0/3706910778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c06dee0 0x7fee5c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 -- 172.21.15.153:0/3706910778 >> 172.21.15.153:0/3706910778 conn(0x7fee5c0d0000 msgr2=0x7fee5c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 -- 172.21.15.153:0/3706910778 shutdown_connections 2023-11-28T12:59:55.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.285+0000 7fee63702700 1 -- 172.21.15.153:0/3706910778 wait complete. 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.286+0000 7fee63702700 1 Processor -- start 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.286+0000 7fee63702700 1 -- start start 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee63702700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c06dee0 0x7fee5c1f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee63702700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c071040 0x7fee5c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee63702700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c160540 0x7fee5c150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee63702700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0b45a0 con 0x7fee5c071040 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee63702700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0b4880 con 0x7fee5c06dee0 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee63702700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c1f5c10 con 0x7fee5c160540 2023-11-28T12:59:55.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee60c9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c071040 0x7fee5c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee60c9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c071040 0x7fee5c0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37658/0 (socket says 172.21.15.153:37658) 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee60c9d700 1 -- 172.21.15.153:0/3156143912 learned_addr learned my addr 172.21.15.153:0/3156143912 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee6149e700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c06dee0 0x7fee5c1f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee61c9f700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c160540 0x7fee5c150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee60c9d700 1 -- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c160540 msgr2=0x7fee5c150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee60c9d700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c160540 0x7fee5c150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee60c9d700 1 -- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c06dee0 msgr2=0x7fee5c1f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.287+0000 7fee60c9d700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c06dee0 0x7fee5c1f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee60c9d700 1 -- 172.21.15.153:0/3156143912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54009a70 con 0x7fee5c071040 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee6149e700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c06dee0 0x7fee5c1f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee60c9d700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c071040 0x7fee5c0b3d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b470 tx=0x7fee5800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:55.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee51ffb700 1 -- 172.21.15.153:0/3156143912 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58011830 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee51ffb700 1 -- 172.21.15.153:0/3156143912 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee580119b0 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c15ccd0 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee51ffb700 1 -- 172.21.15.153:0/3156143912 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58011d60 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee51ffb700 1 -- 172.21.15.153:0/3156143912 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fee5800f050 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c06e870 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.288+0000 7fee51ffb700 1 -- 172.21.15.153:0/3156143912 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fee5801acc0 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.289+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c06a270 con 0x7fee5c071040 2023-11-28T12:59:55.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.291+0000 7fee51ffb700 1 -- 172.21.15.153:0/3156143912 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fee58039b00 con 0x7fee5c071040 2023-11-28T12:59:55.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.360+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee5c1f5a70 con 0x7fee5c071040 2023-11-28T12:59:55.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.361+0000 7fee51ffb700 1 -- 172.21.15.153:0/3156143912 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fee5c1f5a70 con 0x7fee5c071040 2023-11-28T12:59:55.362 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T12:59:55.362 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":10,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T12:59:55.084798+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c071040 msgr2=0x7fee5c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c071040 0x7fee5c0b3d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b470 tx=0x7fee5800b780 comp rx=0 tx=0).stop 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 shutdown_connections 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee5c160540 0x7fee5c150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee5c06dee0 0x7fee5c1f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 --2- 172.21.15.153:0/3156143912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee5c071040 0x7fee5c0b3d00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 >> 172.21.15.153:0/3156143912 conn(0x7fee5c0d0000 msgr2=0x7fee5c0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:55.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 shutdown_connections 2023-11-28T12:59:55.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T12:59:55.362+0000 7fee63702700 1 -- 172.21.15.153:0/3156143912 wait complete. 2023-11-28T12:59:55.375 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T12:59:55.375 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T12:59:55.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2023-11-28T12:59:55.474+0000 7f2d1d018640 -1 Falling back to public interface 2023-11-28T12:59:55.736 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T12:59:55.735+0000 7f10f25e6640 -1 Falling back to public interface 2023-11-28T12:59:55.742 INFO:tasks.ceph.osd.2.smithi153.stderr:2023-11-28T12:59:55.741+0000 7fb49b978640 -1 Falling back to public interface 2023-11-28T12:59:55.745 INFO:tasks.ceph.osd.1.smithi153.stderr:2023-11-28T12:59:55.744+0000 7fd05997c640 -1 Falling back to public interface 2023-11-28T12:59:55.747 INFO:tasks.ceph.osd.3.smithi153.stderr:2023-11-28T12:59:55.746+0000 7f80ea288640 -1 Falling back to public interface 2023-11-28T12:59:55.748 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T12:59:55.747+0000 7f45dc913640 -1 Falling back to public interface 2023-11-28T12:59:55.748 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T12:59:55.747+0000 7f81df1f4640 -1 Falling back to public interface 2023-11-28T12:59:55.748 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T12:59:55.748+0000 7f45d3cd1700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-11-28T12:59:55.751 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T12:59:55.750+0000 7f918ec55640 -1 Falling back to public interface 2023-11-28T12:59:55.845 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:55.844+0000 7ff7c362e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:59:55.890 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:55.890+0000 7fdc580d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:59:55.921 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:55.920+0000 7ff7c362e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T12:59:55.966 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:55.966+0000 7fdc580d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T12:59:56.059 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:56.059+0000 7ff7c362e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:59:56.109 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:56.108+0000 7fdc580d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T12:59:56.179 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T12:59:56.296 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.295+0000 7fcbc6838700 1 Processor -- start 2023-11-28T12:59:56.296 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.295+0000 7fcbc6838700 1 -- start start 2023-11-28T12:59:56.298 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc6838700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcbc00cb0e0 0x7fcbc0065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:56.298 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc6838700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcbc00cec70 0x7fcbc006d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:56.298 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc6838700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc0066580 con 0x7fcbc006ec90 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc6838700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00cded0 con 0x7fcbc00cec70 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc6838700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00ce1b0 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc4dd5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcbc00cec70 0x7fcbc006d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbbffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcbc006ec90 0x7fcbc00ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:55826/0 (socket says 172.21.15.153:55826) 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbbffff700 1 -- 172.21.15.153:0/2666859950 learned_addr learned my addr 172.21.15.153:0/2666859950 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbbf7fe700 1 --2- 172.21.15.153:0/2666859950 >> v2:172.21.15.177:3300/0 conn(0x7fcbc00cb0e0 0x7fcbc0065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc4dd5700 1 -- 172.21.15.153:0/2666859950 >> v2:172.21.15.177:3300/0 conn(0x7fcbc00cb0e0 msgr2=0x7fcbc0065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc4dd5700 1 --2- 172.21.15.153:0/2666859950 >> v2:172.21.15.177:3300/0 conn(0x7fcbc00cb0e0 0x7fcbc0065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc4dd5700 1 -- 172.21.15.153:0/2666859950 >> v1:172.21.15.153:6789/0 conn(0x7fcbc006ec90 legacy=0x7fcbc00ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbbeffd700 1 -- 172.21.15.153:0/2666859950 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043435822 0 0) 0x7fcbc0066580 con 0x7fcbc006ec90 2023-11-28T12:59:56.299 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc4dd5700 1 -- 172.21.15.153:0/2666859950 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00ce490 con 0x7fcbc00cec70 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbbf7fe700 1 --2- 172.21.15.153:0/2666859950 >> v2:172.21.15.177:3300/0 conn(0x7fcbc00cb0e0 0x7fcbc0065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbc4dd5700 1 --2- 172.21.15.153:0/2666859950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cec70 0x7fcbc006d230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0070650 tx=0x7fcbb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8085646f6920f7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.296+0000 7fcbbeffd700 1 -- 172.21.15.153:0/2666859950 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800e070 con 0x7fcbc00cec70 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbbeffd700 1 -- 172.21.15.153:0/2666859950 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb800bbd0 con 0x7fcbc00cec70 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbbeffd700 1 -- 172.21.15.153:0/2666859950 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8012430 con 0x7fcbc00cec70 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 -- 172.21.15.153:0/2666859950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cec70 msgr2=0x7fcbc006d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 --2- 172.21.15.153:0/2666859950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cec70 0x7fcbc006d230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0070650 tx=0x7fcbb800b330 comp rx=0 tx=0).stop 2023-11-28T12:59:56.300 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 -- 172.21.15.153:0/2666859950 shutdown_connections 2023-11-28T12:59:56.301 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 --2- 172.21.15.153:0/2666859950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cec70 0x7fcbc006d230 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.301 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 --2- 172.21.15.153:0/2666859950 >> v2:172.21.15.177:3300/0 conn(0x7fcbc00cb0e0 0x7fcbc0065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.301 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 -- 172.21.15.153:0/2666859950 >> 172.21.15.153:0/2666859950 conn(0x7fcbc00b12b0 msgr2=0x7fcbc00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:56.301 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 -- 172.21.15.153:0/2666859950 shutdown_connections 2023-11-28T12:59:56.301 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.297+0000 7fcbc6838700 1 -- 172.21.15.153:0/2666859950 wait complete. 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.298+0000 7fcbc6838700 1 Processor -- start 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc6838700 1 -- start start 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc6838700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc006ec90 0x7fcbc00c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc6838700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cb0e0 0x7fcbc01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc6838700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc00cec70 0x7fcbc00b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc6838700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00c8470 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc6838700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00c5eb0 con 0x7fcbc00cec70 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc6838700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00c41f0 con 0x7fcbc006ec90 2023-11-28T12:59:56.304 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbbf7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cb0e0 0x7fcbc01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbbf7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cb0e0 0x7fcbc01552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37670/0 (socket says 172.21.15.153:37670) 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbbf7fe700 1 -- 172.21.15.153:0/2719178430 learned_addr learned my addr 172.21.15.153:0/2719178430 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbbffff700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc006ec90 0x7fcbc00c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.299+0000 7fcbc4dd5700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc00cec70 0x7fcbc00b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbf7fe700 1 -- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc006ec90 msgr2=0x7fcbc00c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbf7fe700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc006ec90 0x7fcbc00c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbf7fe700 1 -- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc00cec70 msgr2=0x7fcbc00b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbf7fe700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc00cec70 0x7fcbc00b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbf7fe700 1 -- 172.21.15.153:0/2719178430 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb8009a70 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbf7fe700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cb0e0 0x7fcbc01552e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400e7e0 tx=0x7fcbb400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbcff9700 1 -- 172.21.15.153:0/2719178430 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400c070 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.305 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbcff9700 1 -- 172.21.15.153:0/2719178430 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb4004230 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.306 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc00cb6c0 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.306 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbcff9700 1 -- 172.21.15.153:0/2719178430 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40045a0 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.306 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc00ce770 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.306 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbcff9700 1 -- 172.21.15.153:0/2719178430 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fcbb4019020 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.306 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.300+0000 7fcbbcff9700 1 -- 172.21.15.153:0/2719178430 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fcbb401c400 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.306 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.301+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc006f060 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.306 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.303+0000 7fcbbcff9700 1 -- 172.21.15.153:0/2719178430 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fcbb400f040 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.367 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:56.366+0000 7ff7c362e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T12:59:56.373 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.373+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc00671d0 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.374 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.373+0000 7fcbbcff9700 1 -- 172.21.15.153:0/2719178430 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fcbb40134c0 con 0x7fcbc00cb0e0 2023-11-28T12:59:56.374 INFO:teuthology.misc.health.smithi153.stdout: 2023-11-28T12:59:56.374 INFO:teuthology.misc.health.smithi153.stdout:{"epoch":10,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T12:59:55.084798+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T12:59:56.375 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cb0e0 msgr2=0x7fcbc01552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T12:59:56.375 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cb0e0 0x7fcbc01552e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400e7e0 tx=0x7fcbb400eaf0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.375 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 shutdown_connections 2023-11-28T12:59:56.375 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc00cec70 0x7fcbc00b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.375 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc006ec90 0x7fcbc00c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.376 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 --2- 172.21.15.153:0/2719178430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc00cb0e0 0x7fcbc01552e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T12:59:56.376 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 >> 172.21.15.153:0/2719178430 conn(0x7fcbc00b12b0 msgr2=0x7fcbc00b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T12:59:56.376 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 shutdown_connections 2023-11-28T12:59:56.376 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T12:59:56.374+0000 7fcbc6838700 1 -- 172.21.15.153:0/2719178430 wait complete. 2023-11-28T12:59:56.386 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T12:59:56.411 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:56.411+0000 7fdc580d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T12:59:56.501 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:56.500+0000 7ff7c362e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T12:59:56.543 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:56.542+0000 7fdc580d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T12:59:56.718 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:56.717+0000 7ff7c362e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T12:59:56.763 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:56.762+0000 7fdc580d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T12:59:56.874 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:56.874+0000 7ff7c362e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T12:59:56.920 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:56.919+0000 7fdc580d2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T12:59:56.988 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:56.987+0000 7ff7c362e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T12:59:57.035 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:57.033+0000 7fdc580d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T12:59:57.311 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:57.310+0000 7ff7c362e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T12:59:57.371 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:57.369+0000 7fdc580d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T12:59:57.410 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:57.409+0000 7ff7c362e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T12:59:57.471 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:57.470+0000 7fdc580d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T12:59:57.593 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:57.592+0000 7ff7c362e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T12:59:57.654 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:57.653+0000 7fdc580d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T12:59:58.114 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:58.113+0000 7ff7c362e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T12:59:58.173 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T12:59:58.172+0000 7fdc580d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T12:59:59.011 INFO:tasks.ceph.osd.0.smithi153.stderr:2023-11-28T12:59:59.011+0000 7f2d1d018640 -1 osd.0 0 log_to_monitors true 2023-11-28T12:59:59.138 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:59.138+0000 7ff77a85c700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T12:59:59.139 INFO:tasks.ceph.mgr.y.smithi153.stderr:2023-11-28T12:59:59.139+0000 7ff77a85c700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T12:59:59.252 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T12:59:59.251+0000 7f10f25e6640 -1 osd.5 0 log_to_monitors true 2023-11-28T12:59:59.272 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T12:59:59.271+0000 7f45dc913640 -1 osd.4 0 log_to_monitors true 2023-11-28T12:59:59.289 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T12:59:59.287+0000 7f81df1f4640 -1 osd.7 0 log_to_monitors true 2023-11-28T12:59:59.290 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T12:59:59.288+0000 7f918ec55640 -1 osd.6 0 log_to_monitors true 2023-11-28T12:59:59.310 INFO:tasks.ceph.osd.3.smithi153.stderr:2023-11-28T12:59:59.309+0000 7f80ea288640 -1 osd.3 0 log_to_monitors true 2023-11-28T12:59:59.325 INFO:tasks.ceph.osd.1.smithi153.stderr:2023-11-28T12:59:59.324+0000 7fd05997c640 -1 osd.1 0 log_to_monitors true 2023-11-28T12:59:59.327 INFO:tasks.ceph.osd.2.smithi153.stderr:2023-11-28T12:59:59.326+0000 7fb49b978640 -1 osd.2 0 log_to_monitors true 2023-11-28T13:00:01.232 INFO:tasks.ceph.osd.0.smithi153.stderr:2023-11-28T13:00:01.232+0000 7f2d10bcf700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T13:00:01.247 INFO:tasks.ceph.osd.0.smithi153.stderr:2023-11-28T13:00:01.246+0000 7f2d08faa700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:02.325 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T13:00:02.323+0000 7f918280c700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T13:00:02.335 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T13:00:02.330+0000 7f81d2dab700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T13:00:02.339 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T13:00:02.338+0000 7f917abe7700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:02.345 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T13:00:02.344+0000 7f81cb186700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:02.360 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T13:00:02.359+0000 7f45d04ca700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T13:00:02.366 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T13:00:02.364+0000 7f10e619d700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T13:00:02.374 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T13:00:02.373+0000 7f45c88a5700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:02.396 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T13:00:02.395+0000 7f10de578700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:03.192 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:03.200 INFO:tasks.ceph.osd.1.smithi153.stderr:2023-11-28T13:00:03.199+0000 7fd04d533700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T13:00:03.219 INFO:tasks.ceph.osd.1.smithi153.stderr:2023-11-28T13:00:03.219+0000 7fd04590e700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:03.227 INFO:tasks.ceph.osd.2.smithi153.stderr:2023-11-28T13:00:03.227+0000 7fb48f52f700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T13:00:03.242 INFO:tasks.ceph.osd.2.smithi153.stderr:2023-11-28T13:00:03.242+0000 7fb48790a700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:03.257 INFO:tasks.ceph.osd.3.smithi153.stderr:2023-11-28T13:00:03.256+0000 7f80dde3f700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T13:00:03.271 INFO:tasks.ceph.osd.3.smithi153.stderr:2023-11-28T13:00:03.271+0000 7f80d621a700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:03.317 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.314+0000 7fe016973700 1 Processor -- start 2023-11-28T13:00:03.317 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.314+0000 7fe016973700 1 -- start start 2023-11-28T13:00:03.317 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.314+0000 7fe016973700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe0101658c0 0x7fe010167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.317 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.314+0000 7fe016973700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01015d060 con 0x7fe01006dee0 2023-11-28T13:00:03.317 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.314+0000 7fe016973700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01015d340 con 0x7fe010071040 2023-11-28T13:00:03.317 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.314+0000 7fe016973700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe010161550 con 0x7fe0101658c0 2023-11-28T13:00:03.317 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe01006dee0 0x7fe01006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:55934/0 (socket says 172.21.15.153:55934) 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00f7fe700 1 -- 172.21.15.153:0/3692783775 learned_addr learned my addr 172.21.15.153:0/3692783775 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe014f10700 1 --2- 172.21.15.153:0/3692783775 >> v2:172.21.15.177:3300/0 conn(0x7fe0101658c0 0x7fe010167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582008135 0 0) 0x7fe01015d060 con 0x7fe01006dee0 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff80035b0 con 0x7fe01006dee0 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812003549 0 0) 0x7fe01015d340 con 0x7fe010071040 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe01015d060 con 0x7fe010071040 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe014f10700 1 -- 172.21.15.153:0/3692783775 >> v1:172.21.15.177:6789/0 conn(0x7fe010071040 legacy=0x7fe010071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe014f10700 1 -- 172.21.15.153:0/3692783775 >> v1:172.21.15.153:6789/0 conn(0x7fe01006dee0 legacy=0x7fe01006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4036500321 0 0) 0x7fdff80035b0 con 0x7fe01006dee0 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe014f10700 1 -- 172.21.15.153:0/3692783775 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01015d620 con 0x7fe0101658c0 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe014f10700 1 --2- 172.21.15.153:0/3692783775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101658c0 0x7fe010167ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eba891456117e99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:03.318 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800e070 con 0x7fe0101658c0 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008003a20 con 0x7fe0101658c0 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.315+0000 7fe00effd700 1 -- 172.21.15.153:0/3692783775 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004001670 con 0x7fe0101658c0 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.316+0000 7fe016973700 1 -- 172.21.15.153:0/3692783775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101658c0 msgr2=0x7fe010167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.316+0000 7fe016973700 1 --2- 172.21.15.153:0/3692783775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101658c0 0x7fe010167ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.316+0000 7fe016973700 1 -- 172.21.15.153:0/3692783775 shutdown_connections 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.316+0000 7fe016973700 1 --2- 172.21.15.153:0/3692783775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101658c0 0x7fe010167ca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.316+0000 7fe016973700 1 -- 172.21.15.153:0/3692783775 >> 172.21.15.153:0/3692783775 conn(0x7fe0100d0000 msgr2=0x7fe0100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.316+0000 7fe016973700 1 -- 172.21.15.153:0/3692783775 shutdown_connections 2023-11-28T13:00:03.319 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.316+0000 7fe016973700 1 -- 172.21.15.153:0/3692783775 wait complete. 2023-11-28T13:00:03.322 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.317+0000 7fe016973700 1 Processor -- start 2023-11-28T13:00:03.322 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.317+0000 7fe016973700 1 -- start start 2023-11-28T13:00:03.322 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe016973700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe01006dee0 0x7fe0100b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe016973700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe010071040 0x7fe010156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe016973700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0101658c0 0x7fe010156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe016973700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010157ea0 con 0x7fe0101658c0 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe016973700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100c5740 con 0x7fe01006dee0 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe016973700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100afaa0 con 0x7fe010071040 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0101658c0 0x7fe010156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0101658c0 0x7fe010156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37778/0 (socket says 172.21.15.153:37778) 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 -- 172.21.15.153:0/1276153860 learned_addr learned my addr 172.21.15.153:0/1276153860 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe00f7fe700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe010071040 0x7fe010156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe00ffff700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe01006dee0 0x7fe0100b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.323 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 -- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe010071040 msgr2=0x7fe010156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe010071040 0x7fe010156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 -- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe01006dee0 msgr2=0x7fe0100b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe01006dee0 0x7fe0100b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.318+0000 7fe014f10700 1 -- 172.21.15.153:0/1276153860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008009a70 con 0x7fe0101658c0 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.319+0000 7fe00ffff700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe01006dee0 0x7fe0100b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.319+0000 7fe014f10700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0101658c0 0x7fe010156d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe00800b370 tx=0x7fe008004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.319+0000 7fe00cff9700 1 -- 172.21.15.153:0/1276153860 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008004540 con 0x7fe0101658c0 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.319+0000 7fe00cff9700 1 -- 172.21.15.153:0/1276153860 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0080046a0 con 0x7fe0101658c0 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.319+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01015ccd0 con 0x7fe0101658c0 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.319+0000 7fe00cff9700 1 -- 172.21.15.153:0/1276153860 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008012430 con 0x7fe0101658c0 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.319+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01015d900 con 0x7fe0101658c0 2023-11-28T13:00:03.324 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.320+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01006a270 con 0x7fe0101658c0 2023-11-28T13:00:03.325 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.320+0000 7fe00cff9700 1 -- 172.21.15.153:0/1276153860 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fe008023030 con 0x7fe0101658c0 2023-11-28T13:00:03.325 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.320+0000 7fe00cff9700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdffc0682d0 0x7fdffc06a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.325 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.321+0000 7fe00cff9700 1 -- 172.21.15.153:0/1276153860 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3631+0+0 (secure 0 0 0) 0x7fe0080865e0 con 0x7fe0101658c0 2023-11-28T13:00:03.325 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.321+0000 7fe00ffff700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdffc0682d0 0x7fdffc06a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.325 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.321+0000 7fe00ffff700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdffc0682d0 0x7fdffc06a780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe000007fd0 tx=0x7fe000009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:03.325 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.325+0000 7fe00cff9700 1 -- 172.21.15.153:0/1276153860 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe008053810 con 0x7fe0101658c0 2023-11-28T13:00:03.509 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.509+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0100720a0 con 0x7fe0101658c0 2023-11-28T13:00:03.510 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.509+0000 7fe00cff9700 1 -- 172.21.15.153:0/1276153860 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9293 (secure 0 0 0) 0x7fe008016020 con 0x7fe0101658c0 2023-11-28T13:00:03.510 INFO:teuthology.misc.health.smithi153.stdout: 2023-11-28T13:00:03.510 INFO:teuthology.misc.health.smithi153.stdout:{"epoch":14,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:02.833215+0000","last_up_change":"2023-11-28T13:00:02.833215+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:03.513 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.511+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdffc0682d0 msgr2=0x7fdffc06a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.513 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.511+0000 7fe016973700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdffc0682d0 0x7fdffc06a780 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe000007fd0 tx=0x7fe000009040 comp rx=0 tx=0).stop 2023-11-28T13:00:03.513 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.511+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0101658c0 msgr2=0x7fe010156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.514 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.511+0000 7fe016973700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0101658c0 0x7fe010156d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe00800b370 tx=0x7fe008004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.514 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 shutdown_connections 2023-11-28T13:00:03.514 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdffc0682d0 0x7fdffc06a780 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.514 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0101658c0 0x7fe010156d90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.514 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe01006dee0 0x7fe0100b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.514 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 --2- 172.21.15.153:0/1276153860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe010071040 0x7fe010156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.514 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 >> 172.21.15.153:0/1276153860 conn(0x7fe0100d0000 msgr2=0x7fe0100cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:03.515 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 shutdown_connections 2023-11-28T13:00:03.515 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:03.512+0000 7fe016973700 1 -- 172.21.15.153:0/1276153860 wait complete. 2023-11-28T13:00:03.524 DEBUG:teuthology.misc:5 of 8 OSDs are up 2023-11-28T13:00:10.331 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:10.455 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.454+0000 7f7f18751700 1 Processor -- start 2023-11-28T13:00:10.455 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.454+0000 7f7f18751700 1 -- start start 2023-11-28T13:00:10.457 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.455+0000 7f7f18751700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f10071040 0x7f7f10071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.457 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.455+0000 7f7f18751700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7f1006dee0 0x7f7f1006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.457 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.455+0000 7f7f18751700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f10071950 con 0x7f7f1006cdc0 2023-11-28T13:00:10.457 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.455+0000 7f7f18751700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f10161550 con 0x7f7f1006dee0 2023-11-28T13:00:10.457 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.455+0000 7f7f18751700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1015d060 con 0x7f7f10071040 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f15cec700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7f1006dee0 0x7f7f1006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f164ed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f10071040 0x7f7f10071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f15cec700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7f1006dee0 0x7f7f1006e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:47464/0 (socket says 172.21.15.153:47464) 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f15cec700 1 -- 172.21.15.153:0/2041279413 learned_addr learned my addr 172.21.15.153:0/2041279413 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f164ed700 1 -- 172.21.15.153:0/2041279413 >> v2:172.21.15.177:3300/0 conn(0x7f7f1006dee0 msgr2=0x7f7f1006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f164ed700 1 --2- 172.21.15.153:0/2041279413 >> v2:172.21.15.177:3300/0 conn(0x7f7f1006dee0 0x7f7f1006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f164ed700 1 -- 172.21.15.153:0/2041279413 >> v1:172.21.15.177:6789/0 conn(0x7f7f1006cdc0 legacy=0x7f7f1006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f164ed700 1 -- 172.21.15.153:0/2041279413 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1015d340 con 0x7f7f10071040 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f15cec700 1 --2- 172.21.15.153:0/2041279413 >> v2:172.21.15.177:3300/0 conn(0x7f7f1006dee0 0x7f7f1006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.456+0000 7f7f164ed700 1 --2- 172.21.15.153:0/2041279413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f10071040 0x7f7f10071410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009840 tx=0x7f7f04009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b9a1c0136eb6c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f154eb700 1 -- 172.21.15.153:0/2041279413 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011830 con 0x7f7f10071040 2023-11-28T13:00:10.458 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f154eb700 1 -- 172.21.15.153:0/2041279413 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04011990 con 0x7f7f10071040 2023-11-28T13:00:10.459 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f154eb700 1 -- 172.21.15.153:0/2041279413 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011d20 con 0x7f7f10071040 2023-11-28T13:00:10.461 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 -- 172.21.15.153:0/2041279413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f10071040 msgr2=0x7f7f10071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.461 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 --2- 172.21.15.153:0/2041279413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f10071040 0x7f7f10071410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009840 tx=0x7f7f04009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:10.461 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 -- 172.21.15.153:0/2041279413 shutdown_connections 2023-11-28T13:00:10.461 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 --2- 172.21.15.153:0/2041279413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f10071040 0x7f7f10071410 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.461 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 --2- 172.21.15.153:0/2041279413 >> v2:172.21.15.177:3300/0 conn(0x7f7f1006dee0 0x7f7f1006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.461 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 -- 172.21.15.153:0/2041279413 >> 172.21.15.153:0/2041279413 conn(0x7f7f100d0000 msgr2=0x7f7f100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:10.461 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 -- 172.21.15.153:0/2041279413 shutdown_connections 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.457+0000 7f7f18751700 1 -- 172.21.15.153:0/2041279413 wait complete. 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.458+0000 7f7f18751700 1 Processor -- start 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f18751700 1 -- start start 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f18751700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1006cdc0 0x7f7f10152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f18751700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1006dee0 0x7f7f100b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f18751700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f10071040 0x7f7f100b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f18751700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100b2e30 con 0x7f7f10071040 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f18751700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100b39b0 con 0x7f7f1006dee0 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f18751700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100b3080 con 0x7f7f1006cdc0 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f16cee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f10071040 0x7f7f100b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.462 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f16cee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f10071040 0x7f7f100b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38006/0 (socket says 172.21.15.153:38006) 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.459+0000 7f7f16cee700 1 -- 172.21.15.153:0/2086234810 learned_addr learned my addr 172.21.15.153:0/2086234810 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f164ed700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1006cdc0 0x7f7f10152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f15cec700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1006dee0 0x7f7f100b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f16cee700 1 -- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1006cdc0 msgr2=0x7f7f10152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f16cee700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1006cdc0 0x7f7f10152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f16cee700 1 -- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1006dee0 msgr2=0x7f7f100b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f16cee700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1006dee0 0x7f7f100b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f16cee700 1 -- 172.21.15.153:0/2086234810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009550 con 0x7f7f10071040 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f16cee700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f10071040 0x7f7f100b5210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800cb50 tx=0x7f7f0800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f02ffd700 1 -- 172.21.15.153:0/2086234810 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08003c00 con 0x7f7f10071040 2023-11-28T13:00:10.463 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f02ffd700 1 -- 172.21.15.153:0/2086234810 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f08003d60 con 0x7f7f10071040 2023-11-28T13:00:10.464 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f02ffd700 1 -- 172.21.15.153:0/2086234810 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800aa20 con 0x7f7f10071040 2023-11-28T13:00:10.464 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f040094a0 con 0x7f7f10071040 2023-11-28T13:00:10.464 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.460+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1015d900 con 0x7f7f10071040 2023-11-28T13:00:10.464 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.461+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1015d620 con 0x7f7f10071040 2023-11-28T13:00:10.465 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.461+0000 7f7f02ffd700 1 -- 172.21.15.153:0/2086234810 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7f08010050 con 0x7f7f10071040 2023-11-28T13:00:10.465 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.462+0000 7f7f02ffd700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ef40682f0 0x7f7ef406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.465 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.462+0000 7f7f02ffd700 1 -- 172.21.15.153:0/2086234810 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7f080856c0 con 0x7f7f10071040 2023-11-28T13:00:10.465 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.462+0000 7f7f164ed700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ef40682f0 0x7f7ef406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.465 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.462+0000 7f7f164ed700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ef40682f0 0x7f7ef406a7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f040090b0 tx=0x7f7f04008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:10.467 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.466+0000 7f7f02ffd700 1 -- 172.21.15.153:0/2086234810 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7f08052350 con 0x7f7f10071040 2023-11-28T13:00:10.649 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.649+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f1006e890 con 0x7f7f10071040 2023-11-28T13:00:10.650 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.650+0000 7f7f02ffd700 1 -- 172.21.15.153:0/2086234810 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12526 (secure 0 0 0) 0x7f7f08055970 con 0x7f7f10071040 2023-11-28T13:00:10.650 INFO:teuthology.misc.health.smithi153.stdout: 2023-11-28T13:00:10.651 INFO:teuthology.misc.health.smithi153.stdout:{"epoch":18,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:06.867017+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ef40682f0 msgr2=0x7f7ef406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ef40682f0 0x7f7ef406a7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f040090b0 tx=0x7f7f04008030 comp rx=0 tx=0).stop 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f10071040 msgr2=0x7f7f100b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f10071040 0x7f7f100b5210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800cb50 tx=0x7f7f0800ce60 comp rx=0 tx=0).stop 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 shutdown_connections 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ef40682f0 0x7f7ef406a7a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f10071040 0x7f7f100b5210 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1006cdc0 0x7f7f10152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 --2- 172.21.15.153:0/2086234810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1006dee0 0x7f7f100b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 >> 172.21.15.153:0/2086234810 conn(0x7f7f100d0000 msgr2=0x7f7f1006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:10.653 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 shutdown_connections 2023-11-28T13:00:10.654 INFO:teuthology.misc.health.smithi153.stderr:2023-11-28T13:00:10.652+0000 7f7f18751700 1 -- 172.21.15.153:0/2086234810 wait complete. 2023-11-28T13:00:10.663 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T13:00:10.664 INFO:tasks.ceph:Creating RBD pool 2023-11-28T13:00:10.664 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T13:00:10.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.783+0000 7f060945e700 1 Processor -- start 2023-11-28T13:00:10.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.783+0000 7f060945e700 1 -- start start 2023-11-28T13:00:10.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f060945e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f060415cd30 0x7f0604161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f060945e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f060406a980 0x7f06040d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f060945e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040d0840 con 0x7f06040cfda0 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f060945e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040d0b20 con 0x7f060406a980 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f060945e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0604160670 con 0x7f060415cd30 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f06027fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f06040cfda0 0x7f060406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56174/0 (socket says 172.21.15.153:56174) 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f06027fc700 1 -- 172.21.15.153:0/1557625480 learned_addr learned my addr 172.21.15.153:0/1557625480 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f06037fe700 1 --2- 172.21.15.153:0/1557625480 >> v2:172.21.15.177:3300/0 conn(0x7f060406a980 0x7f06040d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f060415cd30 0x7f0604161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605548889 0 0) 0x7f06040d0840 con 0x7f06040cfda0 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec0035b0 con 0x7f06040cfda0 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1365946040 0 0) 0x7f05ec0035b0 con 0x7f06040cfda0 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f060415cd30 msgr2=0x7f0604161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f060415cd30 0x7f0604161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 >> v2:172.21.15.177:3300/0 conn(0x7f060406a980 msgr2=0x7f06040d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 --2- 172.21.15.153:0/1557625480 >> v2:172.21.15.177:3300/0 conn(0x7f060406a980 0x7f06040d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604160950 con 0x7f06040cfda0 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.784+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f05f4001670 con 0x7f06040cfda0 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 >> v1:172.21.15.153:6789/0 conn(0x7f06040cfda0 legacy=0x7f060406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ec003a00 0x7f05ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec004430 0x7f05ec0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec004f60 0x7f05ec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040d0b20 con 0x7f05ec003a00 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0604160670 con 0x7f05ec004430 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec00bf50 con 0x7f05ec004f60 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f060415cd30 0x7f0604161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:10.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f06037fe700 1 --2- 172.21.15.153:0/1557625480 >> v2:172.21.15.177:3300/0 conn(0x7f060406a980 0x7f06040d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ec003a00 0x7f05ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f06027fc700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec004430 0x7f05ec0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f06037fe700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec004f60 0x7f05ec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 -- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec004f60 msgr2=0x7f05ec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec004f60 0x7f05ec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 -- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec004430 msgr2=0x7f05ec0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec004430 0x7f05ec0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 -- 172.21.15.153:0/1557625480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0035b0 con 0x7f05ec003a00 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ec003a00 0x7f05ec003e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0604070be0 tx=0x7f05ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c8549e8f364bc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004030 con 0x7f05ec003a00 2023-11-28T13:00:10.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f8004190 con 0x7f05ec003a00 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.785+0000 7f0601ffb700 1 -- 172.21.15.153:0/1557625480 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004540 con 0x7f05ec003a00 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.786+0000 7f060945e700 1 -- 172.21.15.153:0/1557625480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ec003a00 msgr2=0x7f05ec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.786+0000 7f060945e700 1 --2- 172.21.15.153:0/1557625480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05ec003a00 0x7f05ec003e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0604070be0 tx=0x7f05ec005880 comp rx=0 tx=0).stop 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.786+0000 7f06037fe700 1 -- 172.21.15.153:0/1557625480 reap_dead start 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.786+0000 7f060945e700 1 -- 172.21.15.153:0/1557625480 shutdown_connections 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.786+0000 7f060945e700 1 -- 172.21.15.153:0/1557625480 >> 172.21.15.153:0/1557625480 conn(0x7f060406b380 msgr2=0x7f060406b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.786+0000 7f060945e700 1 -- 172.21.15.153:0/1557625480 shutdown_connections 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.786+0000 7f060945e700 1 -- 172.21.15.153:0/1557625480 wait complete. 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.787+0000 7f060945e700 1 Processor -- start 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.787+0000 7f060945e700 1 -- start start 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f060945e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec003a00 0x7f06040b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f060945e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f060414e2c0 0x7f06040c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f060945e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06040c40d0 0x7f060415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f060945e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06041f2f50 con 0x7f06040c40d0 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f060945e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040b4a60 con 0x7f060414e2c0 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f060945e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06041f2340 con 0x7f05ec003a00 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06040c40d0 0x7f060415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06040c40d0 0x7f060415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38024/0 (socket says 172.21.15.153:38024) 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 -- 172.21.15.153:0/1925362280 learned_addr learned my addr 172.21.15.153:0/1925362280 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06027fc700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f060414e2c0 0x7f06040c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 -- 172.21.15.153:0/1925362280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec003a00 msgr2=0x7f06040b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec003a00 0x7f06040b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 -- 172.21.15.153:0/1925362280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f060414e2c0 msgr2=0x7f06040c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:10.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f060414e2c0 0x7f06040c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 -- 172.21.15.153:0/1925362280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040cc280 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.788+0000 7f06037fe700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06040c40d0 0x7f060415f510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06040d0e20 tx=0x7f05fc00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.789+0000 7f05eb7fe700 1 -- 172.21.15.153:0/1925362280 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00e040 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.789+0000 7f05eb7fe700 1 -- 172.21.15.153:0/1925362280 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00f040 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.789+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0604160c30 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.789+0000 7f05eb7fe700 1 -- 172.21.15.153:0/1925362280 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc003a20 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.789+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0604161150 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.790+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06040745c0 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.790+0000 7f05eb7fe700 1 -- 172.21.15.153:0/1925362280 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05fc024070 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.790+0000 7f05eb7fe700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05f00682f0 0x7f05f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.790+0000 7f05eb7fe700 1 -- 172.21.15.153:0/1925362280 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05fc085e30 con 0x7f06040c40d0 2023-11-28T13:00:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.790+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05f00682f0 0x7f05f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:10.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.791+0000 7f0602ffd700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05f00682f0 0x7f05f006a7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05f8016040 tx=0x7f05f8013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:10.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.795+0000 7f05eb7fe700 1 -- 172.21.15.153:0/1925362280 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05fc052b40 con 0x7f06040c40d0 2023-11-28T13:00:10.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:10.978+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f06040cd050 con 0x7f06040c40d0 2023-11-28T13:00:11.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.194+0000 7f05eb7fe700 1 -- 172.21.15.153:0/1925362280 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v19) v1 ==== 109+0+0 (secure 0 0 0) 0x7f05fc056160 con 0x7f06040c40d0 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.196+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05f00682f0 msgr2=0x7f05f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.196+0000 7f060945e700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05f00682f0 0x7f05f006a7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05f8016040 tx=0x7f05f8013000 comp rx=0 tx=0).stop 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.196+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06040c40d0 msgr2=0x7f060415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.196+0000 7f060945e700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06040c40d0 0x7f060415f510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06040d0e20 tx=0x7f05fc00f490 comp rx=0 tx=0).stop 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 shutdown_connections 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05f00682f0 0x7f05f006a7a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06040c40d0 0x7f060415f510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05ec003a00 0x7f06040b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 --2- 172.21.15.153:0/1925362280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f060414e2c0 0x7f06040c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 >> 172.21.15.153:0/1925362280 conn(0x7f060406b380 msgr2=0x7f06040ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:11.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 shutdown_connections 2023-11-28T13:00:11.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.197+0000 7f060945e700 1 -- 172.21.15.153:0/1925362280 wait complete. 2023-11-28T13:00:11.199 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'rbd' created 2023-11-28T13:00:11.210 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T13:00:11.340 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.331+0000 7f3a32906700 1 Processor -- start 2023-11-28T13:00:11.340 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.331+0000 7f3a32906700 1 -- start start 2023-11-28T13:00:11.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a32906700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 0x7f3a2c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a32906700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3a2c0cc260 0x7f3a2c06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a32906700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c0cc7b0 con 0x7f3a2c1596b0 2023-11-28T13:00:11.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a32906700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cd610 con 0x7f3a2c0cc260 2023-11-28T13:00:11.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a32906700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c159a80 con 0x7f3a2c0d0470 2023-11-28T13:00:11.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3a2c1596b0 0x7f3a2c0cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56192/0 (socket says 172.21.15.153:56192) 2023-11-28T13:00:11.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2bfff700 1 -- 172.21.15.153:0/2742716893 learned_addr learned my addr 172.21.15.153:0/2742716893 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:11.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a30ea3700 1 --2- 172.21.15.153:0/2742716893 >> v2:172.21.15.177:3300/0 conn(0x7f3a2c0cc260 0x7f3a2c06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 0x7f3a2c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837630348 0 0) 0x7f3a2c0cc7b0 con 0x7f3a2c1596b0 2023-11-28T13:00:11.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a140035b0 con 0x7f3a2c1596b0 2023-11-28T13:00:11.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1840752718 0 0) 0x7f3a140035b0 con 0x7f3a2c1596b0 2023-11-28T13:00:11.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 msgr2=0x7f3a2c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:11.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 0x7f3a2c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 >> v2:172.21.15.177:3300/0 conn(0x7f3a2c0cc260 msgr2=0x7f3a2c06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 --2- 172.21.15.153:0/2742716893 >> v2:172.21.15.177:3300/0 conn(0x7f3a2c0cc260 0x7f3a2c06cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0cca00 con 0x7f3a2c1596b0 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f3a20001670 con 0x7f3a2c1596b0 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.332+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 >> v1:172.21.15.153:6789/0 conn(0x7f3a2c1596b0 legacy=0x7f3a2c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14003a00 0x7f3a14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14004430 0x7f3a140048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 0x7f3a14004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cd610 con 0x7f3a14004430 2023-11-28T13:00:11.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c159a80 con 0x7f3a14003a00 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1400dc20 con 0x7f3a2c0d0470 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a30ea3700 1 --2- 172.21.15.153:0/2742716893 >> v2:172.21.15.177:3300/0 conn(0x7f3a2c0cc260 0x7f3a2c06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14004430 0x7f3a140048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2bfff700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14003a00 0x7f3a14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a30ea3700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 0x7f3a14004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2b7fe700 1 -- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 msgr2=0x7f3a14004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c0d0470 0x7f3a14004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2b7fe700 1 -- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14003a00 msgr2=0x7f3a14003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14003a00 0x7f3a14003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2b7fe700 1 -- 172.21.15.153:0/2742716893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140035b0 con 0x7f3a14004430 2023-11-28T13:00:11.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14004430 0x7f3a140048f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009b30 tx=0x7f3a14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a55ac18140a8fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00e6c0 con 0x7f3a14004430 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c00e820 con 0x7f3a14004430 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.333+0000 7f3a2affd700 1 -- 172.21.15.153:0/2742716893 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00eb90 con 0x7f3a14004430 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.334+0000 7f3a32906700 1 -- 172.21.15.153:0/2742716893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14004430 msgr2=0x7f3a140048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.334+0000 7f3a32906700 1 --2- 172.21.15.153:0/2742716893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a14004430 0x7f3a140048f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009b30 tx=0x7f3a14005880 comp rx=0 tx=0).stop 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.334+0000 7f3a30ea3700 1 -- 172.21.15.153:0/2742716893 reap_dead start 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.334+0000 7f3a32906700 1 -- 172.21.15.153:0/2742716893 shutdown_connections 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.334+0000 7f3a32906700 1 -- 172.21.15.153:0/2742716893 >> 172.21.15.153:0/2742716893 conn(0x7f3a2c06b380 msgr2=0x7f3a2c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.334+0000 7f3a32906700 1 -- 172.21.15.153:0/2742716893 shutdown_connections 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.334+0000 7f3a32906700 1 -- 172.21.15.153:0/2742716893 wait complete. 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.335+0000 7f3a32906700 1 Processor -- start 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.336+0000 7f3a32906700 1 -- start start 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.336+0000 7f3a32906700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14004430 0x7f3a2c1ed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.336+0000 7f3a32906700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a2c155090 0x7f3a2c0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.336+0000 7f3a32906700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c155500 0x7f3a2c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.336+0000 7f3a32906700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c14f160 con 0x7f3a2c155090 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.336+0000 7f3a32906700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c1f17d0 con 0x7f3a14004430 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.336+0000 7f3a32906700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c14eae0 con 0x7f3a2c155500 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a2c155090 0x7f3a2c0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a2c155090 0x7f3a2c0cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38042/0 (socket says 172.21.15.153:38042) 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 -- 172.21.15.153:0/2772284654 learned_addr learned my addr 172.21.15.153:0/2772284654 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2bfff700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14004430 0x7f3a2c1ed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a30ea3700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c155500 0x7f3a2c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 -- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c155500 msgr2=0x7f3a2c0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c155500 0x7f3a2c0cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 -- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14004430 msgr2=0x7f3a2c1ed6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14004430 0x7f3a2c1ed6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 -- 172.21.15.153:0/2772284654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0ccfc0 con 0x7f3a2c155090 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a2b7fe700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a2c155090 0x7f3a2c0cb510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a140056d0 tx=0x7f3a1c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a28ff9700 1 -- 172.21.15.153:0/2772284654 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00e6c0 con 0x7f3a2c155090 2023-11-28T13:00:11.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a28ff9700 1 -- 172.21.15.153:0/2772284654 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c0038c0 con 0x7f3a2c155090 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a28ff9700 1 -- 172.21.15.153:0/2772284654 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c003c70 con 0x7f3a2c155090 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c0ccce0 con 0x7f3a2c155090 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.337+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c0cd2a0 con 0x7f3a2c155090 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.338+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c068690 con 0x7f3a2c155090 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.339+0000 7f3a28ff9700 1 -- 172.21.15.153:0/2772284654 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3a1c019030 con 0x7f3a2c155090 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.339+0000 7f3a28ff9700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a180682f0 0x7f3a1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.339+0000 7f3a28ff9700 1 -- 172.21.15.153:0/2772284654 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f3a1c086e90 con 0x7f3a2c155090 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.339+0000 7f3a2bfff700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a180682f0 0x7f3a1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.339+0000 7f3a2bfff700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a180682f0 0x7f3a1806a7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3a20000fd0 tx=0x7f3a20001c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:11.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.344+0000 7f3a28ff9700 1 -- 172.21.15.153:0/2772284654 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3a1c053a50 con 0x7f3a2c155090 2023-11-28T13:00:11.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:11.528+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3a2c0711f0 con 0x7f3a2c155090 2023-11-28T13:00:12.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.200+0000 7f3a28ff9700 1 -- 172.21.15.153:0/2772284654 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v20) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3a1c057070 con 0x7f3a2c155090 2023-11-28T13:00:12.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a180682f0 msgr2=0x7f3a1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a180682f0 0x7f3a1806a7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3a20000fd0 tx=0x7f3a20001c10 comp rx=0 tx=0).stop 2023-11-28T13:00:12.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a2c155090 msgr2=0x7f3a2c0cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a2c155090 0x7f3a2c0cb510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a140056d0 tx=0x7f3a1c0047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 shutdown_connections 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a2c155500 0x7f3a2c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a14004430 0x7f3a2c1ed6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a180682f0 0x7f3a1806a7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 --2- 172.21.15.153:0/2772284654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a2c155090 0x7f3a2c0cb510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 >> 172.21.15.153:0/2772284654 conn(0x7f3a2c06b380 msgr2=0x7f3a2c073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 shutdown_connections 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.204+0000 7f3a32906700 1 -- 172.21.15.153:0/2772284654 wait complete. 2023-11-28T13:00:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T13:00:12.221 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T13:00:12.221 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:00:12.221 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:00:12.224 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:00:12.224 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:00:12.225 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:00:12.227 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:00:12.227 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:00:12.227 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:00:12.231 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:00:12.231 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:00:12.231 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:00:12.233 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:00:12.234 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T13:00:12.234 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:12.309 INFO:tasks.ceph.mds.a.smithi153.stdout:starting mds.a at 2023-11-28T13:00:12.309 INFO:tasks.ceph.mds.c.smithi153.stdout:starting mds.c at 2023-11-28T13:00:12.309 INFO:tasks.ceph.mds.c.smithi153.stderr:2023-11-28T13:00:12.309+0000 7ff9a7eff700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T13:00:12.321 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T13:00:12.321 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T13:00:12.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.366+0000 7f03a8a63700 1 Processor -- start 2023-11-28T13:00:12.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.366+0000 7f03a8a63700 1 -- start start 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a8a63700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a40b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a8a63700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40d1950 con 0x7f03a40cd7c0 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a8a63700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40d24f0 con 0x7f03a4161340 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a8a63700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40b5800 con 0x7f03a415c9b0 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a2d9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f03a40cd7c0 0x7f03a40b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56240/0 (socket says 172.21.15.153:56240) 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a2d9d700 1 -- 172.21.15.153:0/3556165764 learned_addr learned my addr 172.21.15.153:0/3556165764 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a259c700 1 --2- 172.21.15.153:0/3556165764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a40b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a159a700 1 -- 172.21.15.153:0/3556165764 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310458898 0 0) 0x7f03a40d1950 con 0x7f03a40cd7c0 2023-11-28T13:00:12.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a159a700 1 -- 172.21.15.153:0/3556165764 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f038c0035b0 con 0x7f03a40cd7c0 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a159a700 1 -- 172.21.15.153:0/3556165764 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3130983537 0 0) 0x7f03a40d24f0 con 0x7f03a4161340 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a159a700 1 -- 172.21.15.153:0/3556165764 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a40d1950 con 0x7f03a4161340 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a259c700 1 -- 172.21.15.153:0/3556165764 >> v1:172.21.15.177:6789/0 conn(0x7f03a4161340 legacy=0x7f03a40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a259c700 1 -- 172.21.15.153:0/3556165764 >> v1:172.21.15.153:6789/0 conn(0x7f03a40cd7c0 legacy=0x7f03a40b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a259c700 1 -- 172.21.15.153:0/3556165764 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40b5520 con 0x7f03a415c9b0 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a259c700 1 --2- 172.21.15.153:0/3556165764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a40b4bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0394009870 tx=0x7f0394009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593c1bda0da1fd25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a159a700 1 -- 172.21.15.153:0/3556165764 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394011830 con 0x7f03a415c9b0 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a159a700 1 -- 172.21.15.153:0/3556165764 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394011990 con 0x7f03a415c9b0 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.367+0000 7f03a159a700 1 -- 172.21.15.153:0/3556165764 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394011d20 con 0x7f03a415c9b0 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.368+0000 7f03a8a63700 1 -- 172.21.15.153:0/3556165764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 msgr2=0x7f03a40b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.368+0000 7f03a8a63700 1 --2- 172.21.15.153:0/3556165764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a40b4bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0394009870 tx=0x7f0394009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:12.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.368+0000 7f03a8a63700 1 -- 172.21.15.153:0/3556165764 shutdown_connections 2023-11-28T13:00:12.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.368+0000 7f03a8a63700 1 --2- 172.21.15.153:0/3556165764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a40b4bf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.368+0000 7f03a8a63700 1 -- 172.21.15.153:0/3556165764 >> 172.21.15.153:0/3556165764 conn(0x7f03a4072600 msgr2=0x7f03a40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:12.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.368+0000 7f03a8a63700 1 -- 172.21.15.153:0/3556165764 shutdown_connections 2023-11-28T13:00:12.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.368+0000 7f03a8a63700 1 -- 172.21.15.153:0/3556165764 wait complete. 2023-11-28T13:00:12.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.369+0000 7f03a8a63700 1 Processor -- start 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.369+0000 7f03a8a63700 1 -- start start 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a8a63700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a40cd7c0 0x7f03a41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a8a63700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a41f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a8a63700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a4161340 0x7f03a41e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a8a63700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4157f20 con 0x7f03a4161340 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a8a63700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a41f4370 con 0x7f03a40cd7c0 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a8a63700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40c8b90 con 0x7f03a415c9b0 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a2d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a4161340 0x7f03a41e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a2d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a4161340 0x7f03a41e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38082/0 (socket says 172.21.15.153:38082) 2023-11-28T13:00:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a2d9d700 1 -- 172.21.15.153:0/4290542303 learned_addr learned my addr 172.21.15.153:0/4290542303 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a259c700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a40cd7c0 0x7f03a41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.370+0000 7f03a1d9b700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a41f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a2d9d700 1 -- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 msgr2=0x7f03a41f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a2d9d700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a41f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a2d9d700 1 -- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a40cd7c0 msgr2=0x7f03a41579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a2d9d700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a40cd7c0 0x7f03a41579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a2d9d700 1 -- 172.21.15.153:0/4290542303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940094a0 con 0x7f03a4161340 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a2d9d700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a4161340 0x7f03a41e6440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f039c00caf0 tx=0x7f039c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f0392ffd700 1 -- 172.21.15.153:0/4290542303 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00a9e0 con 0x7f03a4161340 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f0392ffd700 1 -- 172.21.15.153:0/4290542303 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c00ab40 con 0x7f03a4161340 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a40d15e0 con 0x7f03a4161340 2023-11-28T13:00:12.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f0392ffd700 1 -- 172.21.15.153:0/4290542303 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c012400 con 0x7f03a4161340 2023-11-28T13:00:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.371+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a40d1300 con 0x7f03a4161340 2023-11-28T13:00:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.372+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a40b5130 con 0x7f03a4161340 2023-11-28T13:00:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.372+0000 7f0392ffd700 1 -- 172.21.15.153:0/4290542303 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f039c019030 con 0x7f03a4161340 2023-11-28T13:00:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.373+0000 7f0392ffd700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f03840682f0 0x7f038406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.373+0000 7f0392ffd700 1 -- 172.21.15.153:0/4290542303 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f039c085430 con 0x7f03a4161340 2023-11-28T13:00:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.373+0000 7f03a259c700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f03840682f0 0x7f038406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.373+0000 7f03a259c700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f03840682f0 0x7f038406a7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03940097c0 tx=0x7f039401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.377+0000 7f0392ffd700 1 -- 172.21.15.153:0/4290542303 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f039c051fe0 con 0x7f03a4161340 2023-11-28T13:00:12.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.563+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03a40745c0 con 0x7f03a4161340 2023-11-28T13:00:12.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.563+0000 7f0392ffd700 1 -- 172.21.15.153:0/4290542303 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14325 (secure 0 0 0) 0x7f039c055600 con 0x7f03a4161340 2023-11-28T13:00:12.564 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:12.564 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":20,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:12.193210+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f03840682f0 msgr2=0x7f038406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f03840682f0 0x7f038406a7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03940097c0 tx=0x7f039401c040 comp rx=0 tx=0).stop 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a4161340 msgr2=0x7f03a41e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a4161340 0x7f03a41e6440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f039c00caf0 tx=0x7f039c00ce00 comp rx=0 tx=0).stop 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 shutdown_connections 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f03840682f0 0x7f038406a7a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a4161340 0x7f03a41e6440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a40cd7c0 0x7f03a41579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 --2- 172.21.15.153:0/4290542303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a415c9b0 0x7f03a41f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 >> 172.21.15.153:0/4290542303 conn(0x7f03a4072600 msgr2=0x7f03a40b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:12.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 shutdown_connections 2023-11-28T13:00:12.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.566+0000 7f03a8a63700 1 -- 172.21.15.153:0/4290542303 wait complete. 2023-11-28T13:00:12.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:12.579 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:12.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.702+0000 7f8558588700 1 Processor -- start 2023-11-28T13:00:12.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.702+0000 7f8558588700 1 -- start start 2023-11-28T13:00:12.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8558588700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855015cef0 0x7f85500af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8558588700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500cc5f0 con 0x7f855006ba90 2023-11-28T13:00:12.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8558588700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855015d470 con 0x7f85500cbdf0 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8558588700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855015d750 con 0x7f855015cef0 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8556b25700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f855006ba90 0x7f85500ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56254/0 (socket says 172.21.15.153:56254) 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8556b25700 1 -- 172.21.15.153:0/2072921101 learned_addr learned my addr 172.21.15.153:0/2072921101 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8556324700 1 --2- 172.21.15.153:0/2072921101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855015cef0 0x7f85500af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8555322700 1 -- 172.21.15.153:0/2072921101 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373723047 0 0) 0x7f85500cc5f0 con 0x7f855006ba90 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8555322700 1 -- 172.21.15.153:0/2072921101 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f853c0035b0 con 0x7f855006ba90 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8555322700 1 -- 172.21.15.153:0/2072921101 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571850568 0 0) 0x7f855015d470 con 0x7f85500cbdf0 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8555322700 1 -- 172.21.15.153:0/2072921101 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85500cc5f0 con 0x7f85500cbdf0 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8556324700 1 -- 172.21.15.153:0/2072921101 >> v1:172.21.15.177:6789/0 conn(0x7f85500cbdf0 legacy=0x7f855006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8556324700 1 -- 172.21.15.153:0/2072921101 >> v1:172.21.15.153:6789/0 conn(0x7f855006ba90 legacy=0x7f85500ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8556324700 1 -- 172.21.15.153:0/2072921101 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855015da30 con 0x7f855015cef0 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.703+0000 7f8556324700 1 --2- 172.21.15.153:0/2072921101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855015cef0 0x7f85500af400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f854c009870 tx=0x7f854c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3985e743a60d63f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8555322700 1 -- 172.21.15.153:0/2072921101 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c011830 con 0x7f855015cef0 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8555322700 1 -- 172.21.15.153:0/2072921101 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c011990 con 0x7f855015cef0 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8555322700 1 -- 172.21.15.153:0/2072921101 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c011d00 con 0x7f855015cef0 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8558588700 1 -- 172.21.15.153:0/2072921101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855015cef0 msgr2=0x7f85500af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8558588700 1 --2- 172.21.15.153:0/2072921101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855015cef0 0x7f85500af400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f854c009870 tx=0x7f854c009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8558588700 1 -- 172.21.15.153:0/2072921101 shutdown_connections 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8558588700 1 --2- 172.21.15.153:0/2072921101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855015cef0 0x7f85500af400 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8558588700 1 -- 172.21.15.153:0/2072921101 >> 172.21.15.153:0/2072921101 conn(0x7f85500d0000 msgr2=0x7f85500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8558588700 1 -- 172.21.15.153:0/2072921101 shutdown_connections 2023-11-28T13:00:12.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.704+0000 7f8558588700 1 -- 172.21.15.153:0/2072921101 wait complete. 2023-11-28T13:00:12.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.705+0000 7f8558588700 1 Processor -- start 2023-11-28T13:00:12.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.705+0000 7f8558588700 1 -- start start 2023-11-28T13:00:12.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8558588700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855006ba90 0x7f85500c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8558588700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cbdf0 0x7f8550158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8558588700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855015cef0 0x7f855014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8558588700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500b2ec0 con 0x7f855015cef0 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8558588700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855014d9b0 con 0x7f85500cbdf0 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8558588700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500c9670 con 0x7f855006ba90 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556b25700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855015cef0 0x7f855014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556b25700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855015cef0 0x7f855014d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38096/0 (socket says 172.21.15.153:38096) 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556b25700 1 -- 172.21.15.153:0/1825221069 learned_addr learned my addr 172.21.15.153:0/1825221069 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8555b23700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cbdf0 0x7f8550158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556324700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855006ba90 0x7f85500c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556b25700 1 -- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855006ba90 msgr2=0x7f85500c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556b25700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855006ba90 0x7f85500c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556b25700 1 -- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cbdf0 msgr2=0x7f8550158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.706+0000 7f8556b25700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cbdf0 0x7f8550158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.707+0000 7f8556b25700 1 -- 172.21.15.153:0/1825221069 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0094a0 con 0x7f855015cef0 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.707+0000 7f8556b25700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855015cef0 0x7f855014d470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f854800cba0 tx=0x7f854800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.707+0000 7f8546ffd700 1 -- 172.21.15.153:0/1825221069 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800aa40 con 0x7f855015cef0 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.707+0000 7f8546ffd700 1 -- 172.21.15.153:0/1825221069 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854800aba0 con 0x7f855015cef0 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.707+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855015dff0 con 0x7f855015cef0 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.707+0000 7f8546ffd700 1 -- 172.21.15.153:0/1825221069 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548012400 con 0x7f855015cef0 2023-11-28T13:00:12.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.707+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855015dd10 con 0x7f855015cef0 2023-11-28T13:00:12.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.708+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85500cad60 con 0x7f855015cef0 2023-11-28T13:00:12.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.708+0000 7f8546ffd700 1 -- 172.21.15.153:0/1825221069 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8548019030 con 0x7f855015cef0 2023-11-28T13:00:12.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.708+0000 7f8546ffd700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85340682f0 0x7f853406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:12.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.709+0000 7f8546ffd700 1 -- 172.21.15.153:0/1825221069 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8548085480 con 0x7f855015cef0 2023-11-28T13:00:12.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.709+0000 7f8556324700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85340682f0 0x7f853406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:12.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.709+0000 7f8556324700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85340682f0 0x7f853406a7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f854c0090b0 tx=0x7f854c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:12.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.713+0000 7f8546ffd700 1 -- 172.21.15.153:0/1825221069 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8548052030 con 0x7f855015cef0 2023-11-28T13:00:12.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.894+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85500b3000 con 0x7f855015cef0 2023-11-28T13:00:12.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.895+0000 7f8546ffd700 1 -- 172.21.15.153:0/1825221069 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8548055650 con 0x7f855015cef0 2023-11-28T13:00:12.896 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85340682f0 msgr2=0x7f853406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85340682f0 0x7f853406a7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f854c0090b0 tx=0x7f854c008030 comp rx=0 tx=0).stop 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855015cef0 msgr2=0x7f855014d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855015cef0 0x7f855014d470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f854800cba0 tx=0x7f854800ceb0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 shutdown_connections 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85340682f0 0x7f853406a7a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f855015cef0 0x7f855014d470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855006ba90 0x7f85500c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 --2- 172.21.15.153:0/1825221069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85500cbdf0 0x7f8550158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:12.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 >> 172.21.15.153:0/1825221069 conn(0x7f85500d0000 msgr2=0x7f85500cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:12.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 shutdown_connections 2023-11-28T13:00:12.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:12.897+0000 7f8558588700 1 -- 172.21.15.153:0/1825221069 wait complete. 2023-11-28T13:00:12.909 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.032+0000 7f30a9f60700 1 Processor -- start 2023-11-28T13:00:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.032+0000 7f30a9f60700 1 -- start start 2023-11-28T13:00:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a9f60700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f30a4071040 0x7f30a4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a9f60700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f30a4160540 0x7f30a4160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a9f60700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a4071950 con 0x7f30a406dee0 2023-11-28T13:00:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a9f60700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a415d060 con 0x7f30a4160540 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a9f60700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a415d340 con 0x7f30a4071040 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f30a4160540 0x7f30a4160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f30a4160540 0x7f30a4160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38104/0 (socket says 172.21.15.153:38104) 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a3fff700 1 -- 172.21.15.153:0/1299087858 learned_addr learned my addr 172.21.15.153:0/1299087858 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a37fe700 1 --2- 172.21.15.153:0/1299087858 >> v2:172.21.15.177:3300/0 conn(0x7f30a4071040 0x7f30a4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a3fff700 1 -- 172.21.15.153:0/1299087858 >> v2:172.21.15.177:3300/0 conn(0x7f30a4071040 msgr2=0x7f30a4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a3fff700 1 --2- 172.21.15.153:0/1299087858 >> v2:172.21.15.177:3300/0 conn(0x7f30a4071040 0x7f30a4071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a3fff700 1 -- 172.21.15.153:0/1299087858 >> v1:172.21.15.177:6789/0 conn(0x7f30a406dee0 legacy=0x7f30a406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.033+0000 7f30a3fff700 1 -- 172.21.15.153:0/1299087858 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a415d620 con 0x7f30a4160540 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a3fff700 1 --2- 172.21.15.153:0/1299087858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4160540 0x7f30a4160980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30a40d0f50 tx=0x7f309c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b96077b1f35bd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a2ffd700 1 -- 172.21.15.153:0/1299087858 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00e070 con 0x7f30a4160540 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a2ffd700 1 -- 172.21.15.153:0/1299087858 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c003a20 con 0x7f30a4160540 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a2ffd700 1 -- 172.21.15.153:0/1299087858 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c003dd0 con 0x7f30a4160540 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 -- 172.21.15.153:0/1299087858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4160540 msgr2=0x7f30a4160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 --2- 172.21.15.153:0/1299087858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4160540 0x7f30a4160980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30a40d0f50 tx=0x7f309c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 -- 172.21.15.153:0/1299087858 shutdown_connections 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 --2- 172.21.15.153:0/1299087858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4160540 0x7f30a4160980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 --2- 172.21.15.153:0/1299087858 >> v2:172.21.15.177:3300/0 conn(0x7f30a4071040 0x7f30a4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 -- 172.21.15.153:0/1299087858 >> 172.21.15.153:0/1299087858 conn(0x7f30a40d0000 msgr2=0x7f30a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 -- 172.21.15.153:0/1299087858 shutdown_connections 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.034+0000 7f30a9f60700 1 -- 172.21.15.153:0/1299087858 wait complete. 2023-11-28T13:00:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.035+0000 7f30a9f60700 1 Processor -- start 2023-11-28T13:00:13.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.036+0000 7f30a9f60700 1 -- start start 2023-11-28T13:00:13.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.036+0000 7f30a9f60700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30a406dee0 0x7f30a40c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.036+0000 7f30a9f60700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4071040 0x7f30a41f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.036+0000 7f30a9f60700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30a4160540 0x7f30a40ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.036+0000 7f30a9f60700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40c4a30 con 0x7f30a4071040 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.036+0000 7f30a9f60700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40c3aa0 con 0x7f30a4160540 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.036+0000 7f30a9f60700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a41f7280 con 0x7f30a406dee0 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4071040 0x7f30a41f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4071040 0x7f30a41f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38110/0 (socket says 172.21.15.153:38110) 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 -- 172.21.15.153:0/2608136755 learned_addr learned my addr 172.21.15.153:0/2608136755 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f30a3fff700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30a4160540 0x7f30a40ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f30a37fe700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30a406dee0 0x7f30a40c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 -- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30a406dee0 msgr2=0x7f30a40c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30a406dee0 0x7f30a40c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 -- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30a4160540 msgr2=0x7f30a40ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30a4160540 0x7f30a40ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 -- 172.21.15.153:0/2608136755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c009b30 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f309affd700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4071040 0x7f30a41f6d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b470 tx=0x7f308c00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f30a0ff9700 1 -- 172.21.15.153:0/2608136755 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c011830 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f30a0ff9700 1 -- 172.21.15.153:0/2608136755 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c011990 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f30a0ff9700 1 -- 172.21.15.153:0/2608136755 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c011d40 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a415ccd0 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.037+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a415d900 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.038+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a406a270 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.039+0000 7f30a0ff9700 1 -- 172.21.15.153:0/2608136755 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f308c021070 con 0x7f30a4071040 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.039+0000 7f30a0ff9700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f30840682f0 0x7f308406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.039+0000 7f30a0ff9700 1 -- 172.21.15.153:0/2608136755 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f308c085860 con 0x7f30a4071040 2023-11-28T13:00:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.039+0000 7f30a37fe700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f30840682f0 0x7f308406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.040+0000 7f30a37fe700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f30840682f0 0x7f308406a7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f309400a950 tx=0x7f3094008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.043+0000 7f30a0ff9700 1 -- 172.21.15.153:0/2608136755 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f308c052410 con 0x7f30a4071040 2023-11-28T13:00:13.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.226+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f30a4066a20 con 0x7f30a4071040 2023-11-28T13:00:13.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.227+0000 7f30a0ff9700 1 -- 172.21.15.153:0/2608136755 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f308c015020 con 0x7f30a4071040 2023-11-28T13:00:13.228 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f30840682f0 msgr2=0x7f308406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f30840682f0 0x7f308406a7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f309400a950 tx=0x7f3094008040 comp rx=0 tx=0).stop 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4071040 msgr2=0x7f30a41f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4071040 0x7f30a41f6d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b470 tx=0x7f308c00b830 comp rx=0 tx=0).stop 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 shutdown_connections 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30a4160540 0x7f30a40ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30a406dee0 0x7f30a40c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f30840682f0 0x7f308406a7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 --2- 172.21.15.153:0/2608136755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30a4071040 0x7f30a41f6d40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 >> 172.21.15.153:0/2608136755 conn(0x7f30a40d0000 msgr2=0x7f30a4073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 shutdown_connections 2023-11-28T13:00:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.229+0000 7f30a9f60700 1 -- 172.21.15.153:0/2608136755 wait complete. 2023-11-28T13:00:13.242 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:13.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.363+0000 7fe87f07b700 1 Processor -- start 2023-11-28T13:00:13.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.363+0000 7fe87f07b700 1 -- start start 2023-11-28T13:00:13.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87f07b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe878071040 0x7fe878071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87f07b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe878160540 0x7fe8781609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87f07b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe878071950 con 0x7fe87806dee0 2023-11-28T13:00:13.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87f07b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87815d060 con 0x7fe878071040 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87f07b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87815d340 con 0x7fe878160540 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe878071040 0x7fe878071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe878071040 0x7fe878071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38116/0 (socket says 172.21.15.153:38116) 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 -- 172.21.15.153:0/3600988531 learned_addr learned my addr 172.21.15.153:0/3600988531 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87d618700 1 --2- 172.21.15.153:0/3600988531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe878160540 0x7fe8781609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 -- 172.21.15.153:0/3600988531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe878160540 msgr2=0x7fe8781609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 --2- 172.21.15.153:0/3600988531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe878160540 0x7fe8781609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 -- 172.21.15.153:0/3600988531 >> v1:172.21.15.177:6789/0 conn(0x7fe87806dee0 legacy=0x7fe87806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 -- 172.21.15.153:0/3600988531 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87815d620 con 0x7fe878071040 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.364+0000 7fe87ce17700 1 --2- 172.21.15.153:0/3600988531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe878071410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0098f0 tx=0x7fe86c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a03117c56ee5ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe8777fe700 1 -- 172.21.15.153:0/3600988531 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00b8a0 con 0x7fe878071040 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe8777fe700 1 -- 172.21.15.153:0/3600988531 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c00ba00 con 0x7fe878071040 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe8777fe700 1 -- 172.21.15.153:0/3600988531 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00bd90 con 0x7fe878071040 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 -- 172.21.15.153:0/3600988531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 msgr2=0x7fe878071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3600988531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe878071410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0098f0 tx=0x7fe86c009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 -- 172.21.15.153:0/3600988531 shutdown_connections 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3600988531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe878160540 0x7fe8781609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3600988531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe878071410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 -- 172.21.15.153:0/3600988531 >> 172.21.15.153:0/3600988531 conn(0x7fe8780d0000 msgr2=0x7fe8780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 -- 172.21.15.153:0/3600988531 shutdown_connections 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.365+0000 7fe87f07b700 1 -- 172.21.15.153:0/3600988531 wait complete. 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.366+0000 7fe87f07b700 1 Processor -- start 2023-11-28T13:00:13.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.366+0000 7fe87f07b700 1 -- start start 2023-11-28T13:00:13.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe87f07b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87806dee0 0x7fe878152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe87f07b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe8781f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe87f07b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe878160540 0x7fe878153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe87f07b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe878152fe0 con 0x7fe878071040 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe87f07b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780b5c90 con 0x7fe878160540 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe87f07b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780b3570 con 0x7fe87806dee0 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe877fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe8781f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe877fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe8781f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38124/0 (socket says 172.21.15.153:38124) 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.367+0000 7fe877fff700 1 -- 172.21.15.153:0/3978846037 learned_addr learned my addr 172.21.15.153:0/3978846037 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe87d618700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe878160540 0x7fe878153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe87ce17700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87806dee0 0x7fe878152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe877fff700 1 -- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87806dee0 msgr2=0x7fe878152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe877fff700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87806dee0 0x7fe878152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe877fff700 1 -- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe878160540 msgr2=0x7fe878153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe877fff700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe878160540 0x7fe878153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe877fff700 1 -- 172.21.15.153:0/3978846037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c009550 con 0x7fe878071040 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe877fff700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe8781f2a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe86400c480 tx=0x7fe86400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe8757fa700 1 -- 172.21.15.153:0/3978846037 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8640052d0 con 0x7fe878071040 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe8757fa700 1 -- 172.21.15.153:0/3978846037 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864005430 con 0x7fe878071040 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87815ccd0 con 0x7fe878071040 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe8757fa700 1 -- 172.21.15.153:0/3978846037 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8640038c0 con 0x7fe878071040 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.368+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87815d900 con 0x7fe878071040 2023-11-28T13:00:13.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.369+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87806a270 con 0x7fe878071040 2023-11-28T13:00:13.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.369+0000 7fe8757fa700 1 -- 172.21.15.153:0/3978846037 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe864019030 con 0x7fe878071040 2023-11-28T13:00:13.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.370+0000 7fe8757fa700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe85c0682f0 0x7fe85c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.370+0000 7fe8757fa700 1 -- 172.21.15.153:0/3978846037 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe864084c10 con 0x7fe878071040 2023-11-28T13:00:13.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.370+0000 7fe87ce17700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe85c0682f0 0x7fe85c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.370+0000 7fe87ce17700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe85c0682f0 0x7fe85c06a7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe86c014040 tx=0x7fe86c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.374+0000 7fe8757fa700 1 -- 172.21.15.153:0/3978846037 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe8640517c0 con 0x7fe878071040 2023-11-28T13:00:13.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.601+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8780720a0 con 0x7fe878071040 2023-11-28T13:00:13.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.601+0000 7fe8757fa700 1 -- 172.21.15.153:0/3978846037 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3533 (secure 0 0 0) 0x7fe864017090 con 0x7fe878071040 2023-11-28T13:00:13.602 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:13.602 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2499459281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2499459281},{"type":"v1","addr":"172.21.15.177:6834","nonce":2499459281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2708083457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2708083457},{"type":"v1","addr":"172.21.15.153:6837","nonce":2708083457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2730194389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2730194389},{"type":"v1","addr":"172.21.15.153:6836","nonce":2730194389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/923823631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":923823631},{"type":"v1","addr":"172.21.15.177:6835","nonce":923823631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T13:00:13.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe85c0682f0 msgr2=0x7fe85c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe85c0682f0 0x7fe85c06a7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe86c014040 tx=0x7fe86c008030 comp rx=0 tx=0).stop 2023-11-28T13:00:13.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 msgr2=0x7fe8781f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe8781f2a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe86400c480 tx=0x7fe86400c840 comp rx=0 tx=0).stop 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 shutdown_connections 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe878160540 0x7fe878153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87806dee0 0x7fe878152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe85c0682f0 0x7fe85c06a7a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 --2- 172.21.15.153:0/3978846037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe878071040 0x7fe8781f2a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 >> 172.21.15.153:0/3978846037 conn(0x7fe8780d0000 msgr2=0x7fe87815e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 shutdown_connections 2023-11-28T13:00:13.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.603+0000 7fe87f07b700 1 -- 172.21.15.153:0/3978846037 wait complete. 2023-11-28T13:00:13.606 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 3 2023-11-28T13:00:13.616 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:13.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.736+0000 7fc0aee0e700 1 Processor -- start 2023-11-28T13:00:13.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.736+0000 7fc0aee0e700 1 -- start start 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0aee0e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a8071040 0x7fc0a8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0aee0e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc0a806dee0 0x7fc0a806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0aee0e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a8071950 con 0x7fc0a806cdc0 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0aee0e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a8161550 con 0x7fc0a806dee0 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0aee0e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a815d060 con 0x7fc0a8071040 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0a7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc0a806dee0 0x7fc0a806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0a7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc0a806dee0 0x7fc0a806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38132/0 (socket says 172.21.15.153:38132) 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.737+0000 7fc0a7fff700 1 -- 172.21.15.153:0/3528129174 learned_addr learned my addr 172.21.15.153:0/3528129174 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0acbaa700 1 --2- 172.21.15.153:0/3528129174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a8071040 0x7fc0a8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a7fff700 1 -- 172.21.15.153:0/3528129174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a8071040 msgr2=0x7fc0a8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a7fff700 1 --2- 172.21.15.153:0/3528129174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a8071040 0x7fc0a8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a7fff700 1 -- 172.21.15.153:0/3528129174 >> v1:172.21.15.177:6789/0 conn(0x7fc0a806cdc0 legacy=0x7fc0a806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a7fff700 1 -- 172.21.15.153:0/3528129174 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a815d340 con 0x7fc0a806dee0 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a7fff700 1 --2- 172.21.15.153:0/3528129174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806dee0 0x7fc0a806e350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098f0 tx=0x7fc094009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12d717f58397e3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a77fe700 1 -- 172.21.15.153:0/3528129174 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09400b8a0 con 0x7fc0a806dee0 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a77fe700 1 -- 172.21.15.153:0/3528129174 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09400ba00 con 0x7fc0a806dee0 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0a77fe700 1 -- 172.21.15.153:0/3528129174 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09400bd90 con 0x7fc0a806dee0 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0aee0e700 1 -- 172.21.15.153:0/3528129174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806dee0 msgr2=0x7fc0a806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.738+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/3528129174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806dee0 0x7fc0a806e350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098f0 tx=0x7fc094009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.739+0000 7fc0aee0e700 1 -- 172.21.15.153:0/3528129174 shutdown_connections 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.739+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/3528129174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a8071040 0x7fc0a8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.739+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/3528129174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806dee0 0x7fc0a806e350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.739+0000 7fc0aee0e700 1 -- 172.21.15.153:0/3528129174 >> 172.21.15.153:0/3528129174 conn(0x7fc0a80d0000 msgr2=0x7fc0a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:13.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.739+0000 7fc0aee0e700 1 -- 172.21.15.153:0/3528129174 shutdown_connections 2023-11-28T13:00:13.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.739+0000 7fc0aee0e700 1 -- 172.21.15.153:0/3528129174 wait complete. 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.740+0000 7fc0aee0e700 1 Processor -- start 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.740+0000 7fc0aee0e700 1 -- start start 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806cdc0 0x7fc0a81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a806dee0 0x7fc0a80c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a8071040 0x7fc0a80b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a81e7c50 con 0x7fc0a806cdc0 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80afd80 con 0x7fc0a8071040 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80affd0 con 0x7fc0a806dee0 2023-11-28T13:00:13.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806cdc0 0x7fc0a81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806cdc0 0x7fc0a81543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38136/0 (socket says 172.21.15.153:38136) 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 -- 172.21.15.153:0/2049360618 learned_addr learned my addr 172.21.15.153:0/2049360618 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0ad3ab700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a8071040 0x7fc0a80b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0a7fff700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a806dee0 0x7fc0a80c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 -- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a806dee0 msgr2=0x7fc0a80c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a806dee0 0x7fc0a80c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 -- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a8071040 msgr2=0x7fc0a80b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a8071040 0x7fc0a80b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 -- 172.21.15.153:0/2049360618 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094009550 con 0x7fc0a806cdc0 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0ad3ab700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a8071040 0x7fc0a80b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0acbaa700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806cdc0 0x7fc0a81543b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d800 tx=0x7fc09c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0a57fa700 1 -- 172.21.15.153:0/2049360618 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0125f0 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0a57fa700 1 -- 172.21.15.153:0/2049360618 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c012750 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a815d900 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0a57fa700 1 -- 172.21.15.153:0/2049360618 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c012b00 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.741+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a815d620 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.743+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a806e890 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.743+0000 7fc0a57fa700 1 -- 172.21.15.153:0/2049360618 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc09c012c60 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.743+0000 7fc0a57fa700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc08c0682f0 0x7fc08c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.743+0000 7fc0a57fa700 1 -- 172.21.15.153:0/2049360618 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc09c085e30 con 0x7fc0a806cdc0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.743+0000 7fc0a7fff700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc08c0682f0 0x7fc08c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.743+0000 7fc0a7fff700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc08c0682f0 0x7fc08c06a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc094014040 tx=0x7fc094008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.747+0000 7fc0a57fa700 1 -- 172.21.15.153:0/2049360618 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc09c0529e0 con 0x7fc0a806cdc0 2023-11-28T13:00:13.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.932+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0a80b0ec0 con 0x7fc0a806cdc0 2023-11-28T13:00:13.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.933+0000 7fc0a57fa700 1 -- 172.21.15.153:0/2049360618 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+14312 (secure 0 0 0) 0x7fc09c056000 con 0x7fc0a806cdc0 2023-11-28T13:00:13.934 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:13.934 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":21,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:13.198075+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:13.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc08c0682f0 msgr2=0x7fc08c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc08c0682f0 0x7fc08c06a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc094014040 tx=0x7fc094008030 comp rx=0 tx=0).stop 2023-11-28T13:00:13.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806cdc0 msgr2=0x7fc0a81543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806cdc0 0x7fc0a81543b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d800 tx=0x7fc09c00dbc0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 shutdown_connections 2023-11-28T13:00:13.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc08c0682f0 0x7fc08c06a7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc0a806cdc0 0x7fc0a81543b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a8071040 0x7fc0a80b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 --2- 172.21.15.153:0/2049360618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0a806dee0 0x7fc0a80c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 >> 172.21.15.153:0/2049360618 conn(0x7fc0a80d0000 msgr2=0x7fc0a806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:13.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 shutdown_connections 2023-11-28T13:00:13.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:13.935+0000 7fc0aee0e700 1 -- 172.21.15.153:0/2049360618 wait complete. 2023-11-28T13:00:13.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:13.948 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:14.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.071+0000 7f49bf0e7700 1 Processor -- start 2023-11-28T13:00:14.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.071+0000 7f49bf0e7700 1 -- start start 2023-11-28T13:00:14.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49bf0e7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49b815cef0 0x7f49b80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49bf0e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 0x7f49b80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49bf0e7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b80cc5f0 con 0x7f49b80cbdf0 2023-11-28T13:00:14.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49bf0e7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b815d470 con 0x7f49b815cef0 2023-11-28T13:00:14.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49bf0e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b815d750 con 0x7f49b806ba90 2023-11-28T13:00:14.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b7fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f49b80cbdf0 0x7f49b806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56310/0 (socket says 172.21.15.153:56310) 2023-11-28T13:00:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b7fff700 1 -- 172.21.15.153:0/2471319520 learned_addr learned my addr 172.21.15.153:0/2471319520 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49bd684700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 0x7f49b80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49bce83700 1 --2- 172.21.15.153:0/2471319520 >> v2:172.21.15.177:3300/0 conn(0x7f49b815cef0 0x7f49b80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050906148 0 0) 0x7f49b80cc5f0 con 0x7f49b80cbdf0 2023-11-28T13:00:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a80035b0 con 0x7f49b80cbdf0 2023-11-28T13:00:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745757521 0 0) 0x7f49a80035b0 con 0x7f49b80cbdf0 2023-11-28T13:00:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 msgr2=0x7f49b80ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 0x7f49b80ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 >> v2:172.21.15.177:3300/0 conn(0x7f49b815cef0 msgr2=0x7f49b80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 --2- 172.21.15.153:0/2471319520 >> v2:172.21.15.177:3300/0 conn(0x7f49b815cef0 0x7f49b80af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b815da30 con 0x7f49b80cbdf0 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f49a4001670 con 0x7f49b80cbdf0 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.072+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 >> v1:172.21.15.153:6789/0 conn(0x7f49b80cbdf0 legacy=0x7f49b806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49a8003a00 0x7f49a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 0x7f49a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8004d90 0x7f49a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b815d470 con 0x7f49a8003a00 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b815d750 con 0x7f49a8004d90 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a800dc20 con 0x7f49b806ba90 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bce83700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49a8003a00 0x7f49a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b7fff700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 0x7f49a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bd684700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8004d90 0x7f49a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bce83700 1 -- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 msgr2=0x7f49a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bce83700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b806ba90 0x7f49a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bce83700 1 -- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8004d90 msgr2=0x7f49a8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bce83700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8004d90 0x7f49a8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bce83700 1 -- 172.21.15.153:0/2471319520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80035b0 con 0x7f49a8003a00 2023-11-28T13:00:14.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49bce83700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49a8003a00 0x7f49a8003e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009b60 tx=0x7f49a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2cd008316a46a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00e6c0 con 0x7f49a8003a00 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac00e820 con 0x7f49a8003a00 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.073+0000 7f49b77fe700 1 -- 172.21.15.153:0/2471319520 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00eb90 con 0x7f49a8003a00 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.074+0000 7f49bf0e7700 1 -- 172.21.15.153:0/2471319520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49a8003a00 msgr2=0x7f49a8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.074+0000 7f49bf0e7700 1 --2- 172.21.15.153:0/2471319520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49a8003a00 0x7f49a8003e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009b60 tx=0x7f49a8005880 comp rx=0 tx=0).stop 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.074+0000 7f49bd684700 1 -- 172.21.15.153:0/2471319520 reap_dead start 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.074+0000 7f49bf0e7700 1 -- 172.21.15.153:0/2471319520 shutdown_connections 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.074+0000 7f49bf0e7700 1 -- 172.21.15.153:0/2471319520 >> 172.21.15.153:0/2471319520 conn(0x7f49b80d0000 msgr2=0x7f49b80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.074+0000 7f49bf0e7700 1 -- 172.21.15.153:0/2471319520 shutdown_connections 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.074+0000 7f49bf0e7700 1 -- 172.21.15.153:0/2471319520 wait complete. 2023-11-28T13:00:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.075+0000 7f49bf0e7700 1 Processor -- start 2023-11-28T13:00:14.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.075+0000 7f49bf0e7700 1 -- start start 2023-11-28T13:00:14.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bf0e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49a8003a00 0x7f49b80c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bf0e7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49b81f3870 0x7f49b8157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bf0e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b80b2e30 0x7f49b80cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bf0e7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81f49f0 con 0x7f49b81f3870 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bf0e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b80c6470 con 0x7f49b80b2e30 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bf0e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81581e0 con 0x7f49a8003a00 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49b81f3870 0x7f49b8157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49b81f3870 0x7f49b8157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38158/0 (socket says 172.21.15.153:38158) 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 -- 172.21.15.153:0/1241246311 learned_addr learned my addr 172.21.15.153:0/1241246311 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bce83700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49a8003a00 0x7f49b80c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bd684700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b80b2e30 0x7f49b80cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 -- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49a8003a00 msgr2=0x7f49b80c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49a8003a00 0x7f49b80c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 -- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b80b2e30 msgr2=0x7f49b80cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b80b2e30 0x7f49b80cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 -- 172.21.15.153:0/1241246311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b815dff0 con 0x7f49b81f3870 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bce83700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49a8003a00 0x7f49b80c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49bd684700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b80b2e30 0x7f49b80cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b7fff700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49b81f3870 0x7f49b8157ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49a40072c0 tx=0x7f49a4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b57fa700 1 -- 172.21.15.153:0/1241246311 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40077a0 con 0x7f49b81f3870 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.076+0000 7f49b57fa700 1 -- 172.21.15.153:0/1241246311 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a400f8d0 con 0x7f49b81f3870 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.077+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b815dd10 con 0x7f49b81f3870 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.077+0000 7f49b57fa700 1 -- 172.21.15.153:0/1241246311 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a400fc80 con 0x7f49b81f3870 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.077+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b806a270 con 0x7f49b81f3870 2023-11-28T13:00:14.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.078+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b81f4b60 con 0x7f49b81f3870 2023-11-28T13:00:14.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.078+0000 7f49b57fa700 1 -- 172.21.15.153:0/1241246311 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f49a401c030 con 0x7f49b81f3870 2023-11-28T13:00:14.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.078+0000 7f49b57fa700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f499c0682f0 0x7f499c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.078+0000 7f49b57fa700 1 -- 172.21.15.153:0/1241246311 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f49a40852a0 con 0x7f49b81f3870 2023-11-28T13:00:14.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.078+0000 7f49bce83700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f499c0682f0 0x7f499c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.079+0000 7f49bce83700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f499c0682f0 0x7f499c06a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0090b0 tx=0x7f49ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.082+0000 7f49b57fa700 1 -- 172.21.15.153:0/1241246311 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f49a4051e50 con 0x7f49b81f3870 2023-11-28T13:00:14.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.264+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49b806d4e0 con 0x7f49b81f3870 2023-11-28T13:00:14.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.264+0000 7f49b57fa700 1 -- 172.21.15.153:0/1241246311 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49a4055470 con 0x7f49b81f3870 2023-11-28T13:00:14.266 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:14.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f499c0682f0 msgr2=0x7f499c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f499c0682f0 0x7f499c06a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0090b0 tx=0x7f49ac008030 comp rx=0 tx=0).stop 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49b81f3870 msgr2=0x7f49b8157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49b81f3870 0x7f49b8157ca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49a40072c0 tx=0x7f49a4007680 comp rx=0 tx=0).stop 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 shutdown_connections 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b80b2e30 0x7f49b80cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49a8003a00 0x7f49b80c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f499c0682f0 0x7f499c06a7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 --2- 172.21.15.153:0/1241246311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49b81f3870 0x7f49b8157ca0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.266+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 >> 172.21.15.153:0/1241246311 conn(0x7f49b80d0000 msgr2=0x7f49b80af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.267+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 shutdown_connections 2023-11-28T13:00:14.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.267+0000 7f49bf0e7700 1 -- 172.21.15.153:0/1241246311 wait complete. 2023-11-28T13:00:14.279 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:14.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.403+0000 7fcd31511700 1 Processor -- start 2023-11-28T13:00:14.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.403+0000 7fcd31511700 1 -- start start 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd31511700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcd2c15c9b0 0x7fcd2c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd31511700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcd2c0cd7c0 0x7fcd2c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd31511700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2c0d1950 con 0x7fcd2c161340 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd31511700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c0b5800 con 0x7fcd2c0cd7c0 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd31511700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c0ce140 con 0x7fcd2c15c9b0 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcd2c0cd7c0 0x7fcd2c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcd2c0cd7c0 0x7fcd2c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38168/0 (socket says 172.21.15.153:38168) 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/3933367522 learned_addr learned my addr 172.21.15.153:0/3933367522 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:14.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2affd700 1 --2- 172.21.15.153:0/3933367522 >> v2:172.21.15.177:3300/0 conn(0x7fcd2c15c9b0 0x7fcd2c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/3933367522 >> v2:172.21.15.177:3300/0 conn(0x7fcd2c15c9b0 msgr2=0x7fcd2c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 --2- 172.21.15.153:0/3933367522 >> v2:172.21.15.177:3300/0 conn(0x7fcd2c15c9b0 0x7fcd2c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/3933367522 >> v1:172.21.15.177:6789/0 conn(0x7fcd2c161340 legacy=0x7fcd2c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/3933367522 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c0d24f0 con 0x7fcd2c0cd7c0 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd2b7fe700 1 --2- 172.21.15.153:0/3933367522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c0cd7c0 0x7fcd2c0cdc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c067460 tx=0x7fcd2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712c61dbece999ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd29ffb700 1 -- 172.21.15.153:0/3933367522 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400e070 con 0x7fcd2c0cd7c0 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd29ffb700 1 -- 172.21.15.153:0/3933367522 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2400b910 con 0x7fcd2c0cd7c0 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.404+0000 7fcd29ffb700 1 -- 172.21.15.153:0/3933367522 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400bcc0 con 0x7fcd2c0cd7c0 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 -- 172.21.15.153:0/3933367522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c0cd7c0 msgr2=0x7fcd2c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 --2- 172.21.15.153:0/3933367522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c0cd7c0 0x7fcd2c0cdc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c067460 tx=0x7fcd2400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 -- 172.21.15.153:0/3933367522 shutdown_connections 2023-11-28T13:00:14.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 --2- 172.21.15.153:0/3933367522 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c0cd7c0 0x7fcd2c0cdc00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 --2- 172.21.15.153:0/3933367522 >> v2:172.21.15.177:3300/0 conn(0x7fcd2c15c9b0 0x7fcd2c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 -- 172.21.15.153:0/3933367522 >> 172.21.15.153:0/3933367522 conn(0x7fcd2c072600 msgr2=0x7fcd2c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 -- 172.21.15.153:0/3933367522 shutdown_connections 2023-11-28T13:00:14.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.405+0000 7fcd31511700 1 -- 172.21.15.153:0/3933367522 wait complete. 2023-11-28T13:00:14.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.406+0000 7fcd31511700 1 Processor -- start 2023-11-28T13:00:14.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.406+0000 7fcd31511700 1 -- start start 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd31511700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd2c0cd7c0 0x7fcd2c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd31511700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd2c15c9b0 0x7fcd2c0b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd31511700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c161340 0x7fcd2c0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd31511700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c1e7b50 con 0x7fcd2c161340 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd31511700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0c79d0 con 0x7fcd2c15c9b0 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd31511700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0c7cb0 con 0x7fcd2c0cd7c0 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c161340 0x7fcd2c0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c161340 0x7fcd2c0b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38174/0 (socket says 172.21.15.153:38174) 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/1813313054 learned_addr learned my addr 172.21.15.153:0/1813313054 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2a7fc700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd2c15c9b0 0x7fcd2c0b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2affd700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd2c0cd7c0 0x7fcd2c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd2c0cd7c0 msgr2=0x7fcd2c14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd2c0cd7c0 0x7fcd2c14c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd2c15c9b0 msgr2=0x7fcd2c0b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd2c15c9b0 0x7fcd2c0b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 -- 172.21.15.153:0/1813313054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24009b30 con 0x7fcd2c161340 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2a7fc700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd2c15c9b0 0x7fcd2c0b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2affd700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd2c0cd7c0 0x7fcd2c14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.407+0000 7fcd2b7fe700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c161340 0x7fcd2c0b7ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0675d0 tx=0x7fcd24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.408+0000 7fcd137fe700 1 -- 172.21.15.153:0/1813313054 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd240099b0 con 0x7fcd2c161340 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.408+0000 7fcd137fe700 1 -- 172.21.15.153:0/1813313054 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd240092e0 con 0x7fcd2c161340 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.408+0000 7fcd137fe700 1 -- 172.21.15.153:0/1813313054 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2401b630 con 0x7fcd2c161340 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.408+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c0d1300 con 0x7fcd2c161340 2023-11-28T13:00:14.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.408+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c0b5520 con 0x7fcd2c161340 2023-11-28T13:00:14.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.409+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c0d15e0 con 0x7fcd2c161340 2023-11-28T13:00:14.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.409+0000 7fcd137fe700 1 -- 172.21.15.153:0/1813313054 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd2402b070 con 0x7fcd2c161340 2023-11-28T13:00:14.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.410+0000 7fcd137fe700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcd180682f0 0x7fcd1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.410+0000 7fcd2affd700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcd180682f0 0x7fcd1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.410+0000 7fcd137fe700 1 -- 172.21.15.153:0/1813313054 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcd24087100 con 0x7fcd2c161340 2023-11-28T13:00:14.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.410+0000 7fcd2affd700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcd180682f0 0x7fcd1806a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009b90 tx=0x7fcd1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.414+0000 7fcd137fe700 1 -- 172.21.15.153:0/1813313054 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcd24053cb0 con 0x7fcd2c161340 2023-11-28T13:00:14.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.597+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcd2c1e7c90 con 0x7fcd2c161340 2023-11-28T13:00:14.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.598+0000 7fcd137fe700 1 -- 172.21.15.153:0/1813313054 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcd2c1e7c90 con 0x7fcd2c161340 2023-11-28T13:00:14.599 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:14.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcd180682f0 msgr2=0x7fcd1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcd180682f0 0x7fcd1806a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009b90 tx=0x7fcd1c008040 comp rx=0 tx=0).stop 2023-11-28T13:00:14.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c161340 msgr2=0x7fcd2c0b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c161340 0x7fcd2c0b7ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0675d0 tx=0x7fcd24004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 shutdown_connections 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcd180682f0 0x7fcd1806a7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcd2c161340 0x7fcd2c0b7ac0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd2c0cd7c0 0x7fcd2c14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 --2- 172.21.15.153:0/1813313054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd2c15c9b0 0x7fcd2c0b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 >> 172.21.15.153:0/1813313054 conn(0x7fcd2c072600 msgr2=0x7fcd2c0b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 shutdown_connections 2023-11-28T13:00:14.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.600+0000 7fcd31511700 1 -- 172.21.15.153:0/1813313054 wait complete. 2023-11-28T13:00:14.613 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:00:14.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.732+0000 7f39d2c47700 1 Processor -- start 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.732+0000 7f39d2c47700 1 -- start start 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d2c47700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d2c47700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f39cc06a5e0 0x7f39cc06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d2c47700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc0aee40 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d2c47700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39cc15bf10 con 0x7f39cc06a5e0 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d2c47700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc06c930 con 0x7f39cc06e500 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39cbfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f39cc0ae9d0 0x7f39cc06e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56342/0 (socket says 172.21.15.153:56342) 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39cbfff700 1 -- 172.21.15.153:0/3733203553 learned_addr learned my addr 172.21.15.153:0/3733203553 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d11e4700 1 --2- 172.21.15.153:0/3733203553 >> v2:172.21.15.177:3300/0 conn(0x7f39cc06a5e0 0x7f39cc06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d09e3700 1 --2- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39cb7fe700 1 -- 172.21.15.153:0/3733203553 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107809714 0 0) 0x7f39cc0aee40 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39cb7fe700 1 -- 172.21.15.153:0/3733203553 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39bc0035b0 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d11e4700 1 -- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 msgr2=0x7f39cc15b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d11e4700 1 --2- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc15b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d11e4700 1 -- 172.21.15.153:0/3733203553 >> v1:172.21.15.153:6789/0 conn(0x7f39cc0ae9d0 legacy=0x7f39cc06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.733+0000 7f39d11e4700 1 -- 172.21.15.153:0/3733203553 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc06cc10 con 0x7f39cc06a5e0 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d11e4700 1 --2- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 0x7f39cc06aa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009d90 tx=0x7f39c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868d255b1eecb2dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39cb7fe700 1 -- 172.21.15.153:0/3733203553 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c400e070 con 0x7f39cc06a5e0 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39cb7fe700 1 -- 172.21.15.153:0/3733203553 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c400bb20 con 0x7f39cc06a5e0 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39cb7fe700 1 -- 172.21.15.153:0/3733203553 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4012430 con 0x7f39cc06a5e0 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 -- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 msgr2=0x7f39cc06aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 --2- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 0x7f39cc06aa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009d90 tx=0x7f39c400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 -- 172.21.15.153:0/3733203553 shutdown_connections 2023-11-28T13:00:14.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 --2- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc15b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 --2- 172.21.15.153:0/3733203553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 0x7f39cc06aa00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 -- 172.21.15.153:0/3733203553 >> 172.21.15.153:0/3733203553 conn(0x7f39cc0b12b0 msgr2=0x7f39cc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 -- 172.21.15.153:0/3733203553 shutdown_connections 2023-11-28T13:00:14.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.734+0000 7f39d2c47700 1 -- 172.21.15.153:0/3733203553 wait complete. 2023-11-28T13:00:14.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.735+0000 7f39d2c47700 1 Processor -- start 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.735+0000 7f39d2c47700 1 -- start start 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d2c47700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 0x7f39cc14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d2c47700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d2c47700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39cc0ae9d0 0x7f39cc0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d2c47700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0ad4d0 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d2c47700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc1579a0 con 0x7f39cc06a5e0 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d2c47700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0ac8c0 con 0x7f39cc06e500 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d11e4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39cc0ae9d0 0x7f39cc0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d11e4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39cc0ae9d0 0x7f39cc0adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38188/0 (socket says 172.21.15.153:38188) 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.736+0000 7f39d11e4700 1 -- 172.21.15.153:0/4188829790 learned_addr learned my addr 172.21.15.153:0/4188829790 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39cbfff700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d09e3700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 0x7f39cc14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d11e4700 1 -- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 msgr2=0x7f39cc0ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d11e4700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc0ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d11e4700 1 -- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 msgr2=0x7f39cc14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d11e4700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 0x7f39cc14bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d11e4700 1 -- 172.21.15.153:0/4188829790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009a70 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d11e4700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39cc0ae9d0 0x7f39cc0adea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39c4014040 tx=0x7f39c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39c97fa700 1 -- 172.21.15.153:0/4188829790 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4012a90 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39c97fa700 1 -- 172.21.15.153:0/4188829790 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4020070 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc06cef0 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39c97fa700 1 -- 172.21.15.153:0/4188829790 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4012e50 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.737+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc0664c0 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.738+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc15ac80 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.739+0000 7f39c97fa700 1 -- 172.21.15.153:0/4188829790 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39c4004040 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.739+0000 7f39c97fa700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f39b00682f0 0x7f39b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.739+0000 7f39c97fa700 1 -- 172.21.15.153:0/4188829790 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f39c4022070 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.739+0000 7f39d09e3700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f39b00682f0 0x7f39b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.739+0000 7f39d09e3700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f39b00682f0 0x7f39b006a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009ab0 tx=0x7f39c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.743+0000 7f39c97fa700 1 -- 172.21.15.153:0/4188829790 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39c40585b0 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.926+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f39cc0ad610 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.927+0000 7f39c97fa700 1 -- 172.21.15.153:0/4188829790 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v21) v1 ==== 84+0+126 (secure 0 0 0) 0x7f39c4058140 con 0x7f39cc0ae9d0 2023-11-28T13:00:14.928 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:14.928 INFO:teuthology.orchestra.run.smithi153.stdout:[ 2023-11-28T13:00:14.928 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:00:14.928 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 1, 2023-11-28T13:00:14.928 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": ".mgr" 2023-11-28T13:00:14.928 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:00:14.929 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:00:14.929 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 2, 2023-11-28T13:00:14.929 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": "rbd" 2023-11-28T13:00:14.929 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:00:14.929 INFO:teuthology.orchestra.run.smithi153.stdout:] 2023-11-28T13:00:14.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f39b00682f0 msgr2=0x7f39b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f39b00682f0 0x7f39b006a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009ab0 tx=0x7f39c0008040 comp rx=0 tx=0).stop 2023-11-28T13:00:14.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39cc0ae9d0 msgr2=0x7f39cc0adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39cc0ae9d0 0x7f39cc0adea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39c4014040 tx=0x7f39c4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 shutdown_connections 2023-11-28T13:00:14.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f39b00682f0 0x7f39b006a7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39cc0ae9d0 0x7f39cc0adea0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39cc06a5e0 0x7f39cc14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 --2- 172.21.15.153:0/4188829790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39cc06e500 0x7f39cc0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 >> 172.21.15.153:0/4188829790 conn(0x7f39cc0b12b0 msgr2=0x7f39cc0cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 shutdown_connections 2023-11-28T13:00:14.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:14.929+0000 7f39d2c47700 1 -- 172.21.15.153:0/4188829790 wait complete. 2023-11-28T13:00:14.941 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:00:14.942 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:00:15.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.065+0000 7f15a6868700 1 Processor -- start 2023-11-28T13:00:15.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.065+0000 7f15a6868700 1 -- start start 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a6868700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a00af060 0x7f15a0070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a6868700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f15a00af430 0x7f15a006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a6868700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f15a006a270 0x7f15a006a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a6868700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a006dee0 con 0x7f15a006a270 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a6868700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a006ac10 con 0x7f15a00af430 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a6868700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00713c0 con 0x7f15a00af060 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a4e05700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f15a006a270 0x7f15a006a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a4e05700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f15a006a270 0x7f15a006a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38196/0 (socket says 172.21.15.153:38196) 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f15a4e05700 1 -- 172.21.15.153:0/1404606238 learned_addr learned my addr 172.21.15.153:0/1404606238 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.066+0000 7f159f7fe700 1 --2- 172.21.15.153:0/1404606238 >> v2:172.21.15.177:3300/0 conn(0x7f15a00af430 0x7f15a006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f159ffff700 1 --2- 172.21.15.153:0/1404606238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a00af060 0x7f15a0070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a4e05700 1 -- 172.21.15.153:0/1404606238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a00af060 msgr2=0x7f15a0070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a4e05700 1 --2- 172.21.15.153:0/1404606238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a00af060 0x7f15a0070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a4e05700 1 -- 172.21.15.153:0/1404606238 >> v2:172.21.15.177:3300/0 conn(0x7f15a00af430 msgr2=0x7f15a006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a4e05700 1 --2- 172.21.15.153:0/1404606238 >> v2:172.21.15.177:3300/0 conn(0x7f15a00af430 0x7f15a006d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a4e05700 1 -- 172.21.15.153:0/1404606238 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00716a0 con 0x7f15a006a270 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f159ffff700 1 --2- 172.21.15.153:0/1404606238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a00af060 0x7f15a0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a4e05700 1 --2- 172.21.15.153:0/1404606238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a006a270 0x7f15a006a6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15a00d0e30 tx=0x7f159800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc828e0b4c6fc061 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f159effd700 1 -- 172.21.15.153:0/1404606238 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800e070 con 0x7f15a006a270 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f159effd700 1 -- 172.21.15.153:0/1404606238 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800b910 con 0x7f15a006a270 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f159effd700 1 -- 172.21.15.153:0/1404606238 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800bcc0 con 0x7f15a006a270 2023-11-28T13:00:15.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a6868700 1 -- 172.21.15.153:0/1404606238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a006a270 msgr2=0x7f15a006a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a6868700 1 --2- 172.21.15.153:0/1404606238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a006a270 0x7f15a006a6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15a00d0e30 tx=0x7f159800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a6868700 1 -- 172.21.15.153:0/1404606238 shutdown_connections 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a6868700 1 --2- 172.21.15.153:0/1404606238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a00af060 0x7f15a0070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a6868700 1 --2- 172.21.15.153:0/1404606238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a006a270 0x7f15a006a6d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a6868700 1 --2- 172.21.15.153:0/1404606238 >> v2:172.21.15.177:3300/0 conn(0x7f15a00af430 0x7f15a006d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.067+0000 7f15a6868700 1 -- 172.21.15.153:0/1404606238 >> 172.21.15.153:0/1404606238 conn(0x7f15a006b380 msgr2=0x7f15a006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.068+0000 7f15a6868700 1 -- 172.21.15.153:0/1404606238 shutdown_connections 2023-11-28T13:00:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.068+0000 7f15a6868700 1 -- 172.21.15.153:0/1404606238 wait complete. 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.069+0000 7f15a6868700 1 Processor -- start 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.069+0000 7f15a6868700 1 -- start start 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a006a270 0x7f15a00c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a00af060 0x7f15a0150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15a00af430 0x7f15a00adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00c7d30 con 0x7f15a00af060 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00c8830 con 0x7f15a00af430 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00c8b10 con 0x7f15a006a270 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a00af060 0x7f15a0150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a00af060 0x7f15a0150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38200/0 (socket says 172.21.15.153:38200) 2023-11-28T13:00:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 -- 172.21.15.153:0/1878095656 learned_addr learned my addr 172.21.15.153:0/1878095656 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a4e05700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15a00af430 0x7f15a00adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159ffff700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a006a270 0x7f15a00c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 -- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a006a270 msgr2=0x7f15a00c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a006a270 0x7f15a00c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 -- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15a00af430 msgr2=0x7f15a00adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15a00af430 0x7f15a00adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 -- 172.21.15.153:0/1878095656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009b30 con 0x7f15a00af060 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159f7fe700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a00af060 0x7f15a0150270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f159000e980 tx=0x7f159000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159cff9700 1 -- 172.21.15.153:0/1878095656 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590009710 con 0x7f15a00af060 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159cff9700 1 -- 172.21.15.153:0/1878095656 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159000c690 con 0x7f15a00af060 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f159cff9700 1 -- 172.21.15.153:0/1878095656 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159000ca40 con 0x7f15a00af060 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a0071c60 con 0x7f15a00af060 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.070+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a0071980 con 0x7f15a00af060 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.071+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a0071f40 con 0x7f15a00af060 2023-11-28T13:00:15.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.072+0000 7f159cff9700 1 -- 172.21.15.153:0/1878095656 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1590019030 con 0x7f15a00af060 2023-11-28T13:00:15.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.072+0000 7f159cff9700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f158c0682f0 0x7f158c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.072+0000 7f159cff9700 1 -- 172.21.15.153:0/1878095656 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1590085760 con 0x7f15a00af060 2023-11-28T13:00:15.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.072+0000 7f159ffff700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f158c0682f0 0x7f158c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.073+0000 7f159ffff700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f158c0682f0 0x7f158c06a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f159400c940 tx=0x7f1594008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:15.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.076+0000 7f159cff9700 1 -- 172.21.15.153:0/1878095656 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1590052310 con 0x7f15a00af060 2023-11-28T13:00:15.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:15.262+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f15a0068690 con 0x7f15a00af060 2023-11-28T13:00:15.976 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:15.979 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:16.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.216+0000 7f159cff9700 1 -- 172.21.15.153:0/1878095656 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v22) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1590055930 con 0x7f15a00af060 2023-11-28T13:00:16.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.218+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f158c0682f0 msgr2=0x7f158c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.218+0000 7f15a6868700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f158c0682f0 0x7f158c06a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f159400c940 tx=0x7f1594008040 comp rx=0 tx=0).stop 2023-11-28T13:00:16.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.218+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a00af060 msgr2=0x7f15a0150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.218+0000 7f15a6868700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a00af060 0x7f15a0150270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f159000e980 tx=0x7f159000ec90 comp rx=0 tx=0).stop 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 shutdown_connections 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15a00af430 0x7f15a00adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15a006a270 0x7f15a00c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f158c0682f0 0x7f158c06a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 --2- 172.21.15.153:0/1878095656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15a00af060 0x7f15a0150270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 >> 172.21.15.153:0/1878095656 conn(0x7f15a006b380 msgr2=0x7f15a00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 shutdown_connections 2023-11-28T13:00:16.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.219+0000 7f15a6868700 1 -- 172.21.15.153:0/1878095656 wait complete. 2023-11-28T13:00:16.221 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' created 2023-11-28T13:00:16.236 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:00:16.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.366+0000 7f1542279700 1 Processor -- start 2023-11-28T13:00:16.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.366+0000 7f1542279700 1 -- start start 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f1542279700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f153c140950 0x7f153c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f1542279700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c13ff00 con 0x7f153c0cd8c0 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f1542279700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c1404c0 con 0x7f153c0b2e70 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f1542279700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c140dc0 con 0x7f153c140950 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f153c0cd8c0 0x7f153c0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56370/0 (socket says 172.21.15.153:56370) 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153bfff700 1 -- 172.21.15.153:0/1694209727 learned_addr learned my addr 172.21.15.153:0/1694209727 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153b7fe700 1 --2- 172.21.15.153:0/1694209727 >> v2:172.21.15.177:3300/0 conn(0x7f153c140950 0x7f153c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641233895 0 0) 0x7f153c13ff00 con 0x7f153c0cd8c0 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15240035b0 con 0x7f153c0cd8c0 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3568118746 0 0) 0x7f15240035b0 con 0x7f153c0cd8c0 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 >> v2:172.21.15.177:3300/0 conn(0x7f153c140950 msgr2=0x7f153c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 --2- 172.21.15.153:0/1694209727 >> v2:172.21.15.177:3300/0 conn(0x7f153c140950 0x7f153c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 >> v1:172.21.15.177:6789/0 conn(0x7f153c0b2e70 legacy=0x7f153c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c15aa80 con 0x7f153c0cd8c0 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f152c0017f0 con 0x7f153c0cd8c0 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.367+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 >> v1:172.21.15.153:6789/0 conn(0x7f153c0cd8c0 legacy=0x7f153c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f1524003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1524004850 0x7f1524004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1524005200 0x7f152400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c140dc0 con 0x7f1524003a00 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152400c110 con 0x7f1524004850 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152400c2d0 con 0x7f1524005200 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153bfff700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f1524003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153b7fe700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1524004850 0x7f1524004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f1540816700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1524005200 0x7f152400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153bfff700 1 -- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1524005200 msgr2=0x7f152400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153bfff700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1524005200 0x7f152400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153bfff700 1 -- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1524004850 msgr2=0x7f1524004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153bfff700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1524004850 0x7f1524004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153bfff700 1 -- 172.21.15.153:0/1694209727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240035b0 con 0x7f1524003a00 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f1540816700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1524005200 0x7f152400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:16.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f1540816700 1 -- 172.21.15.153:0/1694209727 reap_dead start 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153bfff700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f1524003e10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c240 tx=0x7f1524005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74348aa87a289e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c00cde0 con 0x7f1524003a00 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c007b80 con 0x7f1524003a00 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.368+0000 7f153affd700 1 -- 172.21.15.153:0/1694209727 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c0072d0 con 0x7f1524003a00 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.369+0000 7f1542279700 1 -- 172.21.15.153:0/1694209727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 msgr2=0x7f1524003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.369+0000 7f1542279700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f1524003e10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c240 tx=0x7f1524005880 comp rx=0 tx=0).stop 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.369+0000 7f1542279700 1 -- 172.21.15.153:0/1694209727 shutdown_connections 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.369+0000 7f1542279700 1 --2- 172.21.15.153:0/1694209727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f1524003e10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.369+0000 7f1542279700 1 -- 172.21.15.153:0/1694209727 >> 172.21.15.153:0/1694209727 conn(0x7f153c06aed0 msgr2=0x7f153c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:16.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.369+0000 7f1542279700 1 -- 172.21.15.153:0/1694209727 shutdown_connections 2023-11-28T13:00:16.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.369+0000 7f1542279700 1 -- 172.21.15.153:0/1694209727 wait complete. 2023-11-28T13:00:16.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.370+0000 7f1542279700 1 Processor -- start 2023-11-28T13:00:16.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.370+0000 7f1542279700 1 -- start start 2023-11-28T13:00:16.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1542279700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f153c1f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1542279700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f153c0b6870 0x7f153c141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1542279700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f153c0b2220 0x7f153c0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1542279700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0b2bd0 con 0x7f1524003a00 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1542279700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0b3d00 con 0x7f153c0b6870 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1542279700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0b3fe0 con 0x7f153c0b2220 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f153c1f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f153c1f3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38218/0 (socket says 172.21.15.153:38218) 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 -- 172.21.15.153:0/2415024991 learned_addr learned my addr 172.21.15.153:0/2415024991 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153b7fe700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f153c0b6870 0x7f153c141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 -- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f153c0b2220 msgr2=0x7f153c0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1540816700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f153c0b2220 0x7f153c0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f153c0b2220 0x7f153c0b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 -- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f153c0b6870 msgr2=0x7f153c141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f153c0b6870 0x7f153c141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 -- 172.21.15.153:0/2415024991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c0b35f0 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153b7fe700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f153c0b6870 0x7f153c141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f153bfff700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f153c1f3310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f152c001ab0 tx=0x7f152c001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1538ff9700 1 -- 172.21.15.153:0/2415024991 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c005a30 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1538ff9700 1 -- 172.21.15.153:0/2415024991 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c005b90 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.371+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c0b3310 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.372+0000 7f1538ff9700 1 -- 172.21.15.153:0/2415024991 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c000f70 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.372+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c140150 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.372+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c0b2bd0 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.373+0000 7f1538ff9700 1 -- 172.21.15.153:0/2415024991 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f152c024030 con 0x7f1524003a00 2023-11-28T13:00:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.373+0000 7f1538ff9700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f15280682f0 0x7f152806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.373+0000 7f1538ff9700 1 -- 172.21.15.153:0/2415024991 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f152c086ca0 con 0x7f1524003a00 2023-11-28T13:00:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.373+0000 7f153b7fe700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f15280682f0 0x7f152806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.374+0000 7f153b7fe700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f15280682f0 0x7f152806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b1160 tx=0x7f1530008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.378+0000 7f1538ff9700 1 -- 172.21.15.153:0/2415024991 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f152c0536e0 con 0x7f1524003a00 2023-11-28T13:00:16.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:16.560+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f153c068690 con 0x7f1524003a00 2023-11-28T13:00:17.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.226+0000 7f1538ff9700 1 -- 172.21.15.153:0/2415024991 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v23) v1 ==== 170+0+0 (secure 0 0 0) 0x7f152c0170d0 con 0x7f1524003a00 2023-11-28T13:00:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.231+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f15280682f0 msgr2=0x7f152806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.231+0000 7f1542279700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f15280682f0 0x7f152806a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b1160 tx=0x7f1530008040 comp rx=0 tx=0).stop 2023-11-28T13:00:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 msgr2=0x7f153c1f3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f153c1f3310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f152c001ab0 tx=0x7f152c001b80 comp rx=0 tx=0).stop 2023-11-28T13:00:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 shutdown_connections 2023-11-28T13:00:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f15280682f0 0x7f152806a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1524003a00 0x7f153c1f3310 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f153c0b2220 0x7f153c0b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 --2- 172.21.15.153:0/2415024991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f153c0b6870 0x7f153c141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.232+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 >> 172.21.15.153:0/2415024991 conn(0x7f153c06aed0 msgr2=0x7f153c13eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.235+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 shutdown_connections 2023-11-28T13:00:17.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.236+0000 7f1542279700 1 -- 172.21.15.153:0/2415024991 wait complete. 2023-11-28T13:00:17.243 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' created 2023-11-28T13:00:17.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:00:17.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.383+0000 7f7ba95f8700 1 Processor -- start 2023-11-28T13:00:17.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.383+0000 7f7ba95f8700 1 -- start start 2023-11-28T13:00:17.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba95f8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ba40cd7c0 0x7f7ba40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba95f8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40d1950 con 0x7f7ba415c9b0 2023-11-28T13:00:17.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba95f8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40ce140 con 0x7f7ba4161340 2023-11-28T13:00:17.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba95f8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba40b5800 con 0x7f7ba40cd7c0 2023-11-28T13:00:17.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba37fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ba40cd7c0 0x7f7ba40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba2ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7ba415c9b0 0x7f7ba40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56390/0 (socket says 172.21.15.153:56390) 2023-11-28T13:00:17.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba2ffd700 1 -- 172.21.15.153:0/978236520 learned_addr learned my addr 172.21.15.153:0/978236520 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:17.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba37fe700 1 -- 172.21.15.153:0/978236520 >> v1:172.21.15.177:6789/0 conn(0x7f7ba4161340 legacy=0x7f7ba40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba37fe700 1 -- 172.21.15.153:0/978236520 >> v1:172.21.15.153:6789/0 conn(0x7f7ba415c9b0 legacy=0x7f7ba40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba37fe700 1 -- 172.21.15.153:0/978236520 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40d24f0 con 0x7f7ba40cd7c0 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba1ffb700 1 -- 172.21.15.153:0/978236520 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904643565 0 0) 0x7f7ba40d1950 con 0x7f7ba415c9b0 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.384+0000 7f7ba37fe700 1 --2- 172.21.15.153:0/978236520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40cdc00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009d90 tx=0x7f7b9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedaa4b90d82a36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba1ffb700 1 -- 172.21.15.153:0/978236520 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00e070 con 0x7f7ba40cd7c0 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba1ffb700 1 -- 172.21.15.153:0/978236520 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c00bbd0 con 0x7f7ba40cd7c0 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba1ffb700 1 -- 172.21.15.153:0/978236520 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c012430 con 0x7f7ba40cd7c0 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba95f8700 1 -- 172.21.15.153:0/978236520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba40cd7c0 msgr2=0x7f7ba40cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/978236520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40cdc00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009d90 tx=0x7f7b9c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:17.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba95f8700 1 -- 172.21.15.153:0/978236520 shutdown_connections 2023-11-28T13:00:17.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/978236520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40cdc00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba95f8700 1 -- 172.21.15.153:0/978236520 >> 172.21.15.153:0/978236520 conn(0x7f7ba4072600 msgr2=0x7f7ba40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba95f8700 1 -- 172.21.15.153:0/978236520 shutdown_connections 2023-11-28T13:00:17.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.385+0000 7f7ba95f8700 1 -- 172.21.15.153:0/978236520 wait complete. 2023-11-28T13:00:17.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.387+0000 7f7ba95f8700 1 Processor -- start 2023-11-28T13:00:17.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.387+0000 7f7ba95f8700 1 -- start start 2023-11-28T13:00:17.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba95f8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba95f8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba415c9b0 0x7f7ba40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba95f8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ba4161340 0x7f7ba414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba95f8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40ccee0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba95f8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40c85c0 con 0x7f7ba40cd7c0 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba95f8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40c88a0 con 0x7f7ba4161340 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba415c9b0 0x7f7ba40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba415c9b0 0x7f7ba40cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38234/0 (socket says 172.21.15.153:38234) 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 -- 172.21.15.153:0/738554284 learned_addr learned my addr 172.21.15.153:0/738554284 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba2ffd700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 -- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ba4161340 msgr2=0x7f7ba414d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba37fe700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ba4161340 0x7f7ba414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ba4161340 0x7f7ba414d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 -- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ba40cd7c0 msgr2=0x7f7ba40c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 -- 172.21.15.153:0/738554284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c009a70 con 0x7f7ba415c9b0 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba2ffd700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7ba27fc700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba415c9b0 0x7f7ba40cd160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c4e0 tx=0x7f7b9400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.388+0000 7f7b8b7fe700 1 -- 172.21.15.153:0/738554284 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b940052d0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.389+0000 7f7b8b7fe700 1 -- 172.21.15.153:0/738554284 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94005430 con 0x7f7ba415c9b0 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.389+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba40d1300 con 0x7f7ba415c9b0 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.389+0000 7f7b8b7fe700 1 -- 172.21.15.153:0/738554284 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b940038c0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.389+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba40b5520 con 0x7f7ba415c9b0 2023-11-28T13:00:17.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.390+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba40d15e0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.390+0000 7f7b8b7fe700 1 -- 172.21.15.153:0/738554284 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b94019030 con 0x7f7ba415c9b0 2023-11-28T13:00:17.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.391+0000 7f7b8b7fe700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b900682f0 0x7f7b9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.391+0000 7f7ba2ffd700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b900682f0 0x7f7b9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.391+0000 7f7b8b7fe700 1 -- 172.21.15.153:0/738554284 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f7b94084fe0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.391+0000 7f7ba2ffd700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b900682f0 0x7f7b9006a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b98000f60 tx=0x7f7b98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.395+0000 7f7b8b7fe700 1 -- 172.21.15.153:0/738554284 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b940518a0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.623+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7ba40745c0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.629+0000 7f7b8b7fe700 1 -- 172.21.15.153:0/738554284 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7b94054ec0 con 0x7f7ba415c9b0 2023-11-28T13:00:17.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.635+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b900682f0 msgr2=0x7f7b9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.635+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b900682f0 0x7f7b9006a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b98000f60 tx=0x7f7b98009040 comp rx=0 tx=0).stop 2023-11-28T13:00:17.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.635+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba415c9b0 msgr2=0x7f7ba40cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.635+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba415c9b0 0x7f7ba40cd160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c4e0 tx=0x7f7b9400c8a0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.636+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 shutdown_connections 2023-11-28T13:00:17.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.636+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ba4161340 0x7f7ba414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.636+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ba40cd7c0 0x7f7ba40c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.636+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b900682f0 0x7f7b9006a7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.636+0000 7f7ba95f8700 1 --2- 172.21.15.153:0/738554284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ba415c9b0 0x7f7ba40cd160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.636+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 >> 172.21.15.153:0/738554284 conn(0x7f7ba4072600 msgr2=0x7f7ba40b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.636+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 shutdown_connections 2023-11-28T13:00:17.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.637+0000 7f7ba95f8700 1 -- 172.21.15.153:0/738554284 wait complete. 2023-11-28T13:00:17.647 INFO:teuthology.orchestra.run.smithi153.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T13:00:17.659 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:17.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.778+0000 7f40b2892700 1 Processor -- start 2023-11-28T13:00:17.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.778+0000 7f40b2892700 1 -- start start 2023-11-28T13:00:17.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40b2892700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40ac15cef0 0x7f40ac0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40b2892700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac0cc5f0 con 0x7f40ac0cbdf0 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40b2892700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac15d470 con 0x7f40ac06ba90 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40b2892700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac15d750 con 0x7f40ac15cef0 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40abfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40ac15cef0 0x7f40ac0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40ab7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f40ac0cbdf0 0x7f40ac06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56404/0 (socket says 172.21.15.153:56404) 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40abfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40ac15cef0 0x7f40ac0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38240/0 (socket says 172.21.15.153:38240) 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40ab7fe700 1 -- 172.21.15.153:0/73437099 learned_addr learned my addr 172.21.15.153:0/73437099 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40aaffd700 1 -- 172.21.15.153:0/73437099 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982878877 0 0) 0x7f40ac0cc5f0 con 0x7f40ac0cbdf0 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40abfff700 1 -- 172.21.15.153:0/73437099 >> v1:172.21.15.177:6789/0 conn(0x7f40ac06ba90 legacy=0x7f40ac0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40abfff700 1 -- 172.21.15.153:0/73437099 >> v1:172.21.15.153:6789/0 conn(0x7f40ac0cbdf0 legacy=0x7f40ac06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.779+0000 7f40abfff700 1 -- 172.21.15.153:0/73437099 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac15da30 con 0x7f40ac15cef0 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40abfff700 1 --2- 172.21.15.153:0/73437099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0af400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179655d0bf6270c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40aaffd700 1 -- 172.21.15.153:0/73437099 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0011830 con 0x7f40ac15cef0 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40aaffd700 1 -- 172.21.15.153:0/73437099 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a0011990 con 0x7f40ac15cef0 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40aaffd700 1 -- 172.21.15.153:0/73437099 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0011d20 con 0x7f40ac15cef0 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40b2892700 1 -- 172.21.15.153:0/73437099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 msgr2=0x7f40ac0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40b2892700 1 --2- 172.21.15.153:0/73437099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0af400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a0009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40b2892700 1 -- 172.21.15.153:0/73437099 shutdown_connections 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40b2892700 1 --2- 172.21.15.153:0/73437099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0af400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40b2892700 1 -- 172.21.15.153:0/73437099 >> 172.21.15.153:0/73437099 conn(0x7f40ac0d0000 msgr2=0x7f40ac0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40b2892700 1 -- 172.21.15.153:0/73437099 shutdown_connections 2023-11-28T13:00:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.780+0000 7f40b2892700 1 -- 172.21.15.153:0/73437099 wait complete. 2023-11-28T13:00:17.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.782+0000 7f40b2892700 1 Processor -- start 2023-11-28T13:00:17.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.782+0000 7f40b2892700 1 -- start start 2023-11-28T13:00:17.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.782+0000 7f40b2892700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac06ba90 0x7f40ac1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.782+0000 7f40b2892700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40ac0cbdf0 0x7f40ac0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b2892700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b2892700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac15a7f0 con 0x7f40ac15cef0 2023-11-28T13:00:17.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b2892700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0c7170 con 0x7f40ac06ba90 2023-11-28T13:00:17.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b2892700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0c7410 con 0x7f40ac0cbdf0 2023-11-28T13:00:17.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40abfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac06ba90 0x7f40ac1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40ab7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40ac0cbdf0 0x7f40ac0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38248/0 (socket says 172.21.15.153:38248) 2023-11-28T13:00:17.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 -- 172.21.15.153:0/585165766 learned_addr learned my addr 172.21.15.153:0/585165766 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:17.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 -- 172.21.15.153:0/585165766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40ac0cbdf0 msgr2=0x7f40ac0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40ac0cbdf0 0x7f40ac0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 -- 172.21.15.153:0/585165766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac06ba90 msgr2=0x7f40ac1f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac06ba90 0x7f40ac1f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 -- 172.21.15.153:0/585165766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00094a0 con 0x7f40ac15cef0 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b0e2f700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0b4990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40a400cb90 tx=0x7f40a400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40a8ff9700 1 -- 172.21.15.153:0/585165766 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4003c00 con 0x7f40ac15cef0 2023-11-28T13:00:17.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40a8ff9700 1 -- 172.21.15.153:0/585165766 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a4003d60 con 0x7f40ac15cef0 2023-11-28T13:00:17.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac15dff0 con 0x7f40ac15cef0 2023-11-28T13:00:17.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40a8ff9700 1 -- 172.21.15.153:0/585165766 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a400aae0 con 0x7f40ac15cef0 2023-11-28T13:00:17.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.783+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac15dd10 con 0x7f40ac15cef0 2023-11-28T13:00:17.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.784+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac0cad60 con 0x7f40ac15cef0 2023-11-28T13:00:17.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.785+0000 7f40a8ff9700 1 -- 172.21.15.153:0/585165766 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f40a4010050 con 0x7f40ac15cef0 2023-11-28T13:00:17.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.785+0000 7f40a8ff9700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40980682f0 0x7f409806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.785+0000 7f40a8ff9700 1 -- 172.21.15.153:0/585165766 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40a4085a90 con 0x7f40ac15cef0 2023-11-28T13:00:17.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.785+0000 7f40abfff700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40980682f0 0x7f409806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.785+0000 7f40abfff700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40980682f0 0x7f409806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a00090b0 tx=0x7f40a001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.789+0000 7f40a8ff9700 1 -- 172.21.15.153:0/585165766 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40a40522f0 con 0x7f40ac15cef0 2023-11-28T13:00:17.976 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.974+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40ac06b380 con 0x7f40ac15cef0 2023-11-28T13:00:17.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.975+0000 7f40a8ff9700 1 -- 172.21.15.153:0/585165766 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17968 (secure 0 0 0) 0x7f40a4055910 con 0x7f40ac15cef0 2023-11-28T13:00:17.977 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:17.977 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":24,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:17.624267+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40980682f0 msgr2=0x7f409806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40980682f0 0x7f409806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a00090b0 tx=0x7f40a001c040 comp rx=0 tx=0).stop 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 msgr2=0x7f40ac0b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0b4990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40a400cb90 tx=0x7f40a400cea0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 shutdown_connections 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40980682f0 0x7f409806a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40ac15cef0 0x7f40ac0b4990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40ac06ba90 0x7f40ac1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 --2- 172.21.15.153:0/585165766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40ac0cbdf0 0x7f40ac0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 >> 172.21.15.153:0/585165766 conn(0x7f40ac0d0000 msgr2=0x7f40ac06e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 shutdown_connections 2023-11-28T13:00:17.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:17.978+0000 7f40b2892700 1 -- 172.21.15.153:0/585165766 wait complete. 2023-11-28T13:00:17.992 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:18.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.110+0000 7f8f0bea5700 1 Processor -- start 2023-11-28T13:00:18.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f0bea5700 1 -- start start 2023-11-28T13:00:18.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f0bea5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8f04071040 0x7f8f04071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f0bea5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8f0406dee0 0x7f8f0406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f0bea5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f04071950 con 0x7f8f0406cdc0 2023-11-28T13:00:18.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f0bea5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f04161550 con 0x7f8f04071040 2023-11-28T13:00:18.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f0bea5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0415d060 con 0x7f8f0406dee0 2023-11-28T13:00:18.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f09c41700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8f04071040 0x7f8f04071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f09c41700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8f04071040 0x7f8f04071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38252/0 (socket says 172.21.15.153:38252) 2023-11-28T13:00:18.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.111+0000 7f8f09c41700 1 -- 172.21.15.153:0/794562718 learned_addr learned my addr 172.21.15.153:0/794562718 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:18.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f09440700 1 --2- 172.21.15.153:0/794562718 >> v2:172.21.15.177:3300/0 conn(0x7f8f0406dee0 0x7f8f0406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f09c41700 1 -- 172.21.15.153:0/794562718 >> v2:172.21.15.177:3300/0 conn(0x7f8f0406dee0 msgr2=0x7f8f0406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f09c41700 1 --2- 172.21.15.153:0/794562718 >> v2:172.21.15.177:3300/0 conn(0x7f8f0406dee0 0x7f8f0406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f09c41700 1 -- 172.21.15.153:0/794562718 >> v1:172.21.15.177:6789/0 conn(0x7f8f0406cdc0 legacy=0x7f8f0406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f09c41700 1 -- 172.21.15.153:0/794562718 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0415d340 con 0x7f8f04071040 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f09c41700 1 --2- 172.21.15.153:0/794562718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f04071040 0x7f8f04071410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f0406fb00 tx=0x7f8ef4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b5b120ac292ac3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f08c3f700 1 -- 172.21.15.153:0/794562718 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400b870 con 0x7f8f04071040 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f08c3f700 1 -- 172.21.15.153:0/794562718 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef400b9d0 con 0x7f8f04071040 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.112+0000 7f8f08c3f700 1 -- 172.21.15.153:0/794562718 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400bd60 con 0x7f8f04071040 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 -- 172.21.15.153:0/794562718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f04071040 msgr2=0x7f8f04071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/794562718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f04071040 0x7f8f04071410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f0406fb00 tx=0x7f8ef4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 -- 172.21.15.153:0/794562718 shutdown_connections 2023-11-28T13:00:18.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/794562718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f04071040 0x7f8f04071410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/794562718 >> v2:172.21.15.177:3300/0 conn(0x7f8f0406dee0 0x7f8f0406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 -- 172.21.15.153:0/794562718 >> 172.21.15.153:0/794562718 conn(0x7f8f040d0000 msgr2=0x7f8f040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 -- 172.21.15.153:0/794562718 shutdown_connections 2023-11-28T13:00:18.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.113+0000 7f8f0bea5700 1 -- 172.21.15.153:0/794562718 wait complete. 2023-11-28T13:00:18.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.114+0000 7f8f0bea5700 1 Processor -- start 2023-11-28T13:00:18.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.114+0000 7f8f0bea5700 1 -- start start 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f0406cdc0 0x7f8f040ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f0406dee0 0x7f8f040b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f04071040 0x7f8f040b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040ad640 con 0x7f8f0406dee0 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040ad920 con 0x7f8f0406cdc0 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040adc00 con 0x7f8f04071040 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f0406dee0 0x7f8f040b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f0406dee0 0x7f8f040b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38258/0 (socket says 172.21.15.153:38258) 2023-11-28T13:00:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 -- 172.21.15.153:0/1681363752 learned_addr learned my addr 172.21.15.153:0/1681363752 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09c41700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f0406cdc0 0x7f8f040ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0a442700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f04071040 0x7f8f040b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 -- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f04071040 msgr2=0x7f8f040b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f04071040 0x7f8f040b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 -- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f0406cdc0 msgr2=0x7f8f040ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f0406cdc0 0x7f8f040ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 -- 172.21.15.153:0/1681363752 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4009550 con 0x7f8f0406dee0 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0a442700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f04071040 0x7f8f040b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f09440700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f0406dee0 0x7f8f040b5940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d800 tx=0x7f8f0000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8efa7fc700 1 -- 172.21.15.153:0/1681363752 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000125f0 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8efa7fc700 1 -- 172.21.15.153:0/1681363752 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00012750 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0415d900 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8efa7fc700 1 -- 172.21.15.153:0/1681363752 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00012b00 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.115+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0415d620 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.116+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0406e870 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.117+0000 7f8efa7fc700 1 -- 172.21.15.153:0/1681363752 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8f00012c60 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.117+0000 7f8efa7fc700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ee80682f0 0x7f8ee806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.117+0000 7f8efa7fc700 1 -- 172.21.15.153:0/1681363752 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f000861b0 con 0x7f8f0406dee0 2023-11-28T13:00:18.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.117+0000 7f8f09c41700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ee80682f0 0x7f8ee806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.117+0000 7f8f09c41700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ee80682f0 0x7f8ee806a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4014040 tx=0x7f8ef4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.121+0000 7f8efa7fc700 1 -- 172.21.15.153:0/1681363752 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8f00052a10 con 0x7f8f0406dee0 2023-11-28T13:00:18.310 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.308+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f0406f4a0 con 0x7f8f0406dee0 2023-11-28T13:00:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.310+0000 7f8efa7fc700 1 -- 172.21.15.153:0/1681363752 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17968 (secure 0 0 0) 0x7f8f00056030 con 0x7f8f0406dee0 2023-11-28T13:00:18.312 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:18.312 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":24,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:17.624267+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ee80682f0 msgr2=0x7f8ee806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ee80682f0 0x7f8ee806a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4014040 tx=0x7f8ef4008030 comp rx=0 tx=0).stop 2023-11-28T13:00:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f0406dee0 msgr2=0x7f8f040b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f0406dee0 0x7f8f040b5940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000d800 tx=0x7f8f0000dbc0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 shutdown_connections 2023-11-28T13:00:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f04071040 0x7f8f040b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f0406cdc0 0x7f8f040ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ee80682f0 0x7f8ee806a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 --2- 172.21.15.153:0/1681363752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f0406dee0 0x7f8f040b5940 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 >> 172.21.15.153:0/1681363752 conn(0x7f8f040d0000 msgr2=0x7f8f0406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 shutdown_connections 2023-11-28T13:00:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.312+0000 7f8f0bea5700 1 -- 172.21.15.153:0/1681363752 wait complete. 2023-11-28T13:00:18.326 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:00:18.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.447+0000 7f7605e4f700 1 Processor -- start 2023-11-28T13:00:18.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.447+0000 7f7605e4f700 1 -- start start 2023-11-28T13:00:18.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f7605e4f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f76000ae9d0 0x7f760015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f7605e4f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f760006a5e0 0x7f760006a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f7605e4f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000aee40 con 0x7f760006e500 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f7605e4f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760015bf10 con 0x7f760006a5e0 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f7605e4f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760006c930 con 0x7f76000ae9d0 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75fffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f760006a5e0 0x7f760006a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75ff7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f760006e500 0x7f760006e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56428/0 (socket says 172.21.15.153:56428) 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75ff7fe700 1 -- 172.21.15.153:0/3972225900 learned_addr learned my addr 172.21.15.153:0/3972225900 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75feffd700 1 --2- 172.21.15.153:0/3972225900 >> v2:172.21.15.177:3300/0 conn(0x7f76000ae9d0 0x7f760015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75fffff700 1 -- 172.21.15.153:0/3972225900 >> v2:172.21.15.177:3300/0 conn(0x7f76000ae9d0 msgr2=0x7f760015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75fffff700 1 --2- 172.21.15.153:0/3972225900 >> v2:172.21.15.177:3300/0 conn(0x7f76000ae9d0 0x7f760015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75fffff700 1 -- 172.21.15.153:0/3972225900 >> v1:172.21.15.153:6789/0 conn(0x7f760006e500 legacy=0x7f760006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75fe7fc700 1 -- 172.21.15.153:0/3972225900 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807977490 0 0) 0x7f76000aee40 con 0x7f760006e500 2023-11-28T13:00:18.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75fffff700 1 -- 172.21.15.153:0/3972225900 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760006cc10 con 0x7f760006a5e0 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.448+0000 7f75fffff700 1 --2- 172.21.15.153:0/3972225900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f760006a9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f76000719c0 tx=0x7f75f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df14239e4a0fdca1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f75fe7fc700 1 -- 172.21.15.153:0/3972225900 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f800e070 con 0x7f760006a5e0 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f75fe7fc700 1 -- 172.21.15.153:0/3972225900 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f800bbd0 con 0x7f760006a5e0 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f75fe7fc700 1 -- 172.21.15.153:0/3972225900 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8012430 con 0x7f760006a5e0 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 -- 172.21.15.153:0/3972225900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 msgr2=0x7f760006a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 --2- 172.21.15.153:0/3972225900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f760006a9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f76000719c0 tx=0x7f75f800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 -- 172.21.15.153:0/3972225900 shutdown_connections 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 --2- 172.21.15.153:0/3972225900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f760006a9e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 --2- 172.21.15.153:0/3972225900 >> v2:172.21.15.177:3300/0 conn(0x7f76000ae9d0 0x7f760015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 -- 172.21.15.153:0/3972225900 >> 172.21.15.153:0/3972225900 conn(0x7f76000b12b0 msgr2=0x7f76000b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 -- 172.21.15.153:0/3972225900 shutdown_connections 2023-11-28T13:00:18.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.449+0000 7f7605e4f700 1 -- 172.21.15.153:0/3972225900 wait complete. 2023-11-28T13:00:18.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.451+0000 7f7605e4f700 1 Processor -- start 2023-11-28T13:00:18.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.451+0000 7f7605e4f700 1 -- start start 2023-11-28T13:00:18.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f7605e4f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f76001f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f7605e4f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f760006e500 0x7f7600155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f7605e4f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76000ae9d0 0x7f760015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f7605e4f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76001f6900 con 0x7f760006a5e0 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f7605e4f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7600156470 con 0x7f76000ae9d0 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f7605e4f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7600155b80 con 0x7f760006e500 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f75ff7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f76001f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f75ff7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f76001f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38270/0 (socket says 172.21.15.153:38270) 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f75ff7fe700 1 -- 172.21.15.153:0/4271341991 learned_addr learned my addr 172.21.15.153:0/4271341991 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f75fffff700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76000ae9d0 0x7f760015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.452+0000 7f75feffd700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f760006e500 0x7f7600155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75ff7fe700 1 -- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f760006e500 msgr2=0x7f7600155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75ff7fe700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f760006e500 0x7f7600155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75ff7fe700 1 -- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76000ae9d0 msgr2=0x7f760015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75ff7fe700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76000ae9d0 0x7f760015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75ff7fe700 1 -- 172.21.15.153:0/4271341991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8009a70 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75ff7fe700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f76001f63c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75f40011e0 tx=0x7f75f400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75e7fff700 1 -- 172.21.15.153:0/4271341991 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4004c80 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75e7fff700 1 -- 172.21.15.153:0/4271341991 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f4004e00 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760006cef0 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f75e7fff700 1 -- 172.21.15.153:0/4271341991 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4007310 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.453+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000664c0 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.454+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760015ac80 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.454+0000 7f75e7fff700 1 -- 172.21.15.153:0/4271341991 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f75f401a020 con 0x7f760006a5e0 2023-11-28T13:00:18.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.454+0000 7f75e7fff700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75ec0682f0 0x7f75ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.455+0000 7f75e7fff700 1 -- 172.21.15.153:0/4271341991 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f75f4085960 con 0x7f760006a5e0 2023-11-28T13:00:18.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.455+0000 7f75feffd700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75ec0682f0 0x7f75ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.455+0000 7f75feffd700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75ec0682f0 0x7f75ec06a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76000cdce0 tx=0x7f75f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.459+0000 7f75e7fff700 1 -- 172.21.15.153:0/4271341991 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75f40521c0 con 0x7f760006a5e0 2023-11-28T13:00:18.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:18.690+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7600065fc0 con 0x7f760006a5e0 2023-11-28T13:00:19.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.642+0000 7f75e7fff700 1 -- 172.21.15.153:0/4271341991 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f75f40557e0 con 0x7f760006a5e0 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75ec0682f0 msgr2=0x7f75ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75ec0682f0 0x7f75ec06a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76000cdce0 tx=0x7f75f0008040 comp rx=0 tx=0).stop 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 msgr2=0x7f76001f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f76001f63c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75f40011e0 tx=0x7f75f400cae0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 shutdown_connections 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75ec0682f0 0x7f75ec06a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f760006a5e0 0x7f76001f63c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76000ae9d0 0x7f760015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 --2- 172.21.15.153:0/4271341991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f760006e500 0x7f7600155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 >> 172.21.15.153:0/4271341991 conn(0x7f76000b12b0 msgr2=0x7f760006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:19.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 shutdown_connections 2023-11-28T13:00:19.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.645+0000 7f7605e4f700 1 -- 172.21.15.153:0/4271341991 wait complete. 2023-11-28T13:00:19.662 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:00:19.663 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:00:19.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.789+0000 7f471b149700 1 Processor -- start 2023-11-28T13:00:19.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.789+0000 7f471b149700 1 -- start start 2023-11-28T13:00:19.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f471b149700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4714160740 0x7f47140cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:19.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f471b149700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:19.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f471b149700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f471406b060 0x7f471406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:19.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f471b149700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140ca720 con 0x7f471406b060 2023-11-28T13:00:19.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f471b149700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140cc030 con 0x7f4714160740 2023-11-28T13:00:19.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f471b149700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47140c9e10 con 0x7f4714160b10 2023-11-28T13:00:19.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f47196e6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f471406b060 0x7f471406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:19.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f47196e6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f471406b060 0x7f471406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38282/0 (socket says 172.21.15.153:38282) 2023-11-28T13:00:19.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.790+0000 7f47196e6700 1 -- 172.21.15.153:0/3043575443 learned_addr learned my addr 172.21.15.153:0/3043575443 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:19.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f4718ee5700 1 --2- 172.21.15.153:0/3043575443 >> v2:172.21.15.177:3300/0 conn(0x7f4714160740 0x7f47140cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f4713fff700 1 --2- 172.21.15.153:0/3043575443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47196e6700 1 -- 172.21.15.153:0/3043575443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 msgr2=0x7f47140cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47196e6700 1 --2- 172.21.15.153:0/3043575443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47196e6700 1 -- 172.21.15.153:0/3043575443 >> v2:172.21.15.177:3300/0 conn(0x7f4714160740 msgr2=0x7f47140cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47196e6700 1 --2- 172.21.15.153:0/3043575443 >> v2:172.21.15.177:3300/0 conn(0x7f4714160740 0x7f47140cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47196e6700 1 -- 172.21.15.153:0/3043575443 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471406efc0 con 0x7f471406b060 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47196e6700 1 --2- 172.21.15.153:0/3043575443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f471406b4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47140b23b0 tx=0x7f470c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872a06131343d0ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47137fe700 1 -- 172.21.15.153:0/3043575443 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00e070 con 0x7f471406b060 2023-11-28T13:00:19.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47137fe700 1 -- 172.21.15.153:0/3043575443 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00b9c0 con 0x7f471406b060 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f47137fe700 1 -- 172.21.15.153:0/3043575443 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00bd70 con 0x7f471406b060 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f471b149700 1 -- 172.21.15.153:0/3043575443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 msgr2=0x7f471406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f471b149700 1 --2- 172.21.15.153:0/3043575443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f471406b4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47140b23b0 tx=0x7f470c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f471b149700 1 -- 172.21.15.153:0/3043575443 shutdown_connections 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f471b149700 1 --2- 172.21.15.153:0/3043575443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f471406b4e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f471b149700 1 --2- 172.21.15.153:0/3043575443 >> v2:172.21.15.177:3300/0 conn(0x7f4714160740 0x7f47140cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.791+0000 7f471b149700 1 --2- 172.21.15.153:0/3043575443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.792+0000 7f471b149700 1 -- 172.21.15.153:0/3043575443 >> 172.21.15.153:0/3043575443 conn(0x7f47140b12b0 msgr2=0x7f47140b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.792+0000 7f471b149700 1 -- 172.21.15.153:0/3043575443 shutdown_connections 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.792+0000 7f471b149700 1 -- 172.21.15.153:0/3043575443 wait complete. 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.793+0000 7f471b149700 1 Processor -- start 2023-11-28T13:00:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.793+0000 7f471b149700 1 -- start start 2023-11-28T13:00:19.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f4714159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:19.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714160740 0x7f47141570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:19.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47140c75f0 con 0x7f471406b060 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714159ce0 con 0x7f4714160740 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47140c7820 con 0x7f4714160b10 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f4714159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f4714159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38284/0 (socket says 172.21.15.153:38284) 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 -- 172.21.15.153:0/2599496109 learned_addr learned my addr 172.21.15.153:0/2599496109 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4713fff700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714160740 0x7f47141570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:19.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f47196e6700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 -- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 msgr2=0x7f47140c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 -- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714160740 msgr2=0x7f47141570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714160740 0x7f47141570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 -- 172.21.15.153:0/2599496109 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c009b30 con 0x7f471406b060 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f4718ee5700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f4714159650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f470800e810 tx=0x7f470800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f47117fa700 1 -- 172.21.15.153:0/2599496109 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708013610 con 0x7f471406b060 2023-11-28T13:00:19.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f47117fa700 1 -- 172.21.15.153:0/2599496109 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708013770 con 0x7f471406b060 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f47117fa700 1 -- 172.21.15.153:0/2599496109 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708013b20 con 0x7f471406b060 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47140ca3b0 con 0x7f471406b060 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.794+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47140ca0d0 con 0x7f471406b060 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.795+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47140cd8d0 con 0x7f471406b060 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.796+0000 7f47117fa700 1 -- 172.21.15.153:0/2599496109 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4708023070 con 0x7f471406b060 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.796+0000 7f47117fa700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f80682f0 0x7f46f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.796+0000 7f47117fa700 1 -- 172.21.15.153:0/2599496109 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4708085970 con 0x7f471406b060 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.796+0000 7f4713fff700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f80682f0 0x7f46f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.796+0000 7f4713fff700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f80682f0 0x7f46f806a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f471415ab20 tx=0x7f4700008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:19.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:19.800+0000 7f47117fa700 1 -- 172.21.15.153:0/2599496109 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4708052150 con 0x7f471406b060 2023-11-28T13:00:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.022+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4714159e50 con 0x7f471406b060 2023-11-28T13:00:20.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.647+0000 7f47117fa700 1 -- 172.21.15.153:0/2599496109 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4708055770 con 0x7f471406b060 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f80682f0 msgr2=0x7f46f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f80682f0 0x7f46f806a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f471415ab20 tx=0x7f4700008040 comp rx=0 tx=0).stop 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 msgr2=0x7f4714159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f4714159650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f470800e810 tx=0x7f470800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 shutdown_connections 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f80682f0 0x7f46f806a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f471406b060 0x7f4714159650 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4714160b10 0x7f47140c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 --2- 172.21.15.153:0/2599496109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714160740 0x7f47141570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 >> 172.21.15.153:0/2599496109 conn(0x7f47140b12b0 msgr2=0x7f4714160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:20.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 shutdown_connections 2023-11-28T13:00:20.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.649+0000 7f471b149700 1 -- 172.21.15.153:0/2599496109 wait complete. 2023-11-28T13:00:20.664 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:00:20.664 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:20.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.785+0000 7fcf2f6bb700 1 Processor -- start 2023-11-28T13:00:20.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.785+0000 7fcf2f6bb700 1 -- start start 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.786+0000 7fcf2f6bb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf2815cef0 0x7fcf280af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.786+0000 7fcf2f6bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2806ba90 0x7fcf280ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.786+0000 7fcf2f6bb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf280cc5f0 con 0x7fcf280cbdf0 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.786+0000 7fcf2f6bb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2815d470 con 0x7fcf2815cef0 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.786+0000 7fcf2f6bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2815d750 con 0x7fcf2806ba90 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2dc58700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2806ba90 0x7fcf280ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2d457700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf2815cef0 0x7fcf280af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2d457700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf2815cef0 0x7fcf280af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:47758/0 (socket says 172.21.15.153:47758) 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2dc58700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2806ba90 0x7fcf280ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:55906/0 (socket says 172.21.15.153:55906) 2023-11-28T13:00:20.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2d457700 1 -- 172.21.15.153:0/333486813 learned_addr learned my addr 172.21.15.153:0/333486813 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2cc56700 1 --1- 172.21.15.153:0/333486813 >> v1:172.21.15.177:6789/0 conn(0x7fcf280cbdf0 0x7fcf2806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33070/0 (socket says 172.21.15.153:33070) 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2dc58700 1 -- 172.21.15.153:0/333486813 >> v2:172.21.15.177:3300/0 conn(0x7fcf2815cef0 msgr2=0x7fcf280af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2dc58700 1 --2- 172.21.15.153:0/333486813 >> v2:172.21.15.177:3300/0 conn(0x7fcf2815cef0 0x7fcf280af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2dc58700 1 -- 172.21.15.153:0/333486813 >> v1:172.21.15.177:6789/0 conn(0x7fcf280cbdf0 legacy=0x7fcf2806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2dc58700 1 -- 172.21.15.153:0/333486813 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2815da30 con 0x7fcf2806ba90 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2d457700 1 --2- 172.21.15.153:0/333486813 >> v2:172.21.15.177:3300/0 conn(0x7fcf2815cef0 0x7fcf280af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf2dc58700 1 --2- 172.21.15.153:0/333486813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2806ba90 0x7fcf280ca650 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009dc0 tx=0x7fcf2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999b6516ebab38f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf1ffff700 1 -- 172.21.15.153:0/333486813 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000e070 con 0x7fcf2806ba90 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf1ffff700 1 -- 172.21.15.153:0/333486813 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2000bae0 con 0x7fcf2806ba90 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.787+0000 7fcf1ffff700 1 -- 172.21.15.153:0/333486813 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000be90 con 0x7fcf2806ba90 2023-11-28T13:00:20.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/333486813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2806ba90 msgr2=0x7fcf280ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/333486813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2806ba90 0x7fcf280ca650 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009dc0 tx=0x7fcf2000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:20.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/333486813 shutdown_connections 2023-11-28T13:00:20.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/333486813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2806ba90 0x7fcf280ca650 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/333486813 >> v2:172.21.15.177:3300/0 conn(0x7fcf2815cef0 0x7fcf280af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/333486813 >> 172.21.15.153:0/333486813 conn(0x7fcf280d0000 msgr2=0x7fcf280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:20.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/333486813 shutdown_connections 2023-11-28T13:00:20.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.788+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/333486813 wait complete. 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.789+0000 7fcf2f6bb700 1 Processor -- start 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.789+0000 7fcf2f6bb700 1 -- start start 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2f6bb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf2806ba90 0x7fcf281f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2f6bb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf280cbdf0 0x7fcf2815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2f6bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2815cef0 0x7fcf28159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2f6bb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280c70d0 con 0x7fcf2806ba90 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2f6bb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280c73b0 con 0x7fcf280cbdf0 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2f6bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280c7690 con 0x7fcf2815cef0 2023-11-28T13:00:20.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf2806ba90 0x7fcf281f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf2806ba90 0x7fcf281f2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38298/0 (socket says 172.21.15.153:38298) 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 -- 172.21.15.153:0/490928888 learned_addr learned my addr 172.21.15.153:0/490928888 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2cc56700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf280cbdf0 0x7fcf2815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2dc58700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2815cef0 0x7fcf28159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 -- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2815cef0 msgr2=0x7fcf28159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2815cef0 0x7fcf28159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 -- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf280cbdf0 msgr2=0x7fcf2815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf280cbdf0 0x7fcf2815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 -- 172.21.15.153:0/490928888 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18009550 con 0x7fcf2806ba90 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2cc56700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf280cbdf0 0x7fcf2815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:20.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.790+0000 7fcf2d457700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf2806ba90 0x7fcf281f2890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9b0 tx=0x7fcf1800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.791+0000 7fcf1dffb700 1 -- 172.21.15.153:0/490928888 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800c850 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.791+0000 7fcf1dffb700 1 -- 172.21.15.153:0/490928888 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1800c9b0 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.791+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf20009a70 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.791+0000 7fcf1dffb700 1 -- 172.21.15.153:0/490928888 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800cd20 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.791+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2815dff0 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.792+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2815dd10 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.792+0000 7fcf1dffb700 1 -- 172.21.15.153:0/490928888 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf18019030 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.793+0000 7fcf1dffb700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf0c0682f0 0x7fcf0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.793+0000 7fcf1dffb700 1 -- 172.21.15.153:0/490928888 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf18085c60 con 0x7fcf2806ba90 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.793+0000 7fcf2cc56700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf0c0682f0 0x7fcf0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.793+0000 7fcf2cc56700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf0c0682f0 0x7fcf0c06a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf280cdce0 tx=0x7fcf24007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:20.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:20.797+0000 7fcf1dffb700 1 -- 172.21.15.153:0/490928888 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf180524c0 con 0x7fcf2806ba90 2023-11-28T13:00:21.022 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.021+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf280c82a0 con 0x7fcf2806ba90 2023-11-28T13:00:21.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.021+0000 7fcf1dffb700 1 -- 172.21.15.153:0/490928888 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7fcf18017090 con 0x7fcf2806ba90 2023-11-28T13:00:21.023 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:21.023 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2499459281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2499459281},{"type":"v1","addr":"172.21.15.177:6834","nonce":2499459281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2708083457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2708083457},{"type":"v1","addr":"172.21.15.153:6837","nonce":2708083457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2730194389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2730194389},{"type":"v1","addr":"172.21.15.153:6836","nonce":2730194389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:17.624622+0000","modified":"2023-11-28T13:00:20.642491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/923823631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":923823631},{"type":"v1","addr":"172.21.15.177:6835","nonce":923823631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:00:21.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf0c0682f0 msgr2=0x7fcf0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf0c0682f0 0x7fcf0c06a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf280cdce0 tx=0x7fcf24007df0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf2806ba90 msgr2=0x7fcf281f2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf2806ba90 0x7fcf281f2890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800e9b0 tx=0x7fcf1800ed70 comp rx=0 tx=0).stop 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 shutdown_connections 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf0c0682f0 0x7fcf0c06a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf2806ba90 0x7fcf281f2890 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf2815cef0 0x7fcf28159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 --2- 172.21.15.153:0/490928888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf280cbdf0 0x7fcf2815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 >> 172.21.15.153:0/490928888 conn(0x7fcf280d0000 msgr2=0x7fcf2806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 shutdown_connections 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.023+0000 7fcf2f6bb700 1 -- 172.21.15.153:0/490928888 wait complete. 2023-11-28T13:00:21.026 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 8 2023-11-28T13:00:21.037 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:21.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.156+0000 7f46f3524700 1 Processor -- start 2023-11-28T13:00:21.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.157+0000 7f46f3524700 1 -- start start 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.157+0000 7f46f3524700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f46ec071040 0x7f46ec071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.157+0000 7f46f3524700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46ec06dee0 0x7f46ec06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.157+0000 7f46f3524700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ec071950 con 0x7f46ec06cdc0 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.157+0000 7f46f3524700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ec161550 con 0x7f46ec06dee0 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.157+0000 7f46f3524700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ec15d060 con 0x7f46ec071040 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.157+0000 7f46f0abf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46ec06dee0 0x7f46ec06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f0abf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46ec06dee0 0x7f46ec06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38306/0 (socket says 172.21.15.153:38306) 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f0abf700 1 -- 172.21.15.153:0/1103632621 learned_addr learned my addr 172.21.15.153:0/1103632621 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f12c0700 1 --2- 172.21.15.153:0/1103632621 >> v2:172.21.15.177:3300/0 conn(0x7f46ec071040 0x7f46ec071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f0abf700 1 -- 172.21.15.153:0/1103632621 >> v2:172.21.15.177:3300/0 conn(0x7f46ec071040 msgr2=0x7f46ec071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f0abf700 1 --2- 172.21.15.153:0/1103632621 >> v2:172.21.15.177:3300/0 conn(0x7f46ec071040 0x7f46ec071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f0abf700 1 -- 172.21.15.153:0/1103632621 >> v1:172.21.15.177:6789/0 conn(0x7f46ec06cdc0 legacy=0x7f46ec06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f0abf700 1 -- 172.21.15.153:0/1103632621 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec15d340 con 0x7f46ec06dee0 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f0abf700 1 --2- 172.21.15.153:0/1103632621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06dee0 0x7f46ec06e330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0d0f50 tx=0x7f46e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6aeff7bfb55aa22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46e3fff700 1 -- 172.21.15.153:0/1103632621 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8004230 con 0x7f46ec06dee0 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46e3fff700 1 -- 172.21.15.153:0/1103632621 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e8004390 con 0x7f46ec06dee0 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46e3fff700 1 -- 172.21.15.153:0/1103632621 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e80036a0 con 0x7f46ec06dee0 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f3524700 1 -- 172.21.15.153:0/1103632621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06dee0 msgr2=0x7f46ec06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.158+0000 7f46f3524700 1 --2- 172.21.15.153:0/1103632621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06dee0 0x7f46ec06e330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0d0f50 tx=0x7f46e8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.159+0000 7f46f3524700 1 -- 172.21.15.153:0/1103632621 shutdown_connections 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.159+0000 7f46f3524700 1 --2- 172.21.15.153:0/1103632621 >> v2:172.21.15.177:3300/0 conn(0x7f46ec071040 0x7f46ec071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.159+0000 7f46f3524700 1 --2- 172.21.15.153:0/1103632621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06dee0 0x7f46ec06e330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.159+0000 7f46f3524700 1 -- 172.21.15.153:0/1103632621 >> 172.21.15.153:0/1103632621 conn(0x7f46ec0d0000 msgr2=0x7f46ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.159+0000 7f46f3524700 1 -- 172.21.15.153:0/1103632621 shutdown_connections 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.159+0000 7f46f3524700 1 -- 172.21.15.153:0/1103632621 wait complete. 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.160+0000 7f46f3524700 1 Processor -- start 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.160+0000 7f46f3524700 1 -- start start 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06cdc0 0x7f46ec1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ec06dee0 0x7f46ec1e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ec071040 0x7f46ec1e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0ad830 con 0x7f46ec06cdc0 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec1f6610 con 0x7f46ec06dee0 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec0ace90 con 0x7f46ec071040 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06cdc0 0x7f46ec1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06cdc0 0x7f46ec1542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38312/0 (socket says 172.21.15.153:38312) 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 -- 172.21.15.153:0/1330582528 learned_addr learned my addr 172.21.15.153:0/1330582528 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f0abf700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ec06dee0 0x7f46ec1e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f1ac1700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ec071040 0x7f46ec1e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 -- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ec071040 msgr2=0x7f46ec1e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ec071040 0x7f46ec1e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 -- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ec06dee0 msgr2=0x7f46ec1e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ec06dee0 0x7f46ec1e6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 -- 172.21.15.153:0/1330582528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8009550 con 0x7f46ec06cdc0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f1ac1700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ec071040 0x7f46ec1e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f12c0700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06cdc0 0x7f46ec1542b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00d800 tx=0x7f46dc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46e1ffb700 1 -- 172.21.15.153:0/1330582528 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0125f0 con 0x7f46ec06cdc0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46e1ffb700 1 -- 172.21.15.153:0/1330582528 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc012750 con 0x7f46ec06cdc0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec15d900 con 0x7f46ec06cdc0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46e1ffb700 1 -- 172.21.15.153:0/1330582528 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc012b00 con 0x7f46ec06cdc0 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.161+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec15d620 con 0x7f46ec06cdc0 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.162+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec06e870 con 0x7f46ec06cdc0 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.163+0000 7f46e1ffb700 1 -- 172.21.15.153:0/1330582528 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46dc012c60 con 0x7f46ec06cdc0 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.163+0000 7f46e1ffb700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46d00682f0 0x7f46d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.163+0000 7f46e1ffb700 1 -- 172.21.15.153:0/1330582528 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46dc085f60 con 0x7f46ec06cdc0 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.163+0000 7f46f0abf700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46d00682f0 0x7f46d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.164+0000 7f46f0abf700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46d00682f0 0x7f46d006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009130 tx=0x7f46e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.167+0000 7f46e1ffb700 1 -- 172.21.15.153:0/1330582528 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46dc0527c0 con 0x7f46ec06cdc0 2023-11-28T13:00:21.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.350+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46ec0720a0 con 0x7f46ec06cdc0 2023-11-28T13:00:21.353 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.351+0000 7f46e1ffb700 1 -- 172.21.15.153:0/1330582528 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f46dc055de0 con 0x7f46ec06cdc0 2023-11-28T13:00:21.353 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:21.354 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:21.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46d00682f0 msgr2=0x7f46d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46d00682f0 0x7f46d006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009130 tx=0x7f46e8008030 comp rx=0 tx=0).stop 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06cdc0 msgr2=0x7f46ec1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06cdc0 0x7f46ec1542b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00d800 tx=0x7f46dc00db10 comp rx=0 tx=0).stop 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 shutdown_connections 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46d00682f0 0x7f46d006a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46ec06cdc0 0x7f46ec1542b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ec071040 0x7f46ec1e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 --2- 172.21.15.153:0/1330582528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ec06dee0 0x7f46ec1e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 >> 172.21.15.153:0/1330582528 conn(0x7f46ec0d0000 msgr2=0x7f46ec06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 shutdown_connections 2023-11-28T13:00:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.353+0000 7f46f3524700 1 -- 172.21.15.153:0/1330582528 wait complete. 2023-11-28T13:00:21.367 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:00:21.368 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.485+0000 7fa27236f700 1 Processor -- start 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.485+0000 7fa27236f700 1 -- start start 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa27236f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c06dee0 0x7fa26c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa27236f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa26c160540 0x7fa26c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa27236f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c15d060 con 0x7fa26c071040 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa27236f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c15d340 con 0x7fa26c160540 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa27236f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c15d620 con 0x7fa26c06dee0 2023-11-28T13:00:21.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa26bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa26c071040 0x7fa26c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56484/0 (socket says 172.21.15.153:56484) 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa26bfff700 1 -- 172.21.15.153:0/2192565977 learned_addr learned my addr 172.21.15.153:0/2192565977 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.486+0000 7fa27090c700 1 --2- 172.21.15.153:0/2192565977 >> v2:172.21.15.153:3300/0 conn(0x7fa26c160540 0x7fa26c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa26b7fe700 1 --2- 172.21.15.153:0/2192565977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c06dee0 0x7fa26c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa26affd700 1 -- 172.21.15.153:0/2192565977 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103719830 0 0) 0x7fa26c15d060 con 0x7fa26c071040 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa26affd700 1 -- 172.21.15.153:0/2192565977 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2540035b0 con 0x7fa26c071040 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa27090c700 1 -- 172.21.15.153:0/2192565977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c06dee0 msgr2=0x7fa26c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa27090c700 1 --2- 172.21.15.153:0/2192565977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c06dee0 0x7fa26c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa27090c700 1 -- 172.21.15.153:0/2192565977 >> v1:172.21.15.153:6789/0 conn(0x7fa26c071040 legacy=0x7fa26c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa27090c700 1 -- 172.21.15.153:0/2192565977 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c15d900 con 0x7fa26c160540 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa27090c700 1 --2- 172.21.15.153:0/2192565977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c1609c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa264009d90 tx=0x7fa26400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8133b3dfcdffad2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa26affd700 1 -- 172.21.15.153:0/2192565977 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26400e070 con 0x7fa26c160540 2023-11-28T13:00:21.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa26affd700 1 -- 172.21.15.153:0/2192565977 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26400bb20 con 0x7fa26c160540 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.487+0000 7fa26affd700 1 -- 172.21.15.153:0/2192565977 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264012430 con 0x7fa26c160540 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 -- 172.21.15.153:0/2192565977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 msgr2=0x7fa26c1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 --2- 172.21.15.153:0/2192565977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c1609c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa264009d90 tx=0x7fa26400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 -- 172.21.15.153:0/2192565977 shutdown_connections 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 --2- 172.21.15.153:0/2192565977 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c1609c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 --2- 172.21.15.153:0/2192565977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c06dee0 0x7fa26c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 -- 172.21.15.153:0/2192565977 >> 172.21.15.153:0/2192565977 conn(0x7fa26c0d0000 msgr2=0x7fa26c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 -- 172.21.15.153:0/2192565977 shutdown_connections 2023-11-28T13:00:21.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.488+0000 7fa27236f700 1 -- 172.21.15.153:0/2192565977 wait complete. 2023-11-28T13:00:21.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.489+0000 7fa27236f700 1 Processor -- start 2023-11-28T13:00:21.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.489+0000 7fa27236f700 1 -- start start 2023-11-28T13:00:21.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27236f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa26c06dee0 0x7fa26c14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27236f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c071040 0x7fa26c0c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27236f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c0c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27236f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c1f69a0 con 0x7fa26c160540 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27236f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0b2ed0 con 0x7fa26c06dee0 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27236f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0b31b0 con 0x7fa26c071040 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c0c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c0c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38330/0 (socket says 172.21.15.153:38330) 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 -- 172.21.15.153:0/372146621 learned_addr learned my addr 172.21.15.153:0/372146621 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa26bfff700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa26c06dee0 0x7fa26c14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa26b7fe700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c071040 0x7fa26c0c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 -- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c071040 msgr2=0x7fa26c0c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c071040 0x7fa26c0c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 -- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa26c06dee0 msgr2=0x7fa26c14db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa26c06dee0 0x7fa26c14db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 -- 172.21.15.153:0/372146621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264009a70 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27090c700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c0c6320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa26400b370 tx=0x7fa264004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa268ff9700 1 -- 172.21.15.153:0/372146621 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264012430 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa268ff9700 1 -- 172.21.15.153:0/372146621 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa264020070 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa268ff9700 1 -- 172.21.15.153:0/372146621 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264004280 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.490+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c15ccd0 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.491+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c06e870 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.492+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c06a270 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.492+0000 7fa268ff9700 1 -- 172.21.15.153:0/372146621 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa26402e070 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.492+0000 7fa268ff9700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa2580682f0 0x7fa25806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.492+0000 7fa268ff9700 1 -- 172.21.15.153:0/372146621 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa264022070 con 0x7fa26c160540 2023-11-28T13:00:21.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.492+0000 7fa26bfff700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa2580682f0 0x7fa25806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.492+0000 7fa26bfff700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa2580682f0 0x7fa25806a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0011e0 tx=0x7fa25c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.496+0000 7fa268ff9700 1 -- 172.21.15.153:0/372146621 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa2640541e0 con 0x7fa26c160540 2023-11-28T13:00:21.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.679+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa26c0b3040 con 0x7fa2580682f0 2023-11-28T13:00:21.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.688+0000 7fa268ff9700 1 -- 172.21.15.153:0/372146621 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286273 (secure 0 0 0) 0x7fa26c0b3040 con 0x7fa2580682f0 2023-11-28T13:00:21.693 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:21.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa2580682f0 msgr2=0x7fa25806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa2580682f0 0x7fa25806a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0011e0 tx=0x7fa25c009040 comp rx=0 tx=0).stop 2023-11-28T13:00:21.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 msgr2=0x7fa26c0c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c0c6320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa26400b370 tx=0x7fa264004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 shutdown_connections 2023-11-28T13:00:21.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa2580682f0 0x7fa25806a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa26c160540 0x7fa26c0c6320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa26c06dee0 0x7fa26c14db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 --2- 172.21.15.153:0/372146621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa26c071040 0x7fa26c0c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 >> 172.21.15.153:0/372146621 conn(0x7fa26c0d0000 msgr2=0x7fa26c0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 shutdown_connections 2023-11-28T13:00:21.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.691+0000 7fa27236f700 1 -- 172.21.15.153:0/372146621 wait complete. 2023-11-28T13:00:21.697 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:00:21.705 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-28T13:00:20.135856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592132,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66188,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908532,"statfs":{"total":773094113280,"available":773026336768,"internally_reserved":0,"allocated":3223552,"data_stored":1911904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000784"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217825+0000","last_change":"2023-11-28T13:00:17.635428+0000","last_active":"2023-11-28T13:00:19.217825+0000","last_peered":"2023-11-28T13:00:19.217825+0000","last_clean":"2023-11-28T13:00:19.217825+0000","last_became_active":"2023-11-28T13:00:17.635148+0000","last_became_peered":"2023-11-28T13:00:17.635148+0000","last_unstale":"2023-11-28T13:00:19.217825+0000","last_undegraded":"2023-11-28T13:00:19.217825+0000","last_fullsized":"2023-11-28T13:00:19.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648693+0000","last_change":"2023-11-28T13:00:17.645017+0000","last_active":"2023-11-28T13:00:17.648693+0000","last_peered":"2023-11-28T13:00:17.648693+0000","last_clean":"2023-11-28T13:00:17.648693+0000","last_became_active":"2023-11-28T13:00:17.644524+0000","last_became_peered":"2023-11-28T13:00:17.644524+0000","last_unstale":"2023-11-28T13:00:17.648693+0000","last_undegraded":"2023-11-28T13:00:17.648693+0000","last_fullsized":"2023-11-28T13:00:17.648693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656057+0000","last_change":"2023-11-28T13:00:17.648662+0000","last_active":"2023-11-28T13:00:17.656057+0000","last_peered":"2023-11-28T13:00:17.656057+0000","last_clean":"2023-11-28T13:00:17.656057+0000","last_became_active":"2023-11-28T13:00:17.648271+0000","last_became_peered":"2023-11-28T13:00:17.648271+0000","last_unstale":"2023-11-28T13:00:17.656057+0000","last_undegraded":"2023-11-28T13:00:17.656057+0000","last_fullsized":"2023-11-28T13:00:17.656057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:37.103258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647723+0000","last_change":"2023-11-28T13:00:17.643733+0000","last_active":"2023-11-28T13:00:17.647723+0000","last_peered":"2023-11-28T13:00:17.647723+0000","last_clean":"2023-11-28T13:00:17.647723+0000","last_became_active":"2023-11-28T13:00:17.643428+0000","last_became_peered":"2023-11-28T13:00:17.643428+0000","last_unstale":"2023-11-28T13:00:17.647723+0000","last_undegraded":"2023-11-28T13:00:17.647723+0000","last_fullsized":"2023-11-28T13:00:17.647723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233853+0000","last_change":"2023-11-28T13:00:17.233853+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233853+0000","last_undegraded":"2023-11-28T13:00:17.233853+0000","last_fullsized":"2023-11-28T13:00:17.233853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658806+0000","last_change":"2023-11-28T13:00:17.652679+0000","last_active":"2023-11-28T13:00:17.658806+0000","last_peered":"2023-11-28T13:00:17.658806+0000","last_clean":"2023-11-28T13:00:17.658806+0000","last_became_active":"2023-11-28T13:00:17.648169+0000","last_became_peered":"2023-11-28T13:00:17.648169+0000","last_unstale":"2023-11-28T13:00:17.658806+0000","last_undegraded":"2023-11-28T13:00:17.658806+0000","last_fullsized":"2023-11-28T13:00:17.658806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:58.762522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649103+0000","last_change":"2023-11-28T13:00:17.647769+0000","last_active":"2023-11-28T13:00:17.649103+0000","last_peered":"2023-11-28T13:00:17.649103+0000","last_clean":"2023-11-28T13:00:17.649103+0000","last_became_active":"2023-11-28T13:00:17.647476+0000","last_became_peered":"2023-11-28T13:00:17.647476+0000","last_unstale":"2023-11-28T13:00:17.649103+0000","last_undegraded":"2023-11-28T13:00:17.649103+0000","last_fullsized":"2023-11-28T13:00:17.649103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653892+0000","last_change":"2023-11-28T13:00:17.653233+0000","last_active":"2023-11-28T13:00:17.653892+0000","last_peered":"2023-11-28T13:00:17.653892+0000","last_clean":"2023-11-28T13:00:17.653892+0000","last_became_active":"2023-11-28T13:00:17.652957+0000","last_became_peered":"2023-11-28T13:00:17.652957+0000","last_unstale":"2023-11-28T13:00:17.653892+0000","last_undegraded":"2023-11-28T13:00:17.653892+0000","last_fullsized":"2023-11-28T13:00:17.653892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231803+0000","last_change":"2023-11-28T13:00:17.231803+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231803+0000","last_undegraded":"2023-11-28T13:00:17.231803+0000","last_fullsized":"2023-11-28T13:00:17.231803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645112+0000","last_change":"2023-11-28T13:00:17.642144+0000","last_active":"2023-11-28T13:00:17.645112+0000","last_peered":"2023-11-28T13:00:17.645112+0000","last_clean":"2023-11-28T13:00:17.645112+0000","last_became_active":"2023-11-28T13:00:17.641814+0000","last_became_peered":"2023-11-28T13:00:17.641814+0000","last_unstale":"2023-11-28T13:00:17.645112+0000","last_undegraded":"2023-11-28T13:00:17.645112+0000","last_fullsized":"2023-11-28T13:00:17.645112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653974+0000","last_change":"2023-11-28T13:00:17.643190+0000","last_active":"2023-11-28T13:00:17.653974+0000","last_peered":"2023-11-28T13:00:17.653974+0000","last_clean":"2023-11-28T13:00:17.653974+0000","last_became_active":"2023-11-28T13:00:17.642741+0000","last_became_peered":"2023-11-28T13:00:17.642741+0000","last_unstale":"2023-11-28T13:00:17.653974+0000","last_undegraded":"2023-11-28T13:00:17.653974+0000","last_fullsized":"2023-11-28T13:00:17.653974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:33.241895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217739+0000","last_change":"2023-11-28T13:00:17.654503+0000","last_active":"2023-11-28T13:00:19.217739+0000","last_peered":"2023-11-28T13:00:19.217739+0000","last_clean":"2023-11-28T13:00:19.217739+0000","last_became_active":"2023-11-28T13:00:17.654204+0000","last_became_peered":"2023-11-28T13:00:17.654204+0000","last_unstale":"2023-11-28T13:00:19.217739+0000","last_undegraded":"2023-11-28T13:00:19.217739+0000","last_fullsized":"2023-11-28T13:00:19.217739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:27.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217678+0000","last_change":"2023-11-28T13:00:17.634718+0000","last_active":"2023-11-28T13:00:19.217678+0000","last_peered":"2023-11-28T13:00:19.217678+0000","last_clean":"2023-11-28T13:00:19.217678+0000","last_became_active":"2023-11-28T13:00:17.634420+0000","last_became_peered":"2023-11-28T13:00:17.634420+0000","last_unstale":"2023-11-28T13:00:19.217678+0000","last_undegraded":"2023-11-28T13:00:19.217678+0000","last_fullsized":"2023-11-28T13:00:19.217678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.567102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645127+0000","last_change":"2023-11-28T13:00:17.641361+0000","last_active":"2023-11-28T13:00:17.645127+0000","last_peered":"2023-11-28T13:00:17.645127+0000","last_clean":"2023-11-28T13:00:17.645127+0000","last_became_active":"2023-11-28T13:00:17.641078+0000","last_became_peered":"2023-11-28T13:00:17.641078+0000","last_unstale":"2023-11-28T13:00:17.645127+0000","last_undegraded":"2023-11-28T13:00:17.645127+0000","last_fullsized":"2023-11-28T13:00:17.645127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:38.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649123+0000","last_change":"2023-11-28T13:00:17.647800+0000","last_active":"2023-11-28T13:00:17.649123+0000","last_peered":"2023-11-28T13:00:17.649123+0000","last_clean":"2023-11-28T13:00:17.649123+0000","last_became_active":"2023-11-28T13:00:17.647519+0000","last_became_peered":"2023-11-28T13:00:17.647519+0000","last_unstale":"2023-11-28T13:00:17.649123+0000","last_undegraded":"2023-11-28T13:00:17.649123+0000","last_fullsized":"2023-11-28T13:00:17.649123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656251+0000","last_change":"2023-11-28T13:00:17.651090+0000","last_active":"2023-11-28T13:00:17.656251+0000","last_peered":"2023-11-28T13:00:17.656251+0000","last_clean":"2023-11-28T13:00:17.656251+0000","last_became_active":"2023-11-28T13:00:17.649174+0000","last_became_peered":"2023-11-28T13:00:17.649174+0000","last_unstale":"2023-11-28T13:00:17.656251+0000","last_undegraded":"2023-11-28T13:00:17.656251+0000","last_fullsized":"2023-11-28T13:00:17.656251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:32.070272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231312+0000","last_change":"2023-11-28T13:00:17.231312+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231312+0000","last_undegraded":"2023-11-28T13:00:17.231312+0000","last_fullsized":"2023-11-28T13:00:17.231312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656166+0000","last_change":"2023-11-28T13:00:17.648771+0000","last_active":"2023-11-28T13:00:17.656166+0000","last_peered":"2023-11-28T13:00:17.656166+0000","last_clean":"2023-11-28T13:00:17.656166+0000","last_became_active":"2023-11-28T13:00:17.646367+0000","last_became_peered":"2023-11-28T13:00:17.646367+0000","last_unstale":"2023-11-28T13:00:17.656166+0000","last_undegraded":"2023-11-28T13:00:17.656166+0000","last_fullsized":"2023-11-28T13:00:17.656166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:33.549492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217793+0000","last_change":"2023-11-28T13:00:17.654191+0000","last_active":"2023-11-28T13:00:19.217793+0000","last_peered":"2023-11-28T13:00:19.217793+0000","last_clean":"2023-11-28T13:00:19.217793+0000","last_became_active":"2023-11-28T13:00:17.653922+0000","last_became_peered":"2023-11-28T13:00:17.653922+0000","last_unstale":"2023-11-28T13:00:19.217793+0000","last_undegraded":"2023-11-28T13:00:19.217793+0000","last_fullsized":"2023-11-28T13:00:19.217793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.066202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658937+0000","last_change":"2023-11-28T13:00:17.653882+0000","last_active":"2023-11-28T13:00:17.658937+0000","last_peered":"2023-11-28T13:00:17.658937+0000","last_clean":"2023-11-28T13:00:17.658937+0000","last_became_active":"2023-11-28T13:00:17.653497+0000","last_became_peered":"2023-11-28T13:00:17.653497+0000","last_unstale":"2023-11-28T13:00:17.658937+0000","last_undegraded":"2023-11-28T13:00:17.658937+0000","last_fullsized":"2023-11-28T13:00:17.658937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:59.591407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654270+0000","last_change":"2023-11-28T13:00:17.652105+0000","last_active":"2023-11-28T13:00:17.654270+0000","last_peered":"2023-11-28T13:00:17.654270+0000","last_clean":"2023-11-28T13:00:17.654270+0000","last_became_active":"2023-11-28T13:00:17.651804+0000","last_became_peered":"2023-11-28T13:00:17.651804+0000","last_unstale":"2023-11-28T13:00:17.654270+0000","last_undegraded":"2023-11-28T13:00:17.654270+0000","last_fullsized":"2023-11-28T13:00:17.654270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232480+0000","last_change":"2023-11-28T13:00:17.232480+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232480+0000","last_undegraded":"2023-11-28T13:00:17.232480+0000","last_fullsized":"2023-11-28T13:00:17.232480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215336+0000","last_change":"2023-11-28T13:00:17.645773+0000","last_active":"2023-11-28T13:00:19.215336+0000","last_peered":"2023-11-28T13:00:19.215336+0000","last_clean":"2023-11-28T13:00:19.215336+0000","last_became_active":"2023-11-28T13:00:17.643545+0000","last_became_peered":"2023-11-28T13:00:17.643545+0000","last_unstale":"2023-11-28T13:00:19.215336+0000","last_undegraded":"2023-11-28T13:00:19.215336+0000","last_fullsized":"2023-11-28T13:00:19.215336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:36.504863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654082+0000","last_change":"2023-11-28T13:00:17.651625+0000","last_active":"2023-11-28T13:00:17.654082+0000","last_peered":"2023-11-28T13:00:17.654082+0000","last_clean":"2023-11-28T13:00:17.654082+0000","last_became_active":"2023-11-28T13:00:17.651210+0000","last_became_peered":"2023-11-28T13:00:17.651210+0000","last_unstale":"2023-11-28T13:00:17.654082+0000","last_undegraded":"2023-11-28T13:00:17.654082+0000","last_fullsized":"2023-11-28T13:00:17.654082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635880+0000","last_change":"2023-11-28T13:00:17.646690+0000","last_active":"2023-11-28T13:00:18.635880+0000","last_peered":"2023-11-28T13:00:18.635880+0000","last_clean":"2023-11-28T13:00:18.635880+0000","last_became_active":"2023-11-28T13:00:17.646438+0000","last_became_peered":"2023-11-28T13:00:17.646438+0000","last_unstale":"2023-11-28T13:00:18.635880+0000","last_undegraded":"2023-11-28T13:00:18.635880+0000","last_fullsized":"2023-11-28T13:00:18.635880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:52.870009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636047+0000","last_change":"2023-11-28T13:00:17.641404+0000","last_active":"2023-11-28T13:00:18.636047+0000","last_peered":"2023-11-28T13:00:18.636047+0000","last_clean":"2023-11-28T13:00:18.636047+0000","last_became_active":"2023-11-28T13:00:17.641140+0000","last_became_peered":"2023-11-28T13:00:17.641140+0000","last_unstale":"2023-11-28T13:00:18.636047+0000","last_undegraded":"2023-11-28T13:00:18.636047+0000","last_fullsized":"2023-11-28T13:00:18.636047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648907+0000","last_change":"2023-11-28T13:00:17.645292+0000","last_active":"2023-11-28T13:00:17.648907+0000","last_peered":"2023-11-28T13:00:17.648907+0000","last_clean":"2023-11-28T13:00:17.648907+0000","last_became_active":"2023-11-28T13:00:17.645001+0000","last_became_peered":"2023-11-28T13:00:17.645001+0000","last_unstale":"2023-11-28T13:00:17.648907+0000","last_undegraded":"2023-11-28T13:00:17.648907+0000","last_fullsized":"2023-11-28T13:00:17.648907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652791+0000","last_change":"2023-11-28T13:00:17.245858+0000","last_active":"2023-11-28T13:00:17.652791+0000","last_peered":"2023-11-28T13:00:17.652791+0000","last_clean":"2023-11-28T13:00:17.652791+0000","last_became_active":"2023-11-28T13:00:17.245568+0000","last_became_peered":"2023-11-28T13:00:17.245568+0000","last_unstale":"2023-11-28T13:00:17.652791+0000","last_undegraded":"2023-11-28T13:00:17.652791+0000","last_fullsized":"2023-11-28T13:00:17.652791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215475+0000","last_change":"2023-11-28T13:00:17.644504+0000","last_active":"2023-11-28T13:00:19.215475+0000","last_peered":"2023-11-28T13:00:19.215475+0000","last_clean":"2023-11-28T13:00:19.215475+0000","last_became_active":"2023-11-28T13:00:17.644099+0000","last_became_peered":"2023-11-28T13:00:17.644099+0000","last_unstale":"2023-11-28T13:00:19.215475+0000","last_undegraded":"2023-11-28T13:00:19.215475+0000","last_fullsized":"2023-11-28T13:00:19.215475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635846+0000","last_change":"2023-11-28T13:00:17.236124+0000","last_active":"2023-11-28T13:00:18.635846+0000","last_peered":"2023-11-28T13:00:18.635846+0000","last_clean":"2023-11-28T13:00:18.635846+0000","last_became_active":"2023-11-28T13:00:17.235500+0000","last_became_peered":"2023-11-28T13:00:17.235500+0000","last_unstale":"2023-11-28T13:00:18.635846+0000","last_undegraded":"2023-11-28T13:00:18.635846+0000","last_fullsized":"2023-11-28T13:00:18.635846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:19.252999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.234634+0000","last_change":"2023-11-28T13:00:17.234634+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.234634+0000","last_undegraded":"2023-11-28T13:00:17.234634+0000","last_fullsized":"2023-11-28T13:00:17.234634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215390+0000","last_change":"2023-11-28T13:00:17.240445+0000","last_active":"2023-11-28T13:00:19.215390+0000","last_peered":"2023-11-28T13:00:19.215390+0000","last_clean":"2023-11-28T13:00:19.215390+0000","last_became_active":"2023-11-28T13:00:17.240095+0000","last_became_peered":"2023-11-28T13:00:17.240095+0000","last_unstale":"2023-11-28T13:00:19.215390+0000","last_undegraded":"2023-11-28T13:00:19.215390+0000","last_fullsized":"2023-11-28T13:00:19.215390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648526+0000","last_change":"2023-11-28T13:00:17.644329+0000","last_active":"2023-11-28T13:00:17.648526+0000","last_peered":"2023-11-28T13:00:17.648526+0000","last_clean":"2023-11-28T13:00:17.648526+0000","last_became_active":"2023-11-28T13:00:17.644046+0000","last_became_peered":"2023-11-28T13:00:17.644046+0000","last_unstale":"2023-11-28T13:00:17.648526+0000","last_undegraded":"2023-11-28T13:00:17.648526+0000","last_fullsized":"2023-11-28T13:00:17.648526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.849042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247832+0000","last_change":"2023-11-28T13:00:17.242514+0000","last_active":"2023-11-28T13:00:17.247832+0000","last_peered":"2023-11-28T13:00:17.247832+0000","last_clean":"2023-11-28T13:00:17.247832+0000","last_became_active":"2023-11-28T13:00:17.242258+0000","last_became_peered":"2023-11-28T13:00:17.242258+0000","last_unstale":"2023-11-28T13:00:17.247832+0000","last_undegraded":"2023-11-28T13:00:17.247832+0000","last_fullsized":"2023-11-28T13:00:17.247832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215295+0000","last_change":"2023-11-28T13:00:17.642442+0000","last_active":"2023-11-28T13:00:19.215295+0000","last_peered":"2023-11-28T13:00:19.215295+0000","last_clean":"2023-11-28T13:00:19.215295+0000","last_became_active":"2023-11-28T13:00:17.642162+0000","last_became_peered":"2023-11-28T13:00:17.642162+0000","last_unstale":"2023-11-28T13:00:19.215295+0000","last_undegraded":"2023-11-28T13:00:19.215295+0000","last_fullsized":"2023-11-28T13:00:19.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.635989+0000","last_change":"2023-11-28T13:00:17.238867+0000","last_active":"2023-11-28T13:00:17.635989+0000","last_peered":"2023-11-28T13:00:17.635989+0000","last_clean":"2023-11-28T13:00:17.635989+0000","last_became_active":"2023-11-28T13:00:17.238545+0000","last_became_peered":"2023-11-28T13:00:17.238545+0000","last_unstale":"2023-11-28T13:00:17.635989+0000","last_undegraded":"2023-11-28T13:00:17.635989+0000","last_fullsized":"2023-11-28T13:00:17.635989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215335+0000","last_change":"2023-11-28T13:00:17.645527+0000","last_active":"2023-11-28T13:00:19.215335+0000","last_peered":"2023-11-28T13:00:19.215335+0000","last_clean":"2023-11-28T13:00:19.215335+0000","last_became_active":"2023-11-28T13:00:17.644057+0000","last_became_peered":"2023-11-28T13:00:17.644057+0000","last_unstale":"2023-11-28T13:00:19.215335+0000","last_undegraded":"2023-11-28T13:00:19.215335+0000","last_fullsized":"2023-11-28T13:00:19.215335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:55.013485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651526+0000","last_change":"2023-11-28T13:00:17.242375+0000","last_active":"2023-11-28T13:00:17.651526+0000","last_peered":"2023-11-28T13:00:17.651526+0000","last_clean":"2023-11-28T13:00:17.651526+0000","last_became_active":"2023-11-28T13:00:17.242010+0000","last_became_peered":"2023-11-28T13:00:17.242010+0000","last_unstale":"2023-11-28T13:00:17.651526+0000","last_undegraded":"2023-11-28T13:00:17.651526+0000","last_fullsized":"2023-11-28T13:00:17.651526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.522601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635882+0000","last_change":"2023-11-28T13:00:17.647427+0000","last_active":"2023-11-28T13:00:18.635882+0000","last_peered":"2023-11-28T13:00:18.635882+0000","last_clean":"2023-11-28T13:00:18.635882+0000","last_became_active":"2023-11-28T13:00:17.646853+0000","last_became_peered":"2023-11-28T13:00:17.646853+0000","last_unstale":"2023-11-28T13:00:18.635882+0000","last_undegraded":"2023-11-28T13:00:18.635882+0000","last_fullsized":"2023-11-28T13:00:18.635882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.661600+0000","last_change":"2023-11-28T13:00:17.243823+0000","last_active":"2023-11-28T13:00:17.661600+0000","last_peered":"2023-11-28T13:00:17.661600+0000","last_clean":"2023-11-28T13:00:17.661600+0000","last_became_active":"2023-11-28T13:00:17.241238+0000","last_became_peered":"2023-11-28T13:00:17.241238+0000","last_unstale":"2023-11-28T13:00:17.661600+0000","last_undegraded":"2023-11-28T13:00:17.661600+0000","last_fullsized":"2023-11-28T13:00:17.661600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656243+0000","last_change":"2023-11-28T13:00:17.649986+0000","last_active":"2023-11-28T13:00:17.656243+0000","last_peered":"2023-11-28T13:00:17.656243+0000","last_clean":"2023-11-28T13:00:17.656243+0000","last_became_active":"2023-11-28T13:00:17.649402+0000","last_became_peered":"2023-11-28T13:00:17.649402+0000","last_unstale":"2023-11-28T13:00:17.656243+0000","last_undegraded":"2023-11-28T13:00:17.656243+0000","last_fullsized":"2023-11-28T13:00:17.656243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:15.471790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656508+0000","last_change":"2023-11-28T13:00:17.247047+0000","last_active":"2023-11-28T13:00:17.656508+0000","last_peered":"2023-11-28T13:00:17.656508+0000","last_clean":"2023-11-28T13:00:17.656508+0000","last_became_active":"2023-11-28T13:00:17.246742+0000","last_became_peered":"2023-11-28T13:00:17.246742+0000","last_unstale":"2023-11-28T13:00:17.656508+0000","last_undegraded":"2023-11-28T13:00:17.656508+0000","last_fullsized":"2023-11-28T13:00:17.656508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:50.559956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215527+0000","last_change":"2023-11-28T13:00:17.641605+0000","last_active":"2023-11-28T13:00:19.215527+0000","last_peered":"2023-11-28T13:00:19.215527+0000","last_clean":"2023-11-28T13:00:19.215527+0000","last_became_active":"2023-11-28T13:00:17.641350+0000","last_became_peered":"2023-11-28T13:00:17.641350+0000","last_unstale":"2023-11-28T13:00:19.215527+0000","last_undegraded":"2023-11-28T13:00:19.215527+0000","last_fullsized":"2023-11-28T13:00:19.215527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245165+0000","last_change":"2023-11-28T13:00:17.240432+0000","last_active":"2023-11-28T13:00:17.245165+0000","last_peered":"2023-11-28T13:00:17.245165+0000","last_clean":"2023-11-28T13:00:17.245165+0000","last_became_active":"2023-11-28T13:00:17.239192+0000","last_became_peered":"2023-11-28T13:00:17.239192+0000","last_unstale":"2023-11-28T13:00:17.245165+0000","last_undegraded":"2023-11-28T13:00:17.245165+0000","last_fullsized":"2023-11-28T13:00:17.245165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651770+0000","last_change":"2023-11-28T13:00:17.648774+0000","last_active":"2023-11-28T13:00:17.651770+0000","last_peered":"2023-11-28T13:00:17.651770+0000","last_clean":"2023-11-28T13:00:17.651770+0000","last_became_active":"2023-11-28T13:00:17.647403+0000","last_became_peered":"2023-11-28T13:00:17.647403+0000","last_unstale":"2023-11-28T13:00:17.651770+0000","last_undegraded":"2023-11-28T13:00:17.651770+0000","last_fullsized":"2023-11-28T13:00:17.651770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:17.592744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633096+0000","last_change":"2023-11-28T13:00:17.245977+0000","last_active":"2023-11-28T13:00:17.633096+0000","last_peered":"2023-11-28T13:00:17.633096+0000","last_clean":"2023-11-28T13:00:17.633096+0000","last_became_active":"2023-11-28T13:00:17.245651+0000","last_became_peered":"2023-11-28T13:00:17.245651+0000","last_unstale":"2023-11-28T13:00:17.633096+0000","last_undegraded":"2023-11-28T13:00:17.633096+0000","last_fullsized":"2023-11-28T13:00:17.633096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:30.760573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.235374+0000","last_change":"2023-11-28T13:00:17.235374+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.235374+0000","last_undegraded":"2023-11-28T13:00:17.235374+0000","last_fullsized":"2023-11-28T13:00:17.235374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650468+0000","last_change":"2023-11-28T13:00:17.240716+0000","last_active":"2023-11-28T13:00:17.650468+0000","last_peered":"2023-11-28T13:00:17.650468+0000","last_clean":"2023-11-28T13:00:17.650468+0000","last_became_active":"2023-11-28T13:00:17.239928+0000","last_became_peered":"2023-11-28T13:00:17.239928+0000","last_unstale":"2023-11-28T13:00:17.650468+0000","last_undegraded":"2023-11-28T13:00:17.650468+0000","last_fullsized":"2023-11-28T13:00:17.650468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:32.086174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215680+0000","last_change":"2023-11-28T13:00:17.643923+0000","last_active":"2023-11-28T13:00:19.215680+0000","last_peered":"2023-11-28T13:00:19.215680+0000","last_clean":"2023-11-28T13:00:19.215680+0000","last_became_active":"2023-11-28T13:00:17.643579+0000","last_became_peered":"2023-11-28T13:00:17.643579+0000","last_unstale":"2023-11-28T13:00:19.215680+0000","last_undegraded":"2023-11-28T13:00:19.215680+0000","last_fullsized":"2023-11-28T13:00:19.215680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654531+0000","last_change":"2023-11-28T13:00:17.247259+0000","last_active":"2023-11-28T13:00:17.654531+0000","last_peered":"2023-11-28T13:00:17.654531+0000","last_clean":"2023-11-28T13:00:17.654531+0000","last_became_active":"2023-11-28T13:00:17.246943+0000","last_became_peered":"2023-11-28T13:00:17.246943+0000","last_unstale":"2023-11-28T13:00:17.654531+0000","last_undegraded":"2023-11-28T13:00:17.654531+0000","last_fullsized":"2023-11-28T13:00:17.654531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:02.643388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647850+0000","last_change":"2023-11-28T13:00:17.644322+0000","last_active":"2023-11-28T13:00:17.647850+0000","last_peered":"2023-11-28T13:00:17.647850+0000","last_clean":"2023-11-28T13:00:17.647850+0000","last_became_active":"2023-11-28T13:00:17.644054+0000","last_became_peered":"2023-11-28T13:00:17.644054+0000","last_unstale":"2023-11-28T13:00:17.647850+0000","last_undegraded":"2023-11-28T13:00:17.647850+0000","last_fullsized":"2023-11-28T13:00:17.647850+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:46.098419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656474+0000","last_change":"2023-11-28T13:00:17.247274+0000","last_active":"2023-11-28T13:00:17.656474+0000","last_peered":"2023-11-28T13:00:17.656474+0000","last_clean":"2023-11-28T13:00:17.656474+0000","last_became_active":"2023-11-28T13:00:17.246851+0000","last_became_peered":"2023-11-28T13:00:17.246851+0000","last_unstale":"2023-11-28T13:00:17.656474+0000","last_undegraded":"2023-11-28T13:00:17.656474+0000","last_fullsized":"2023-11-28T13:00:17.656474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:17.621082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652526+0000","last_change":"2023-11-28T13:00:17.640395+0000","last_active":"2023-11-28T13:00:17.652526+0000","last_peered":"2023-11-28T13:00:17.652526+0000","last_clean":"2023-11-28T13:00:17.652526+0000","last_became_active":"2023-11-28T13:00:17.640195+0000","last_became_peered":"2023-11-28T13:00:17.640195+0000","last_unstale":"2023-11-28T13:00:17.652526+0000","last_undegraded":"2023-11-28T13:00:17.652526+0000","last_fullsized":"2023-11-28T13:00:17.652526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:14.878366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.646780+0000","last_change":"2023-11-28T13:00:17.240698+0000","last_active":"2023-11-28T13:00:17.646780+0000","last_peered":"2023-11-28T13:00:17.646780+0000","last_clean":"2023-11-28T13:00:17.646780+0000","last_became_active":"2023-11-28T13:00:17.240387+0000","last_became_peered":"2023-11-28T13:00:17.240387+0000","last_unstale":"2023-11-28T13:00:17.646780+0000","last_undegraded":"2023-11-28T13:00:17.646780+0000","last_fullsized":"2023-11-28T13:00:17.646780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:05.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648509+0000","last_change":"2023-11-28T13:00:17.647318+0000","last_active":"2023-11-28T13:00:17.648509+0000","last_peered":"2023-11-28T13:00:17.648509+0000","last_clean":"2023-11-28T13:00:17.648509+0000","last_became_active":"2023-11-28T13:00:17.647043+0000","last_became_peered":"2023-11-28T13:00:17.647043+0000","last_unstale":"2023-11-28T13:00:17.648509+0000","last_undegraded":"2023-11-28T13:00:17.648509+0000","last_fullsized":"2023-11-28T13:00:17.648509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:38.321116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634419+0000","last_change":"2023-11-28T13:00:17.235843+0000","last_active":"2023-11-28T13:00:17.634419+0000","last_peered":"2023-11-28T13:00:17.634419+0000","last_clean":"2023-11-28T13:00:17.634419+0000","last_became_active":"2023-11-28T13:00:17.235495+0000","last_became_peered":"2023-11-28T13:00:17.235495+0000","last_unstale":"2023-11-28T13:00:17.634419+0000","last_undegraded":"2023-11-28T13:00:17.634419+0000","last_fullsized":"2023-11-28T13:00:17.634419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232454+0000","last_change":"2023-11-28T13:00:17.232454+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232454+0000","last_undegraded":"2023-11-28T13:00:17.232454+0000","last_fullsized":"2023-11-28T13:00:17.232454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215456+0000","last_change":"2023-11-28T13:00:17.240709+0000","last_active":"2023-11-28T13:00:19.215456+0000","last_peered":"2023-11-28T13:00:19.215456+0000","last_clean":"2023-11-28T13:00:19.215456+0000","last_became_active":"2023-11-28T13:00:17.239893+0000","last_became_peered":"2023-11-28T13:00:17.239893+0000","last_unstale":"2023-11-28T13:00:19.215456+0000","last_undegraded":"2023-11-28T13:00:19.215456+0000","last_fullsized":"2023-11-28T13:00:19.215456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658839+0000","last_change":"2023-11-28T13:00:17.653235+0000","last_active":"2023-11-28T13:00:17.658839+0000","last_peered":"2023-11-28T13:00:17.658839+0000","last_clean":"2023-11-28T13:00:17.658839+0000","last_became_active":"2023-11-28T13:00:17.650249+0000","last_became_peered":"2023-11-28T13:00:17.650249+0000","last_unstale":"2023-11-28T13:00:17.658839+0000","last_undegraded":"2023-11-28T13:00:17.658839+0000","last_fullsized":"2023-11-28T13:00:17.658839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631372+0000","last_change":"2023-11-28T13:00:17.245224+0000","last_active":"2023-11-28T13:00:17.631372+0000","last_peered":"2023-11-28T13:00:17.631372+0000","last_clean":"2023-11-28T13:00:17.631372+0000","last_became_active":"2023-11-28T13:00:17.244913+0000","last_became_peered":"2023-11-28T13:00:17.244913+0000","last_unstale":"2023-11-28T13:00:17.631372+0000","last_undegraded":"2023-11-28T13:00:17.631372+0000","last_fullsized":"2023-11-28T13:00:17.631372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653954+0000","last_change":"2023-11-28T13:00:17.643015+0000","last_active":"2023-11-28T13:00:17.653954+0000","last_peered":"2023-11-28T13:00:17.653954+0000","last_clean":"2023-11-28T13:00:17.653954+0000","last_became_active":"2023-11-28T13:00:17.642470+0000","last_became_peered":"2023-11-28T13:00:17.642470+0000","last_unstale":"2023-11-28T13:00:17.653954+0000","last_undegraded":"2023-11-28T13:00:17.653954+0000","last_fullsized":"2023-11-28T13:00:17.653954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:41.679867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.639452+0000","last_change":"2023-11-28T13:00:17.244207+0000","last_active":"2023-11-28T13:00:17.639452+0000","last_peered":"2023-11-28T13:00:17.639452+0000","last_clean":"2023-11-28T13:00:17.639452+0000","last_became_active":"2023-11-28T13:00:17.243861+0000","last_became_peered":"2023-11-28T13:00:17.243861+0000","last_unstale":"2023-11-28T13:00:17.639452+0000","last_undegraded":"2023-11-28T13:00:17.639452+0000","last_fullsized":"2023-11-28T13:00:17.639452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656262+0000","last_change":"2023-11-28T13:00:17.651269+0000","last_active":"2023-11-28T13:00:17.656262+0000","last_peered":"2023-11-28T13:00:17.656262+0000","last_clean":"2023-11-28T13:00:17.656262+0000","last_became_active":"2023-11-28T13:00:17.650883+0000","last_became_peered":"2023-11-28T13:00:17.650883+0000","last_unstale":"2023-11-28T13:00:17.656262+0000","last_undegraded":"2023-11-28T13:00:17.656262+0000","last_fullsized":"2023-11-28T13:00:17.656262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215191+0000","last_change":"2023-11-28T13:00:17.238617+0000","last_active":"2023-11-28T13:00:19.215191+0000","last_peered":"2023-11-28T13:00:19.215191+0000","last_clean":"2023-11-28T13:00:19.215191+0000","last_became_active":"2023-11-28T13:00:17.238344+0000","last_became_peered":"2023-11-28T13:00:17.238344+0000","last_unstale":"2023-11-28T13:00:19.215191+0000","last_undegraded":"2023-11-28T13:00:19.215191+0000","last_fullsized":"2023-11-28T13:00:19.215191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:46.225565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636017+0000","last_change":"2023-11-28T13:00:17.649564+0000","last_active":"2023-11-28T13:00:18.636017+0000","last_peered":"2023-11-28T13:00:18.636017+0000","last_clean":"2023-11-28T13:00:18.636017+0000","last_became_active":"2023-11-28T13:00:17.649312+0000","last_became_peered":"2023-11-28T13:00:17.649312+0000","last_unstale":"2023-11-28T13:00:18.636017+0000","last_undegraded":"2023-11-28T13:00:18.636017+0000","last_fullsized":"2023-11-28T13:00:18.636017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634764+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:00:17.634764+0000","last_peered":"2023-11-28T13:00:17.634764+0000","last_clean":"2023-11-28T13:00:17.634764+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:00:17.634764+0000","last_undegraded":"2023-11-28T13:00:17.634764+0000","last_fullsized":"2023-11-28T13:00:17.634764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.636391+0000","last_change":"2023-11-28T13:00:17.240503+0000","last_active":"2023-11-28T13:00:17.636391+0000","last_peered":"2023-11-28T13:00:17.636391+0000","last_clean":"2023-11-28T13:00:17.636391+0000","last_became_active":"2023-11-28T13:00:17.240256+0000","last_became_peered":"2023-11-28T13:00:17.240256+0000","last_unstale":"2023-11-28T13:00:17.636391+0000","last_undegraded":"2023-11-28T13:00:17.636391+0000","last_fullsized":"2023-11-28T13:00:17.636391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635946+0000","last_change":"2023-11-28T13:00:17.645964+0000","last_active":"2023-11-28T13:00:18.635946+0000","last_peered":"2023-11-28T13:00:18.635946+0000","last_clean":"2023-11-28T13:00:18.635946+0000","last_became_active":"2023-11-28T13:00:17.645384+0000","last_became_peered":"2023-11-28T13:00:17.645384+0000","last_unstale":"2023-11-28T13:00:18.635946+0000","last_undegraded":"2023-11-28T13:00:18.635946+0000","last_fullsized":"2023-11-28T13:00:18.635946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234232+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:00:17.234232+0000","last_peered":"2023-11-28T13:00:17.234232+0000","last_clean":"2023-11-28T13:00:17.234232+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:00:17.234232+0000","last_undegraded":"2023-11-28T13:00:17.234232+0000","last_fullsized":"2023-11-28T13:00:17.234232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.640096+0000","last_change":"2023-11-28T13:00:17.245773+0000","last_active":"2023-11-28T13:00:17.640096+0000","last_peered":"2023-11-28T13:00:17.640096+0000","last_clean":"2023-11-28T13:00:17.640096+0000","last_became_active":"2023-11-28T13:00:17.245409+0000","last_became_peered":"2023-11-28T13:00:17.245409+0000","last_unstale":"2023-11-28T13:00:17.640096+0000","last_undegraded":"2023-11-28T13:00:17.640096+0000","last_fullsized":"2023-11-28T13:00:17.640096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:00.519142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636216+0000","last_change":"2023-11-28T13:00:17.641395+0000","last_active":"2023-11-28T13:00:18.636216+0000","last_peered":"2023-11-28T13:00:18.636216+0000","last_clean":"2023-11-28T13:00:18.636216+0000","last_became_active":"2023-11-28T13:00:17.641149+0000","last_became_peered":"2023-11-28T13:00:17.641149+0000","last_unstale":"2023-11-28T13:00:18.636216+0000","last_undegraded":"2023-11-28T13:00:18.636216+0000","last_fullsized":"2023-11-28T13:00:18.636216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.628022+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:00:17.628022+0000","last_peered":"2023-11-28T13:00:17.628022+0000","last_clean":"2023-11-28T13:00:17.628022+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:00:17.628022+0000","last_undegraded":"2023-11-28T13:00:17.628022+0000","last_fullsized":"2023-11-28T13:00:17.628022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635610+0000","last_change":"2023-11-28T13:00:17.237802+0000","last_active":"2023-11-28T13:00:18.635610+0000","last_peered":"2023-11-28T13:00:18.635610+0000","last_clean":"2023-11-28T13:00:18.635610+0000","last_became_active":"2023-11-28T13:00:17.237487+0000","last_became_peered":"2023-11-28T13:00:17.237487+0000","last_unstale":"2023-11-28T13:00:18.635610+0000","last_undegraded":"2023-11-28T13:00:18.635610+0000","last_fullsized":"2023-11-28T13:00:18.635610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635877+0000","last_change":"2023-11-28T13:00:17.644511+0000","last_active":"2023-11-28T13:00:18.635877+0000","last_peered":"2023-11-28T13:00:18.635877+0000","last_clean":"2023-11-28T13:00:18.635877+0000","last_became_active":"2023-11-28T13:00:17.644095+0000","last_became_peered":"2023-11-28T13:00:17.644095+0000","last_unstale":"2023-11-28T13:00:18.635877+0000","last_undegraded":"2023-11-28T13:00:18.635877+0000","last_fullsized":"2023-11-28T13:00:18.635877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234538+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:00:17.234538+0000","last_peered":"2023-11-28T13:00:17.234538+0000","last_clean":"2023-11-28T13:00:17.234538+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:00:17.234538+0000","last_undegraded":"2023-11-28T13:00:17.234538+0000","last_fullsized":"2023-11-28T13:00:17.234538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235309+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:00:17.235309+0000","last_peered":"2023-11-28T13:00:17.235309+0000","last_clean":"2023-11-28T13:00:17.235309+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:00:17.235309+0000","last_undegraded":"2023-11-28T13:00:17.235309+0000","last_fullsized":"2023-11-28T13:00:17.235309+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215311+0000","last_change":"2023-11-28T13:00:17.239373+0000","last_active":"2023-11-28T13:00:19.215311+0000","last_peered":"2023-11-28T13:00:19.215311+0000","last_clean":"2023-11-28T13:00:19.215311+0000","last_became_active":"2023-11-28T13:00:17.238864+0000","last_became_peered":"2023-11-28T13:00:17.238864+0000","last_unstale":"2023-11-28T13:00:19.215311+0000","last_undegraded":"2023-11-28T13:00:19.215311+0000","last_fullsized":"2023-11-28T13:00:19.215311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:17.763933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653117+0000","last_change":"2023-11-28T13:00:17.649771+0000","last_active":"2023-11-28T13:00:17.653117+0000","last_peered":"2023-11-28T13:00:17.653117+0000","last_clean":"2023-11-28T13:00:17.653117+0000","last_became_active":"2023-11-28T13:00:17.649475+0000","last_became_peered":"2023-11-28T13:00:17.649475+0000","last_unstale":"2023-11-28T13:00:17.653117+0000","last_undegraded":"2023-11-28T13:00:17.653117+0000","last_fullsized":"2023-11-28T13:00:17.653117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235498+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:00:17.235498+0000","last_peered":"2023-11-28T13:00:17.235498+0000","last_clean":"2023-11-28T13:00:17.235498+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:00:17.235498+0000","last_undegraded":"2023-11-28T13:00:17.235498+0000","last_fullsized":"2023-11-28T13:00:17.235498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253107+0000","last_change":"2023-11-28T13:00:17.251262+0000","last_active":"2023-11-28T13:00:17.253107+0000","last_peered":"2023-11-28T13:00:17.253107+0000","last_clean":"2023-11-28T13:00:17.253107+0000","last_became_active":"2023-11-28T13:00:17.250948+0000","last_became_peered":"2023-11-28T13:00:17.250948+0000","last_unstale":"2023-11-28T13:00:17.253107+0000","last_undegraded":"2023-11-28T13:00:17.253107+0000","last_fullsized":"2023-11-28T13:00:17.253107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:37.311866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233919+0000","last_change":"2023-11-28T13:00:17.233919+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233919+0000","last_undegraded":"2023-11-28T13:00:17.233919+0000","last_fullsized":"2023-11-28T13:00:17.233919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.236149+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:00:17.236149+0000","last_peered":"2023-11-28T13:00:17.236149+0000","last_clean":"2023-11-28T13:00:17.236149+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:00:17.236149+0000","last_undegraded":"2023-11-28T13:00:17.236149+0000","last_fullsized":"2023-11-28T13:00:17.236149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636224+0000","last_change":"2023-11-28T13:00:17.237999+0000","last_active":"2023-11-28T13:00:18.636224+0000","last_peered":"2023-11-28T13:00:18.636224+0000","last_clean":"2023-11-28T13:00:18.636224+0000","last_became_active":"2023-11-28T13:00:17.237625+0000","last_became_peered":"2023-11-28T13:00:17.237625+0000","last_unstale":"2023-11-28T13:00:18.636224+0000","last_undegraded":"2023-11-28T13:00:18.636224+0000","last_fullsized":"2023-11-28T13:00:18.636224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233381+0000","last_change":"2023-11-28T13:00:17.233381+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233381+0000","last_undegraded":"2023-11-28T13:00:17.233381+0000","last_fullsized":"2023-11-28T13:00:17.233381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215660+0000","last_change":"2023-11-28T13:00:17.239284+0000","last_active":"2023-11-28T13:00:19.215660+0000","last_peered":"2023-11-28T13:00:19.215660+0000","last_clean":"2023-11-28T13:00:19.215660+0000","last_became_active":"2023-11-28T13:00:17.238997+0000","last_became_peered":"2023-11-28T13:00:17.238997+0000","last_unstale":"2023-11-28T13:00:19.215660+0000","last_undegraded":"2023-11-28T13:00:19.215660+0000","last_fullsized":"2023-11-28T13:00:19.215660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:55.423382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633198+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:00:17.633198+0000","last_peered":"2023-11-28T13:00:17.633198+0000","last_clean":"2023-11-28T13:00:17.633198+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:00:17.633198+0000","last_undegraded":"2023-11-28T13:00:17.633198+0000","last_fullsized":"2023-11-28T13:00:17.633198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215108+0000","last_change":"2023-11-28T13:00:17.646149+0000","last_active":"2023-11-28T13:00:19.215108+0000","last_peered":"2023-11-28T13:00:19.215108+0000","last_clean":"2023-11-28T13:00:19.215108+0000","last_became_active":"2023-11-28T13:00:17.645893+0000","last_became_peered":"2023-11-28T13:00:17.645893+0000","last_unstale":"2023-11-28T13:00:19.215108+0000","last_undegraded":"2023-11-28T13:00:19.215108+0000","last_fullsized":"2023-11-28T13:00:19.215108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:24.466207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.226618+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:00:17.226618+0000","last_peered":"2023-11-28T13:00:17.226618+0000","last_clean":"2023-11-28T13:00:17.226618+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:00:17.226618+0000","last_undegraded":"2023-11-28T13:00:17.226618+0000","last_fullsized":"2023-11-28T13:00:17.226618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245166+0000","last_change":"2023-11-28T13:00:17.240379+0000","last_active":"2023-11-28T13:00:17.245166+0000","last_peered":"2023-11-28T13:00:17.245166+0000","last_clean":"2023-11-28T13:00:17.245166+0000","last_became_active":"2023-11-28T13:00:17.239505+0000","last_became_peered":"2023-11-28T13:00:17.239505+0000","last_unstale":"2023-11-28T13:00:17.245166+0000","last_undegraded":"2023-11-28T13:00:17.245166+0000","last_fullsized":"2023-11-28T13:00:17.245166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655975+0000","last_change":"2023-11-28T13:00:17.646003+0000","last_active":"2023-11-28T13:00:17.655975+0000","last_peered":"2023-11-28T13:00:17.655975+0000","last_clean":"2023-11-28T13:00:17.655975+0000","last_became_active":"2023-11-28T13:00:17.645727+0000","last_became_peered":"2023-11-28T13:00:17.645727+0000","last_unstale":"2023-11-28T13:00:17.655975+0000","last_undegraded":"2023-11-28T13:00:17.655975+0000","last_fullsized":"2023-11-28T13:00:17.655975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:30.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253113+0000","last_change":"2023-11-28T13:00:17.251398+0000","last_active":"2023-11-28T13:00:17.253113+0000","last_peered":"2023-11-28T13:00:17.253113+0000","last_clean":"2023-11-28T13:00:17.253113+0000","last_became_active":"2023-11-28T13:00:17.251054+0000","last_became_peered":"2023-11-28T13:00:17.251054+0000","last_unstale":"2023-11-28T13:00:17.253113+0000","last_undegraded":"2023-11-28T13:00:17.253113+0000","last_fullsized":"2023-11-28T13:00:17.253113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217848+0000","last_change":"2023-11-28T13:00:17.641355+0000","last_active":"2023-11-28T13:00:19.217848+0000","last_peered":"2023-11-28T13:00:19.217848+0000","last_clean":"2023-11-28T13:00:19.217848+0000","last_became_active":"2023-11-28T13:00:17.641093+0000","last_became_peered":"2023-11-28T13:00:17.641093+0000","last_unstale":"2023-11-28T13:00:19.217848+0000","last_undegraded":"2023-11-28T13:00:19.217848+0000","last_fullsized":"2023-11-28T13:00:19.217848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:41.816686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655479+0000","last_change":"2023-11-28T13:00:17.246896+0000","last_active":"2023-11-28T13:00:17.655479+0000","last_peered":"2023-11-28T13:00:17.655479+0000","last_clean":"2023-11-28T13:00:17.655479+0000","last_became_active":"2023-11-28T13:00:17.246604+0000","last_became_peered":"2023-11-28T13:00:17.246604+0000","last_unstale":"2023-11-28T13:00:17.655479+0000","last_undegraded":"2023-11-28T13:00:17.655479+0000","last_fullsized":"2023-11-28T13:00:17.655479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656231+0000","last_change":"2023-11-28T13:00:17.648973+0000","last_active":"2023-11-28T13:00:17.656231+0000","last_peered":"2023-11-28T13:00:17.656231+0000","last_clean":"2023-11-28T13:00:17.656231+0000","last_became_active":"2023-11-28T13:00:17.648323+0000","last_became_peered":"2023-11-28T13:00:17.648323+0000","last_unstale":"2023-11-28T13:00:17.656231+0000","last_undegraded":"2023-11-28T13:00:17.656231+0000","last_fullsized":"2023-11-28T13:00:17.656231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654699+0000","last_change":"2023-11-28T13:00:17.247013+0000","last_active":"2023-11-28T13:00:17.654699+0000","last_peered":"2023-11-28T13:00:17.654699+0000","last_clean":"2023-11-28T13:00:17.654699+0000","last_became_active":"2023-11-28T13:00:17.246645+0000","last_became_peered":"2023-11-28T13:00:17.246645+0000","last_unstale":"2023-11-28T13:00:17.654699+0000","last_undegraded":"2023-11-28T13:00:17.654699+0000","last_fullsized":"2023-11-28T13:00:17.654699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656116+0000","last_change":"2023-11-28T13:00:17.649078+0000","last_active":"2023-11-28T13:00:17.656116+0000","last_peered":"2023-11-28T13:00:17.656116+0000","last_clean":"2023-11-28T13:00:17.656116+0000","last_became_active":"2023-11-28T13:00:17.648432+0000","last_became_peered":"2023-11-28T13:00:17.648432+0000","last_unstale":"2023-11-28T13:00:17.656116+0000","last_undegraded":"2023-11-28T13:00:17.656116+0000","last_fullsized":"2023-11-28T13:00:17.656116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215495+0000","last_change":"2023-11-28T13:00:17.234074+0000","last_active":"2023-11-28T13:00:19.215495+0000","last_peered":"2023-11-28T13:00:19.215495+0000","last_clean":"2023-11-28T13:00:19.215495+0000","last_became_active":"2023-11-28T13:00:17.233732+0000","last_became_peered":"2023-11-28T13:00:17.233732+0000","last_unstale":"2023-11-28T13:00:19.215495+0000","last_undegraded":"2023-11-28T13:00:19.215495+0000","last_fullsized":"2023-11-28T13:00:19.215495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:36.950864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215231+0000","last_change":"2023-11-28T13:00:17.641105+0000","last_active":"2023-11-28T13:00:19.215231+0000","last_peered":"2023-11-28T13:00:19.215231+0000","last_clean":"2023-11-28T13:00:19.215231+0000","last_became_active":"2023-11-28T13:00:17.640854+0000","last_became_peered":"2023-11-28T13:00:17.640854+0000","last_unstale":"2023-11-28T13:00:19.215231+0000","last_undegraded":"2023-11-28T13:00:19.215231+0000","last_fullsized":"2023-11-28T13:00:19.215231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247814+0000","last_change":"2023-11-28T13:00:17.242508+0000","last_active":"2023-11-28T13:00:17.247814+0000","last_peered":"2023-11-28T13:00:17.247814+0000","last_clean":"2023-11-28T13:00:17.247814+0000","last_became_active":"2023-11-28T13:00:17.242230+0000","last_became_peered":"2023-11-28T13:00:17.242230+0000","last_unstale":"2023-11-28T13:00:17.247814+0000","last_undegraded":"2023-11-28T13:00:17.247814+0000","last_fullsized":"2023-11-28T13:00:17.247814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656224+0000","last_change":"2023-11-28T13:00:17.650868+0000","last_active":"2023-11-28T13:00:17.656224+0000","last_peered":"2023-11-28T13:00:17.656224+0000","last_clean":"2023-11-28T13:00:17.656224+0000","last_became_active":"2023-11-28T13:00:17.649842+0000","last_became_peered":"2023-11-28T13:00:17.649842+0000","last_unstale":"2023-11-28T13:00:17.656224+0000","last_undegraded":"2023-11-28T13:00:17.656224+0000","last_fullsized":"2023-11-28T13:00:17.656224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645743+0000","last_change":"2023-11-28T13:00:17.641330+0000","last_active":"2023-11-28T13:00:17.645743+0000","last_peered":"2023-11-28T13:00:17.645743+0000","last_clean":"2023-11-28T13:00:17.645743+0000","last_became_active":"2023-11-28T13:00:17.641029+0000","last_became_peered":"2023-11-28T13:00:17.641029+0000","last_unstale":"2023-11-28T13:00:17.645743+0000","last_undegraded":"2023-11-28T13:00:17.645743+0000","last_fullsized":"2023-11-28T13:00:17.645743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:28.607875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215202+0000","last_change":"2023-11-28T13:00:17.643816+0000","last_active":"2023-11-28T13:00:19.215202+0000","last_peered":"2023-11-28T13:00:19.215202+0000","last_clean":"2023-11-28T13:00:19.215202+0000","last_became_active":"2023-11-28T13:00:17.643260+0000","last_became_peered":"2023-11-28T13:00:17.643260+0000","last_unstale":"2023-11-28T13:00:19.215202+0000","last_undegraded":"2023-11-28T13:00:19.215202+0000","last_fullsized":"2023-11-28T13:00:19.215202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656140+0000","last_change":"2023-11-28T13:00:17.649431+0000","last_active":"2023-11-28T13:00:17.656140+0000","last_peered":"2023-11-28T13:00:17.656140+0000","last_clean":"2023-11-28T13:00:17.656140+0000","last_became_active":"2023-11-28T13:00:17.648763+0000","last_became_peered":"2023-11-28T13:00:17.648763+0000","last_unstale":"2023-11-28T13:00:17.656140+0000","last_undegraded":"2023-11-28T13:00:17.656140+0000","last_fullsized":"2023-11-28T13:00:17.656140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:10.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215472+0000","last_change":"2023-11-28T13:00:17.649148+0000","last_active":"2023-11-28T13:00:19.215472+0000","last_peered":"2023-11-28T13:00:19.215472+0000","last_clean":"2023-11-28T13:00:19.215472+0000","last_became_active":"2023-11-28T13:00:17.648925+0000","last_became_peered":"2023-11-28T13:00:17.648925+0000","last_unstale":"2023-11-28T13:00:19.215472+0000","last_undegraded":"2023-11-28T13:00:19.215472+0000","last_fullsized":"2023-11-28T13:00:19.215472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636138+0000","last_change":"2023-11-28T13:00:17.648409+0000","last_active":"2023-11-28T13:00:18.636138+0000","last_peered":"2023-11-28T13:00:18.636138+0000","last_clean":"2023-11-28T13:00:18.636138+0000","last_became_active":"2023-11-28T13:00:17.648164+0000","last_became_peered":"2023-11-28T13:00:17.648164+0000","last_unstale":"2023-11-28T13:00:18.636138+0000","last_undegraded":"2023-11-28T13:00:18.636138+0000","last_fullsized":"2023-11-28T13:00:18.636138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":96947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7940,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363900,"statfs":{"total":96636764160,"available":96628633600,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":91808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8928,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362912,"statfs":{"total":96636764160,"available":96627621888,"internally_reserved":0,"allocated":811008,"data_stored":666055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9112,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362728,"statfs":{"total":96636764160,"available":96627433472,"internally_reserved":0,"allocated":868352,"data_stored":682336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8000,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363840,"statfs":{"total":96636764160,"available":96628572160,"internally_reserved":0,"allocated":253952,"data_stored":90840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7992,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363848,"statfs":{"total":96636764160,"available":96628580352,"internally_reserved":0,"allocated":245760,"data_stored":90204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8068,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363772,"statfs":{"total":96636764160,"available":96628502528,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:21.710 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:21.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.863+0000 7fafad492700 1 Processor -- start 2023-11-28T13:00:21.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafad492700 1 -- start start 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafad492700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fafa8160740 0x7fafa80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafad492700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa8160b10 0x7fafa80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafad492700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa806bbf0 con 0x7fafa806b060 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafad492700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa80ca720 con 0x7fafa8160740 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafad492700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80cc030 con 0x7fafa8160b10 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafa6ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fafa8160740 0x7fafa80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafa77fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fafa806b060 0x7fafa806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56500/0 (socket says 172.21.15.153:56500) 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafa6ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fafa8160740 0x7fafa80cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38334/0 (socket says 172.21.15.153:38334) 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.864+0000 7fafa77fe700 1 -- 172.21.15.153:0/894851640 learned_addr learned my addr 172.21.15.153:0/894851640 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa67fc700 1 --2- 172.21.15.153:0/894851640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa8160b10 0x7fafa80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa6ffd700 1 -- 172.21.15.153:0/894851640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa8160b10 msgr2=0x7fafa80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa6ffd700 1 --2- 172.21.15.153:0/894851640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa8160b10 0x7fafa80cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa6ffd700 1 -- 172.21.15.153:0/894851640 >> v1:172.21.15.153:6789/0 conn(0x7fafa806b060 legacy=0x7fafa806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa6ffd700 1 -- 172.21.15.153:0/894851640 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa80c9e10 con 0x7fafa8160740 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa5ffb700 1 -- 172.21.15.153:0/894851640 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913662229 0 0) 0x7fafa806bbf0 con 0x7fafa806b060 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa6ffd700 1 --2- 172.21.15.153:0/894851640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160740 0x7fafa80cbaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf98009840 tx=0x7faf98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e450d7495db30b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa5ffb700 1 -- 172.21.15.153:0/894851640 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98011830 con 0x7fafa8160740 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa5ffb700 1 -- 172.21.15.153:0/894851640 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf98011990 con 0x7fafa8160740 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.865+0000 7fafa5ffb700 1 -- 172.21.15.153:0/894851640 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98011d20 con 0x7fafa8160740 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 -- 172.21.15.153:0/894851640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160740 msgr2=0x7fafa80cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 --2- 172.21.15.153:0/894851640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160740 0x7fafa80cbaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf98009840 tx=0x7faf98009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:21.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 -- 172.21.15.153:0/894851640 shutdown_connections 2023-11-28T13:00:21.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 --2- 172.21.15.153:0/894851640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160740 0x7fafa80cbaf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 --2- 172.21.15.153:0/894851640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa8160b10 0x7fafa80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 -- 172.21.15.153:0/894851640 >> 172.21.15.153:0/894851640 conn(0x7fafa80b12b0 msgr2=0x7fafa80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 -- 172.21.15.153:0/894851640 shutdown_connections 2023-11-28T13:00:21.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.866+0000 7fafad492700 1 -- 172.21.15.153:0/894851640 wait complete. 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.867+0000 7fafad492700 1 Processor -- start 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.867+0000 7fafad492700 1 -- start start 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafad492700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa806b060 0x7fafa815e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafad492700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafa8160740 0x7fafa81f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafad492700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160b10 0x7fafa815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafad492700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa81f5d30 con 0x7fafa8160b10 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafad492700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80cdda0 con 0x7fafa8160740 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafad492700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80cdac0 con 0x7fafa806b060 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafa77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160b10 0x7fafa815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafa77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160b10 0x7fafa815eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38342/0 (socket says 172.21.15.153:38342) 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.868+0000 7fafa77fe700 1 -- 172.21.15.153:0/573451709 learned_addr learned my addr 172.21.15.153:0/573451709 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa6ffd700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa806b060 0x7fafa815e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa67fc700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafa8160740 0x7fafa81f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa77fe700 1 -- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa806b060 msgr2=0x7fafa815e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa77fe700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa806b060 0x7fafa815e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa77fe700 1 -- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafa8160740 msgr2=0x7fafa81f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa77fe700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafa8160740 0x7fafa81f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa77fe700 1 -- 172.21.15.153:0/573451709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980094a0 con 0x7fafa8160b10 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa6ffd700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa806b060 0x7fafa815e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafa77fe700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160b10 0x7fafa815eed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafa000cb60 tx=0x7fafa000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7faf9f7fe700 1 -- 172.21.15.153:0/573451709 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa000ab50 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7faf9f7fe700 1 -- 172.21.15.153:0/573451709 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa000acb0 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa80ca0d0 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7faf9f7fe700 1 -- 172.21.15.153:0/573451709 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0007310 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.869+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa806efc0 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.870+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa80ca3b0 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.870+0000 7faf9f7fe700 1 -- 172.21.15.153:0/573451709 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fafa001a030 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.871+0000 7faf9f7fe700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faf880682c0 0x7faf8806a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.871+0000 7fafa6ffd700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faf880682c0 0x7faf8806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.871+0000 7faf9f7fe700 1 -- 172.21.15.153:0/573451709 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fafa0085940 con 0x7fafa8160b10 2023-11-28T13:00:21.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.871+0000 7fafa6ffd700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faf880682c0 0x7faf8806a770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf98000c00 tx=0x7faf98008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.877 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:21.875+0000 7faf9f7fe700 1 -- 172.21.15.153:0/573451709 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fafa00521a0 con 0x7fafa8160b10 2023-11-28T13:00:22.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.050+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fafa8066110 con 0x7faf880682c0 2023-11-28T13:00:22.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.058+0000 7faf9f7fe700 1 -- 172.21.15.153:0/573451709 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286273 (secure 0 0 0) 0x7fafa8066110 con 0x7faf880682c0 2023-11-28T13:00:22.065 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faf880682c0 msgr2=0x7faf8806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faf880682c0 0x7faf8806a770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf98000c00 tx=0x7faf98008030 comp rx=0 tx=0).stop 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160b10 msgr2=0x7fafa815eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160b10 0x7fafa815eed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafa000cb60 tx=0x7fafa000cf20 comp rx=0 tx=0).stop 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 shutdown_connections 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faf880682c0 0x7faf8806a770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafa8160b10 0x7fafa815eed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafa806b060 0x7fafa815e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 --2- 172.21.15.153:0/573451709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafa8160740 0x7fafa81f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 >> 172.21.15.153:0/573451709 conn(0x7fafa80b12b0 msgr2=0x7fafa80ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 shutdown_connections 2023-11-28T13:00:22.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.061+0000 7fafad492700 1 -- 172.21.15.153:0/573451709 wait complete. 2023-11-28T13:00:22.069 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:00:22.077 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-28T13:00:20.135856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592132,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66188,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908532,"statfs":{"total":773094113280,"available":773026336768,"internally_reserved":0,"allocated":3223552,"data_stored":1911904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000784"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217825+0000","last_change":"2023-11-28T13:00:17.635428+0000","last_active":"2023-11-28T13:00:19.217825+0000","last_peered":"2023-11-28T13:00:19.217825+0000","last_clean":"2023-11-28T13:00:19.217825+0000","last_became_active":"2023-11-28T13:00:17.635148+0000","last_became_peered":"2023-11-28T13:00:17.635148+0000","last_unstale":"2023-11-28T13:00:19.217825+0000","last_undegraded":"2023-11-28T13:00:19.217825+0000","last_fullsized":"2023-11-28T13:00:19.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648693+0000","last_change":"2023-11-28T13:00:17.645017+0000","last_active":"2023-11-28T13:00:17.648693+0000","last_peered":"2023-11-28T13:00:17.648693+0000","last_clean":"2023-11-28T13:00:17.648693+0000","last_became_active":"2023-11-28T13:00:17.644524+0000","last_became_peered":"2023-11-28T13:00:17.644524+0000","last_unstale":"2023-11-28T13:00:17.648693+0000","last_undegraded":"2023-11-28T13:00:17.648693+0000","last_fullsized":"2023-11-28T13:00:17.648693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656057+0000","last_change":"2023-11-28T13:00:17.648662+0000","last_active":"2023-11-28T13:00:17.656057+0000","last_peered":"2023-11-28T13:00:17.656057+0000","last_clean":"2023-11-28T13:00:17.656057+0000","last_became_active":"2023-11-28T13:00:17.648271+0000","last_became_peered":"2023-11-28T13:00:17.648271+0000","last_unstale":"2023-11-28T13:00:17.656057+0000","last_undegraded":"2023-11-28T13:00:17.656057+0000","last_fullsized":"2023-11-28T13:00:17.656057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:37.103258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647723+0000","last_change":"2023-11-28T13:00:17.643733+0000","last_active":"2023-11-28T13:00:17.647723+0000","last_peered":"2023-11-28T13:00:17.647723+0000","last_clean":"2023-11-28T13:00:17.647723+0000","last_became_active":"2023-11-28T13:00:17.643428+0000","last_became_peered":"2023-11-28T13:00:17.643428+0000","last_unstale":"2023-11-28T13:00:17.647723+0000","last_undegraded":"2023-11-28T13:00:17.647723+0000","last_fullsized":"2023-11-28T13:00:17.647723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233853+0000","last_change":"2023-11-28T13:00:17.233853+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233853+0000","last_undegraded":"2023-11-28T13:00:17.233853+0000","last_fullsized":"2023-11-28T13:00:17.233853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658806+0000","last_change":"2023-11-28T13:00:17.652679+0000","last_active":"2023-11-28T13:00:17.658806+0000","last_peered":"2023-11-28T13:00:17.658806+0000","last_clean":"2023-11-28T13:00:17.658806+0000","last_became_active":"2023-11-28T13:00:17.648169+0000","last_became_peered":"2023-11-28T13:00:17.648169+0000","last_unstale":"2023-11-28T13:00:17.658806+0000","last_undegraded":"2023-11-28T13:00:17.658806+0000","last_fullsized":"2023-11-28T13:00:17.658806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:58.762522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649103+0000","last_change":"2023-11-28T13:00:17.647769+0000","last_active":"2023-11-28T13:00:17.649103+0000","last_peered":"2023-11-28T13:00:17.649103+0000","last_clean":"2023-11-28T13:00:17.649103+0000","last_became_active":"2023-11-28T13:00:17.647476+0000","last_became_peered":"2023-11-28T13:00:17.647476+0000","last_unstale":"2023-11-28T13:00:17.649103+0000","last_undegraded":"2023-11-28T13:00:17.649103+0000","last_fullsized":"2023-11-28T13:00:17.649103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653892+0000","last_change":"2023-11-28T13:00:17.653233+0000","last_active":"2023-11-28T13:00:17.653892+0000","last_peered":"2023-11-28T13:00:17.653892+0000","last_clean":"2023-11-28T13:00:17.653892+0000","last_became_active":"2023-11-28T13:00:17.652957+0000","last_became_peered":"2023-11-28T13:00:17.652957+0000","last_unstale":"2023-11-28T13:00:17.653892+0000","last_undegraded":"2023-11-28T13:00:17.653892+0000","last_fullsized":"2023-11-28T13:00:17.653892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231803+0000","last_change":"2023-11-28T13:00:17.231803+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231803+0000","last_undegraded":"2023-11-28T13:00:17.231803+0000","last_fullsized":"2023-11-28T13:00:17.231803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645112+0000","last_change":"2023-11-28T13:00:17.642144+0000","last_active":"2023-11-28T13:00:17.645112+0000","last_peered":"2023-11-28T13:00:17.645112+0000","last_clean":"2023-11-28T13:00:17.645112+0000","last_became_active":"2023-11-28T13:00:17.641814+0000","last_became_peered":"2023-11-28T13:00:17.641814+0000","last_unstale":"2023-11-28T13:00:17.645112+0000","last_undegraded":"2023-11-28T13:00:17.645112+0000","last_fullsized":"2023-11-28T13:00:17.645112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653974+0000","last_change":"2023-11-28T13:00:17.643190+0000","last_active":"2023-11-28T13:00:17.653974+0000","last_peered":"2023-11-28T13:00:17.653974+0000","last_clean":"2023-11-28T13:00:17.653974+0000","last_became_active":"2023-11-28T13:00:17.642741+0000","last_became_peered":"2023-11-28T13:00:17.642741+0000","last_unstale":"2023-11-28T13:00:17.653974+0000","last_undegraded":"2023-11-28T13:00:17.653974+0000","last_fullsized":"2023-11-28T13:00:17.653974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:33.241895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217739+0000","last_change":"2023-11-28T13:00:17.654503+0000","last_active":"2023-11-28T13:00:19.217739+0000","last_peered":"2023-11-28T13:00:19.217739+0000","last_clean":"2023-11-28T13:00:19.217739+0000","last_became_active":"2023-11-28T13:00:17.654204+0000","last_became_peered":"2023-11-28T13:00:17.654204+0000","last_unstale":"2023-11-28T13:00:19.217739+0000","last_undegraded":"2023-11-28T13:00:19.217739+0000","last_fullsized":"2023-11-28T13:00:19.217739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:27.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217678+0000","last_change":"2023-11-28T13:00:17.634718+0000","last_active":"2023-11-28T13:00:19.217678+0000","last_peered":"2023-11-28T13:00:19.217678+0000","last_clean":"2023-11-28T13:00:19.217678+0000","last_became_active":"2023-11-28T13:00:17.634420+0000","last_became_peered":"2023-11-28T13:00:17.634420+0000","last_unstale":"2023-11-28T13:00:19.217678+0000","last_undegraded":"2023-11-28T13:00:19.217678+0000","last_fullsized":"2023-11-28T13:00:19.217678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.567102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645127+0000","last_change":"2023-11-28T13:00:17.641361+0000","last_active":"2023-11-28T13:00:17.645127+0000","last_peered":"2023-11-28T13:00:17.645127+0000","last_clean":"2023-11-28T13:00:17.645127+0000","last_became_active":"2023-11-28T13:00:17.641078+0000","last_became_peered":"2023-11-28T13:00:17.641078+0000","last_unstale":"2023-11-28T13:00:17.645127+0000","last_undegraded":"2023-11-28T13:00:17.645127+0000","last_fullsized":"2023-11-28T13:00:17.645127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:38.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649123+0000","last_change":"2023-11-28T13:00:17.647800+0000","last_active":"2023-11-28T13:00:17.649123+0000","last_peered":"2023-11-28T13:00:17.649123+0000","last_clean":"2023-11-28T13:00:17.649123+0000","last_became_active":"2023-11-28T13:00:17.647519+0000","last_became_peered":"2023-11-28T13:00:17.647519+0000","last_unstale":"2023-11-28T13:00:17.649123+0000","last_undegraded":"2023-11-28T13:00:17.649123+0000","last_fullsized":"2023-11-28T13:00:17.649123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656251+0000","last_change":"2023-11-28T13:00:17.651090+0000","last_active":"2023-11-28T13:00:17.656251+0000","last_peered":"2023-11-28T13:00:17.656251+0000","last_clean":"2023-11-28T13:00:17.656251+0000","last_became_active":"2023-11-28T13:00:17.649174+0000","last_became_peered":"2023-11-28T13:00:17.649174+0000","last_unstale":"2023-11-28T13:00:17.656251+0000","last_undegraded":"2023-11-28T13:00:17.656251+0000","last_fullsized":"2023-11-28T13:00:17.656251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:32.070272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231312+0000","last_change":"2023-11-28T13:00:17.231312+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231312+0000","last_undegraded":"2023-11-28T13:00:17.231312+0000","last_fullsized":"2023-11-28T13:00:17.231312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656166+0000","last_change":"2023-11-28T13:00:17.648771+0000","last_active":"2023-11-28T13:00:17.656166+0000","last_peered":"2023-11-28T13:00:17.656166+0000","last_clean":"2023-11-28T13:00:17.656166+0000","last_became_active":"2023-11-28T13:00:17.646367+0000","last_became_peered":"2023-11-28T13:00:17.646367+0000","last_unstale":"2023-11-28T13:00:17.656166+0000","last_undegraded":"2023-11-28T13:00:17.656166+0000","last_fullsized":"2023-11-28T13:00:17.656166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:33.549492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217793+0000","last_change":"2023-11-28T13:00:17.654191+0000","last_active":"2023-11-28T13:00:19.217793+0000","last_peered":"2023-11-28T13:00:19.217793+0000","last_clean":"2023-11-28T13:00:19.217793+0000","last_became_active":"2023-11-28T13:00:17.653922+0000","last_became_peered":"2023-11-28T13:00:17.653922+0000","last_unstale":"2023-11-28T13:00:19.217793+0000","last_undegraded":"2023-11-28T13:00:19.217793+0000","last_fullsized":"2023-11-28T13:00:19.217793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.066202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658937+0000","last_change":"2023-11-28T13:00:17.653882+0000","last_active":"2023-11-28T13:00:17.658937+0000","last_peered":"2023-11-28T13:00:17.658937+0000","last_clean":"2023-11-28T13:00:17.658937+0000","last_became_active":"2023-11-28T13:00:17.653497+0000","last_became_peered":"2023-11-28T13:00:17.653497+0000","last_unstale":"2023-11-28T13:00:17.658937+0000","last_undegraded":"2023-11-28T13:00:17.658937+0000","last_fullsized":"2023-11-28T13:00:17.658937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:59.591407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654270+0000","last_change":"2023-11-28T13:00:17.652105+0000","last_active":"2023-11-28T13:00:17.654270+0000","last_peered":"2023-11-28T13:00:17.654270+0000","last_clean":"2023-11-28T13:00:17.654270+0000","last_became_active":"2023-11-28T13:00:17.651804+0000","last_became_peered":"2023-11-28T13:00:17.651804+0000","last_unstale":"2023-11-28T13:00:17.654270+0000","last_undegraded":"2023-11-28T13:00:17.654270+0000","last_fullsized":"2023-11-28T13:00:17.654270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232480+0000","last_change":"2023-11-28T13:00:17.232480+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232480+0000","last_undegraded":"2023-11-28T13:00:17.232480+0000","last_fullsized":"2023-11-28T13:00:17.232480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215336+0000","last_change":"2023-11-28T13:00:17.645773+0000","last_active":"2023-11-28T13:00:19.215336+0000","last_peered":"2023-11-28T13:00:19.215336+0000","last_clean":"2023-11-28T13:00:19.215336+0000","last_became_active":"2023-11-28T13:00:17.643545+0000","last_became_peered":"2023-11-28T13:00:17.643545+0000","last_unstale":"2023-11-28T13:00:19.215336+0000","last_undegraded":"2023-11-28T13:00:19.215336+0000","last_fullsized":"2023-11-28T13:00:19.215336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:36.504863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654082+0000","last_change":"2023-11-28T13:00:17.651625+0000","last_active":"2023-11-28T13:00:17.654082+0000","last_peered":"2023-11-28T13:00:17.654082+0000","last_clean":"2023-11-28T13:00:17.654082+0000","last_became_active":"2023-11-28T13:00:17.651210+0000","last_became_peered":"2023-11-28T13:00:17.651210+0000","last_unstale":"2023-11-28T13:00:17.654082+0000","last_undegraded":"2023-11-28T13:00:17.654082+0000","last_fullsized":"2023-11-28T13:00:17.654082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635880+0000","last_change":"2023-11-28T13:00:17.646690+0000","last_active":"2023-11-28T13:00:18.635880+0000","last_peered":"2023-11-28T13:00:18.635880+0000","last_clean":"2023-11-28T13:00:18.635880+0000","last_became_active":"2023-11-28T13:00:17.646438+0000","last_became_peered":"2023-11-28T13:00:17.646438+0000","last_unstale":"2023-11-28T13:00:18.635880+0000","last_undegraded":"2023-11-28T13:00:18.635880+0000","last_fullsized":"2023-11-28T13:00:18.635880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:52.870009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636047+0000","last_change":"2023-11-28T13:00:17.641404+0000","last_active":"2023-11-28T13:00:18.636047+0000","last_peered":"2023-11-28T13:00:18.636047+0000","last_clean":"2023-11-28T13:00:18.636047+0000","last_became_active":"2023-11-28T13:00:17.641140+0000","last_became_peered":"2023-11-28T13:00:17.641140+0000","last_unstale":"2023-11-28T13:00:18.636047+0000","last_undegraded":"2023-11-28T13:00:18.636047+0000","last_fullsized":"2023-11-28T13:00:18.636047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648907+0000","last_change":"2023-11-28T13:00:17.645292+0000","last_active":"2023-11-28T13:00:17.648907+0000","last_peered":"2023-11-28T13:00:17.648907+0000","last_clean":"2023-11-28T13:00:17.648907+0000","last_became_active":"2023-11-28T13:00:17.645001+0000","last_became_peered":"2023-11-28T13:00:17.645001+0000","last_unstale":"2023-11-28T13:00:17.648907+0000","last_undegraded":"2023-11-28T13:00:17.648907+0000","last_fullsized":"2023-11-28T13:00:17.648907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652791+0000","last_change":"2023-11-28T13:00:17.245858+0000","last_active":"2023-11-28T13:00:17.652791+0000","last_peered":"2023-11-28T13:00:17.652791+0000","last_clean":"2023-11-28T13:00:17.652791+0000","last_became_active":"2023-11-28T13:00:17.245568+0000","last_became_peered":"2023-11-28T13:00:17.245568+0000","last_unstale":"2023-11-28T13:00:17.652791+0000","last_undegraded":"2023-11-28T13:00:17.652791+0000","last_fullsized":"2023-11-28T13:00:17.652791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215475+0000","last_change":"2023-11-28T13:00:17.644504+0000","last_active":"2023-11-28T13:00:19.215475+0000","last_peered":"2023-11-28T13:00:19.215475+0000","last_clean":"2023-11-28T13:00:19.215475+0000","last_became_active":"2023-11-28T13:00:17.644099+0000","last_became_peered":"2023-11-28T13:00:17.644099+0000","last_unstale":"2023-11-28T13:00:19.215475+0000","last_undegraded":"2023-11-28T13:00:19.215475+0000","last_fullsized":"2023-11-28T13:00:19.215475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635846+0000","last_change":"2023-11-28T13:00:17.236124+0000","last_active":"2023-11-28T13:00:18.635846+0000","last_peered":"2023-11-28T13:00:18.635846+0000","last_clean":"2023-11-28T13:00:18.635846+0000","last_became_active":"2023-11-28T13:00:17.235500+0000","last_became_peered":"2023-11-28T13:00:17.235500+0000","last_unstale":"2023-11-28T13:00:18.635846+0000","last_undegraded":"2023-11-28T13:00:18.635846+0000","last_fullsized":"2023-11-28T13:00:18.635846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:19.252999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.234634+0000","last_change":"2023-11-28T13:00:17.234634+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.234634+0000","last_undegraded":"2023-11-28T13:00:17.234634+0000","last_fullsized":"2023-11-28T13:00:17.234634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215390+0000","last_change":"2023-11-28T13:00:17.240445+0000","last_active":"2023-11-28T13:00:19.215390+0000","last_peered":"2023-11-28T13:00:19.215390+0000","last_clean":"2023-11-28T13:00:19.215390+0000","last_became_active":"2023-11-28T13:00:17.240095+0000","last_became_peered":"2023-11-28T13:00:17.240095+0000","last_unstale":"2023-11-28T13:00:19.215390+0000","last_undegraded":"2023-11-28T13:00:19.215390+0000","last_fullsized":"2023-11-28T13:00:19.215390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648526+0000","last_change":"2023-11-28T13:00:17.644329+0000","last_active":"2023-11-28T13:00:17.648526+0000","last_peered":"2023-11-28T13:00:17.648526+0000","last_clean":"2023-11-28T13:00:17.648526+0000","last_became_active":"2023-11-28T13:00:17.644046+0000","last_became_peered":"2023-11-28T13:00:17.644046+0000","last_unstale":"2023-11-28T13:00:17.648526+0000","last_undegraded":"2023-11-28T13:00:17.648526+0000","last_fullsized":"2023-11-28T13:00:17.648526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.849042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247832+0000","last_change":"2023-11-28T13:00:17.242514+0000","last_active":"2023-11-28T13:00:17.247832+0000","last_peered":"2023-11-28T13:00:17.247832+0000","last_clean":"2023-11-28T13:00:17.247832+0000","last_became_active":"2023-11-28T13:00:17.242258+0000","last_became_peered":"2023-11-28T13:00:17.242258+0000","last_unstale":"2023-11-28T13:00:17.247832+0000","last_undegraded":"2023-11-28T13:00:17.247832+0000","last_fullsized":"2023-11-28T13:00:17.247832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215295+0000","last_change":"2023-11-28T13:00:17.642442+0000","last_active":"2023-11-28T13:00:19.215295+0000","last_peered":"2023-11-28T13:00:19.215295+0000","last_clean":"2023-11-28T13:00:19.215295+0000","last_became_active":"2023-11-28T13:00:17.642162+0000","last_became_peered":"2023-11-28T13:00:17.642162+0000","last_unstale":"2023-11-28T13:00:19.215295+0000","last_undegraded":"2023-11-28T13:00:19.215295+0000","last_fullsized":"2023-11-28T13:00:19.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.635989+0000","last_change":"2023-11-28T13:00:17.238867+0000","last_active":"2023-11-28T13:00:17.635989+0000","last_peered":"2023-11-28T13:00:17.635989+0000","last_clean":"2023-11-28T13:00:17.635989+0000","last_became_active":"2023-11-28T13:00:17.238545+0000","last_became_peered":"2023-11-28T13:00:17.238545+0000","last_unstale":"2023-11-28T13:00:17.635989+0000","last_undegraded":"2023-11-28T13:00:17.635989+0000","last_fullsized":"2023-11-28T13:00:17.635989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215335+0000","last_change":"2023-11-28T13:00:17.645527+0000","last_active":"2023-11-28T13:00:19.215335+0000","last_peered":"2023-11-28T13:00:19.215335+0000","last_clean":"2023-11-28T13:00:19.215335+0000","last_became_active":"2023-11-28T13:00:17.644057+0000","last_became_peered":"2023-11-28T13:00:17.644057+0000","last_unstale":"2023-11-28T13:00:19.215335+0000","last_undegraded":"2023-11-28T13:00:19.215335+0000","last_fullsized":"2023-11-28T13:00:19.215335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:55.013485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651526+0000","last_change":"2023-11-28T13:00:17.242375+0000","last_active":"2023-11-28T13:00:17.651526+0000","last_peered":"2023-11-28T13:00:17.651526+0000","last_clean":"2023-11-28T13:00:17.651526+0000","last_became_active":"2023-11-28T13:00:17.242010+0000","last_became_peered":"2023-11-28T13:00:17.242010+0000","last_unstale":"2023-11-28T13:00:17.651526+0000","last_undegraded":"2023-11-28T13:00:17.651526+0000","last_fullsized":"2023-11-28T13:00:17.651526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.522601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635882+0000","last_change":"2023-11-28T13:00:17.647427+0000","last_active":"2023-11-28T13:00:18.635882+0000","last_peered":"2023-11-28T13:00:18.635882+0000","last_clean":"2023-11-28T13:00:18.635882+0000","last_became_active":"2023-11-28T13:00:17.646853+0000","last_became_peered":"2023-11-28T13:00:17.646853+0000","last_unstale":"2023-11-28T13:00:18.635882+0000","last_undegraded":"2023-11-28T13:00:18.635882+0000","last_fullsized":"2023-11-28T13:00:18.635882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.661600+0000","last_change":"2023-11-28T13:00:17.243823+0000","last_active":"2023-11-28T13:00:17.661600+0000","last_peered":"2023-11-28T13:00:17.661600+0000","last_clean":"2023-11-28T13:00:17.661600+0000","last_became_active":"2023-11-28T13:00:17.241238+0000","last_became_peered":"2023-11-28T13:00:17.241238+0000","last_unstale":"2023-11-28T13:00:17.661600+0000","last_undegraded":"2023-11-28T13:00:17.661600+0000","last_fullsized":"2023-11-28T13:00:17.661600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656243+0000","last_change":"2023-11-28T13:00:17.649986+0000","last_active":"2023-11-28T13:00:17.656243+0000","last_peered":"2023-11-28T13:00:17.656243+0000","last_clean":"2023-11-28T13:00:17.656243+0000","last_became_active":"2023-11-28T13:00:17.649402+0000","last_became_peered":"2023-11-28T13:00:17.649402+0000","last_unstale":"2023-11-28T13:00:17.656243+0000","last_undegraded":"2023-11-28T13:00:17.656243+0000","last_fullsized":"2023-11-28T13:00:17.656243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:15.471790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656508+0000","last_change":"2023-11-28T13:00:17.247047+0000","last_active":"2023-11-28T13:00:17.656508+0000","last_peered":"2023-11-28T13:00:17.656508+0000","last_clean":"2023-11-28T13:00:17.656508+0000","last_became_active":"2023-11-28T13:00:17.246742+0000","last_became_peered":"2023-11-28T13:00:17.246742+0000","last_unstale":"2023-11-28T13:00:17.656508+0000","last_undegraded":"2023-11-28T13:00:17.656508+0000","last_fullsized":"2023-11-28T13:00:17.656508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:50.559956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215527+0000","last_change":"2023-11-28T13:00:17.641605+0000","last_active":"2023-11-28T13:00:19.215527+0000","last_peered":"2023-11-28T13:00:19.215527+0000","last_clean":"2023-11-28T13:00:19.215527+0000","last_became_active":"2023-11-28T13:00:17.641350+0000","last_became_peered":"2023-11-28T13:00:17.641350+0000","last_unstale":"2023-11-28T13:00:19.215527+0000","last_undegraded":"2023-11-28T13:00:19.215527+0000","last_fullsized":"2023-11-28T13:00:19.215527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245165+0000","last_change":"2023-11-28T13:00:17.240432+0000","last_active":"2023-11-28T13:00:17.245165+0000","last_peered":"2023-11-28T13:00:17.245165+0000","last_clean":"2023-11-28T13:00:17.245165+0000","last_became_active":"2023-11-28T13:00:17.239192+0000","last_became_peered":"2023-11-28T13:00:17.239192+0000","last_unstale":"2023-11-28T13:00:17.245165+0000","last_undegraded":"2023-11-28T13:00:17.245165+0000","last_fullsized":"2023-11-28T13:00:17.245165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651770+0000","last_change":"2023-11-28T13:00:17.648774+0000","last_active":"2023-11-28T13:00:17.651770+0000","last_peered":"2023-11-28T13:00:17.651770+0000","last_clean":"2023-11-28T13:00:17.651770+0000","last_became_active":"2023-11-28T13:00:17.647403+0000","last_became_peered":"2023-11-28T13:00:17.647403+0000","last_unstale":"2023-11-28T13:00:17.651770+0000","last_undegraded":"2023-11-28T13:00:17.651770+0000","last_fullsized":"2023-11-28T13:00:17.651770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:17.592744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633096+0000","last_change":"2023-11-28T13:00:17.245977+0000","last_active":"2023-11-28T13:00:17.633096+0000","last_peered":"2023-11-28T13:00:17.633096+0000","last_clean":"2023-11-28T13:00:17.633096+0000","last_became_active":"2023-11-28T13:00:17.245651+0000","last_became_peered":"2023-11-28T13:00:17.245651+0000","last_unstale":"2023-11-28T13:00:17.633096+0000","last_undegraded":"2023-11-28T13:00:17.633096+0000","last_fullsized":"2023-11-28T13:00:17.633096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:30.760573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.235374+0000","last_change":"2023-11-28T13:00:17.235374+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.235374+0000","last_undegraded":"2023-11-28T13:00:17.235374+0000","last_fullsized":"2023-11-28T13:00:17.235374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650468+0000","last_change":"2023-11-28T13:00:17.240716+0000","last_active":"2023-11-28T13:00:17.650468+0000","last_peered":"2023-11-28T13:00:17.650468+0000","last_clean":"2023-11-28T13:00:17.650468+0000","last_became_active":"2023-11-28T13:00:17.239928+0000","last_became_peered":"2023-11-28T13:00:17.239928+0000","last_unstale":"2023-11-28T13:00:17.650468+0000","last_undegraded":"2023-11-28T13:00:17.650468+0000","last_fullsized":"2023-11-28T13:00:17.650468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:32.086174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215680+0000","last_change":"2023-11-28T13:00:17.643923+0000","last_active":"2023-11-28T13:00:19.215680+0000","last_peered":"2023-11-28T13:00:19.215680+0000","last_clean":"2023-11-28T13:00:19.215680+0000","last_became_active":"2023-11-28T13:00:17.643579+0000","last_became_peered":"2023-11-28T13:00:17.643579+0000","last_unstale":"2023-11-28T13:00:19.215680+0000","last_undegraded":"2023-11-28T13:00:19.215680+0000","last_fullsized":"2023-11-28T13:00:19.215680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654531+0000","last_change":"2023-11-28T13:00:17.247259+0000","last_active":"2023-11-28T13:00:17.654531+0000","last_peered":"2023-11-28T13:00:17.654531+0000","last_clean":"2023-11-28T13:00:17.654531+0000","last_became_active":"2023-11-28T13:00:17.246943+0000","last_became_peered":"2023-11-28T13:00:17.246943+0000","last_unstale":"2023-11-28T13:00:17.654531+0000","last_undegraded":"2023-11-28T13:00:17.654531+0000","last_fullsized":"2023-11-28T13:00:17.654531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:02.643388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647850+0000","last_change":"2023-11-28T13:00:17.644322+0000","last_active":"2023-11-28T13:00:17.647850+0000","last_peered":"2023-11-28T13:00:17.647850+0000","last_clean":"2023-11-28T13:00:17.647850+0000","last_became_active":"2023-11-28T13:00:17.644054+0000","last_became_peered":"2023-11-28T13:00:17.644054+0000","last_unstale":"2023-11-28T13:00:17.647850+0000","last_undegraded":"2023-11-28T13:00:17.647850+0000","last_fullsized":"2023-11-28T13:00:17.647850+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:46.098419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656474+0000","last_change":"2023-11-28T13:00:17.247274+0000","last_active":"2023-11-28T13:00:17.656474+0000","last_peered":"2023-11-28T13:00:17.656474+0000","last_clean":"2023-11-28T13:00:17.656474+0000","last_became_active":"2023-11-28T13:00:17.246851+0000","last_became_peered":"2023-11-28T13:00:17.246851+0000","last_unstale":"2023-11-28T13:00:17.656474+0000","last_undegraded":"2023-11-28T13:00:17.656474+0000","last_fullsized":"2023-11-28T13:00:17.656474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:17.621082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652526+0000","last_change":"2023-11-28T13:00:17.640395+0000","last_active":"2023-11-28T13:00:17.652526+0000","last_peered":"2023-11-28T13:00:17.652526+0000","last_clean":"2023-11-28T13:00:17.652526+0000","last_became_active":"2023-11-28T13:00:17.640195+0000","last_became_peered":"2023-11-28T13:00:17.640195+0000","last_unstale":"2023-11-28T13:00:17.652526+0000","last_undegraded":"2023-11-28T13:00:17.652526+0000","last_fullsized":"2023-11-28T13:00:17.652526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:14.878366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.646780+0000","last_change":"2023-11-28T13:00:17.240698+0000","last_active":"2023-11-28T13:00:17.646780+0000","last_peered":"2023-11-28T13:00:17.646780+0000","last_clean":"2023-11-28T13:00:17.646780+0000","last_became_active":"2023-11-28T13:00:17.240387+0000","last_became_peered":"2023-11-28T13:00:17.240387+0000","last_unstale":"2023-11-28T13:00:17.646780+0000","last_undegraded":"2023-11-28T13:00:17.646780+0000","last_fullsized":"2023-11-28T13:00:17.646780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:05.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648509+0000","last_change":"2023-11-28T13:00:17.647318+0000","last_active":"2023-11-28T13:00:17.648509+0000","last_peered":"2023-11-28T13:00:17.648509+0000","last_clean":"2023-11-28T13:00:17.648509+0000","last_became_active":"2023-11-28T13:00:17.647043+0000","last_became_peered":"2023-11-28T13:00:17.647043+0000","last_unstale":"2023-11-28T13:00:17.648509+0000","last_undegraded":"2023-11-28T13:00:17.648509+0000","last_fullsized":"2023-11-28T13:00:17.648509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:38.321116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634419+0000","last_change":"2023-11-28T13:00:17.235843+0000","last_active":"2023-11-28T13:00:17.634419+0000","last_peered":"2023-11-28T13:00:17.634419+0000","last_clean":"2023-11-28T13:00:17.634419+0000","last_became_active":"2023-11-28T13:00:17.235495+0000","last_became_peered":"2023-11-28T13:00:17.235495+0000","last_unstale":"2023-11-28T13:00:17.634419+0000","last_undegraded":"2023-11-28T13:00:17.634419+0000","last_fullsized":"2023-11-28T13:00:17.634419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232454+0000","last_change":"2023-11-28T13:00:17.232454+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232454+0000","last_undegraded":"2023-11-28T13:00:17.232454+0000","last_fullsized":"2023-11-28T13:00:17.232454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215456+0000","last_change":"2023-11-28T13:00:17.240709+0000","last_active":"2023-11-28T13:00:19.215456+0000","last_peered":"2023-11-28T13:00:19.215456+0000","last_clean":"2023-11-28T13:00:19.215456+0000","last_became_active":"2023-11-28T13:00:17.239893+0000","last_became_peered":"2023-11-28T13:00:17.239893+0000","last_unstale":"2023-11-28T13:00:19.215456+0000","last_undegraded":"2023-11-28T13:00:19.215456+0000","last_fullsized":"2023-11-28T13:00:19.215456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658839+0000","last_change":"2023-11-28T13:00:17.653235+0000","last_active":"2023-11-28T13:00:17.658839+0000","last_peered":"2023-11-28T13:00:17.658839+0000","last_clean":"2023-11-28T13:00:17.658839+0000","last_became_active":"2023-11-28T13:00:17.650249+0000","last_became_peered":"2023-11-28T13:00:17.650249+0000","last_unstale":"2023-11-28T13:00:17.658839+0000","last_undegraded":"2023-11-28T13:00:17.658839+0000","last_fullsized":"2023-11-28T13:00:17.658839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631372+0000","last_change":"2023-11-28T13:00:17.245224+0000","last_active":"2023-11-28T13:00:17.631372+0000","last_peered":"2023-11-28T13:00:17.631372+0000","last_clean":"2023-11-28T13:00:17.631372+0000","last_became_active":"2023-11-28T13:00:17.244913+0000","last_became_peered":"2023-11-28T13:00:17.244913+0000","last_unstale":"2023-11-28T13:00:17.631372+0000","last_undegraded":"2023-11-28T13:00:17.631372+0000","last_fullsized":"2023-11-28T13:00:17.631372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653954+0000","last_change":"2023-11-28T13:00:17.643015+0000","last_active":"2023-11-28T13:00:17.653954+0000","last_peered":"2023-11-28T13:00:17.653954+0000","last_clean":"2023-11-28T13:00:17.653954+0000","last_became_active":"2023-11-28T13:00:17.642470+0000","last_became_peered":"2023-11-28T13:00:17.642470+0000","last_unstale":"2023-11-28T13:00:17.653954+0000","last_undegraded":"2023-11-28T13:00:17.653954+0000","last_fullsized":"2023-11-28T13:00:17.653954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:41.679867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.639452+0000","last_change":"2023-11-28T13:00:17.244207+0000","last_active":"2023-11-28T13:00:17.639452+0000","last_peered":"2023-11-28T13:00:17.639452+0000","last_clean":"2023-11-28T13:00:17.639452+0000","last_became_active":"2023-11-28T13:00:17.243861+0000","last_became_peered":"2023-11-28T13:00:17.243861+0000","last_unstale":"2023-11-28T13:00:17.639452+0000","last_undegraded":"2023-11-28T13:00:17.639452+0000","last_fullsized":"2023-11-28T13:00:17.639452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656262+0000","last_change":"2023-11-28T13:00:17.651269+0000","last_active":"2023-11-28T13:00:17.656262+0000","last_peered":"2023-11-28T13:00:17.656262+0000","last_clean":"2023-11-28T13:00:17.656262+0000","last_became_active":"2023-11-28T13:00:17.650883+0000","last_became_peered":"2023-11-28T13:00:17.650883+0000","last_unstale":"2023-11-28T13:00:17.656262+0000","last_undegraded":"2023-11-28T13:00:17.656262+0000","last_fullsized":"2023-11-28T13:00:17.656262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215191+0000","last_change":"2023-11-28T13:00:17.238617+0000","last_active":"2023-11-28T13:00:19.215191+0000","last_peered":"2023-11-28T13:00:19.215191+0000","last_clean":"2023-11-28T13:00:19.215191+0000","last_became_active":"2023-11-28T13:00:17.238344+0000","last_became_peered":"2023-11-28T13:00:17.238344+0000","last_unstale":"2023-11-28T13:00:19.215191+0000","last_undegraded":"2023-11-28T13:00:19.215191+0000","last_fullsized":"2023-11-28T13:00:19.215191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:46.225565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636017+0000","last_change":"2023-11-28T13:00:17.649564+0000","last_active":"2023-11-28T13:00:18.636017+0000","last_peered":"2023-11-28T13:00:18.636017+0000","last_clean":"2023-11-28T13:00:18.636017+0000","last_became_active":"2023-11-28T13:00:17.649312+0000","last_became_peered":"2023-11-28T13:00:17.649312+0000","last_unstale":"2023-11-28T13:00:18.636017+0000","last_undegraded":"2023-11-28T13:00:18.636017+0000","last_fullsized":"2023-11-28T13:00:18.636017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634764+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:00:17.634764+0000","last_peered":"2023-11-28T13:00:17.634764+0000","last_clean":"2023-11-28T13:00:17.634764+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:00:17.634764+0000","last_undegraded":"2023-11-28T13:00:17.634764+0000","last_fullsized":"2023-11-28T13:00:17.634764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.636391+0000","last_change":"2023-11-28T13:00:17.240503+0000","last_active":"2023-11-28T13:00:17.636391+0000","last_peered":"2023-11-28T13:00:17.636391+0000","last_clean":"2023-11-28T13:00:17.636391+0000","last_became_active":"2023-11-28T13:00:17.240256+0000","last_became_peered":"2023-11-28T13:00:17.240256+0000","last_unstale":"2023-11-28T13:00:17.636391+0000","last_undegraded":"2023-11-28T13:00:17.636391+0000","last_fullsized":"2023-11-28T13:00:17.636391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635946+0000","last_change":"2023-11-28T13:00:17.645964+0000","last_active":"2023-11-28T13:00:18.635946+0000","last_peered":"2023-11-28T13:00:18.635946+0000","last_clean":"2023-11-28T13:00:18.635946+0000","last_became_active":"2023-11-28T13:00:17.645384+0000","last_became_peered":"2023-11-28T13:00:17.645384+0000","last_unstale":"2023-11-28T13:00:18.635946+0000","last_undegraded":"2023-11-28T13:00:18.635946+0000","last_fullsized":"2023-11-28T13:00:18.635946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234232+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:00:17.234232+0000","last_peered":"2023-11-28T13:00:17.234232+0000","last_clean":"2023-11-28T13:00:17.234232+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:00:17.234232+0000","last_undegraded":"2023-11-28T13:00:17.234232+0000","last_fullsized":"2023-11-28T13:00:17.234232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.640096+0000","last_change":"2023-11-28T13:00:17.245773+0000","last_active":"2023-11-28T13:00:17.640096+0000","last_peered":"2023-11-28T13:00:17.640096+0000","last_clean":"2023-11-28T13:00:17.640096+0000","last_became_active":"2023-11-28T13:00:17.245409+0000","last_became_peered":"2023-11-28T13:00:17.245409+0000","last_unstale":"2023-11-28T13:00:17.640096+0000","last_undegraded":"2023-11-28T13:00:17.640096+0000","last_fullsized":"2023-11-28T13:00:17.640096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:00.519142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636216+0000","last_change":"2023-11-28T13:00:17.641395+0000","last_active":"2023-11-28T13:00:18.636216+0000","last_peered":"2023-11-28T13:00:18.636216+0000","last_clean":"2023-11-28T13:00:18.636216+0000","last_became_active":"2023-11-28T13:00:17.641149+0000","last_became_peered":"2023-11-28T13:00:17.641149+0000","last_unstale":"2023-11-28T13:00:18.636216+0000","last_undegraded":"2023-11-28T13:00:18.636216+0000","last_fullsized":"2023-11-28T13:00:18.636216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.628022+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:00:17.628022+0000","last_peered":"2023-11-28T13:00:17.628022+0000","last_clean":"2023-11-28T13:00:17.628022+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:00:17.628022+0000","last_undegraded":"2023-11-28T13:00:17.628022+0000","last_fullsized":"2023-11-28T13:00:17.628022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635610+0000","last_change":"2023-11-28T13:00:17.237802+0000","last_active":"2023-11-28T13:00:18.635610+0000","last_peered":"2023-11-28T13:00:18.635610+0000","last_clean":"2023-11-28T13:00:18.635610+0000","last_became_active":"2023-11-28T13:00:17.237487+0000","last_became_peered":"2023-11-28T13:00:17.237487+0000","last_unstale":"2023-11-28T13:00:18.635610+0000","last_undegraded":"2023-11-28T13:00:18.635610+0000","last_fullsized":"2023-11-28T13:00:18.635610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635877+0000","last_change":"2023-11-28T13:00:17.644511+0000","last_active":"2023-11-28T13:00:18.635877+0000","last_peered":"2023-11-28T13:00:18.635877+0000","last_clean":"2023-11-28T13:00:18.635877+0000","last_became_active":"2023-11-28T13:00:17.644095+0000","last_became_peered":"2023-11-28T13:00:17.644095+0000","last_unstale":"2023-11-28T13:00:18.635877+0000","last_undegraded":"2023-11-28T13:00:18.635877+0000","last_fullsized":"2023-11-28T13:00:18.635877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234538+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:00:17.234538+0000","last_peered":"2023-11-28T13:00:17.234538+0000","last_clean":"2023-11-28T13:00:17.234538+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:00:17.234538+0000","last_undegraded":"2023-11-28T13:00:17.234538+0000","last_fullsized":"2023-11-28T13:00:17.234538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235309+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:00:17.235309+0000","last_peered":"2023-11-28T13:00:17.235309+0000","last_clean":"2023-11-28T13:00:17.235309+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:00:17.235309+0000","last_undegraded":"2023-11-28T13:00:17.235309+0000","last_fullsized":"2023-11-28T13:00:17.235309+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215311+0000","last_change":"2023-11-28T13:00:17.239373+0000","last_active":"2023-11-28T13:00:19.215311+0000","last_peered":"2023-11-28T13:00:19.215311+0000","last_clean":"2023-11-28T13:00:19.215311+0000","last_became_active":"2023-11-28T13:00:17.238864+0000","last_became_peered":"2023-11-28T13:00:17.238864+0000","last_unstale":"2023-11-28T13:00:19.215311+0000","last_undegraded":"2023-11-28T13:00:19.215311+0000","last_fullsized":"2023-11-28T13:00:19.215311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:17.763933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653117+0000","last_change":"2023-11-28T13:00:17.649771+0000","last_active":"2023-11-28T13:00:17.653117+0000","last_peered":"2023-11-28T13:00:17.653117+0000","last_clean":"2023-11-28T13:00:17.653117+0000","last_became_active":"2023-11-28T13:00:17.649475+0000","last_became_peered":"2023-11-28T13:00:17.649475+0000","last_unstale":"2023-11-28T13:00:17.653117+0000","last_undegraded":"2023-11-28T13:00:17.653117+0000","last_fullsized":"2023-11-28T13:00:17.653117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235498+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:00:17.235498+0000","last_peered":"2023-11-28T13:00:17.235498+0000","last_clean":"2023-11-28T13:00:17.235498+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:00:17.235498+0000","last_undegraded":"2023-11-28T13:00:17.235498+0000","last_fullsized":"2023-11-28T13:00:17.235498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253107+0000","last_change":"2023-11-28T13:00:17.251262+0000","last_active":"2023-11-28T13:00:17.253107+0000","last_peered":"2023-11-28T13:00:17.253107+0000","last_clean":"2023-11-28T13:00:17.253107+0000","last_became_active":"2023-11-28T13:00:17.250948+0000","last_became_peered":"2023-11-28T13:00:17.250948+0000","last_unstale":"2023-11-28T13:00:17.253107+0000","last_undegraded":"2023-11-28T13:00:17.253107+0000","last_fullsized":"2023-11-28T13:00:17.253107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:37.311866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233919+0000","last_change":"2023-11-28T13:00:17.233919+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233919+0000","last_undegraded":"2023-11-28T13:00:17.233919+0000","last_fullsized":"2023-11-28T13:00:17.233919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.236149+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:00:17.236149+0000","last_peered":"2023-11-28T13:00:17.236149+0000","last_clean":"2023-11-28T13:00:17.236149+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:00:17.236149+0000","last_undegraded":"2023-11-28T13:00:17.236149+0000","last_fullsized":"2023-11-28T13:00:17.236149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636224+0000","last_change":"2023-11-28T13:00:17.237999+0000","last_active":"2023-11-28T13:00:18.636224+0000","last_peered":"2023-11-28T13:00:18.636224+0000","last_clean":"2023-11-28T13:00:18.636224+0000","last_became_active":"2023-11-28T13:00:17.237625+0000","last_became_peered":"2023-11-28T13:00:17.237625+0000","last_unstale":"2023-11-28T13:00:18.636224+0000","last_undegraded":"2023-11-28T13:00:18.636224+0000","last_fullsized":"2023-11-28T13:00:18.636224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233381+0000","last_change":"2023-11-28T13:00:17.233381+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233381+0000","last_undegraded":"2023-11-28T13:00:17.233381+0000","last_fullsized":"2023-11-28T13:00:17.233381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215660+0000","last_change":"2023-11-28T13:00:17.239284+0000","last_active":"2023-11-28T13:00:19.215660+0000","last_peered":"2023-11-28T13:00:19.215660+0000","last_clean":"2023-11-28T13:00:19.215660+0000","last_became_active":"2023-11-28T13:00:17.238997+0000","last_became_peered":"2023-11-28T13:00:17.238997+0000","last_unstale":"2023-11-28T13:00:19.215660+0000","last_undegraded":"2023-11-28T13:00:19.215660+0000","last_fullsized":"2023-11-28T13:00:19.215660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:55.423382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633198+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:00:17.633198+0000","last_peered":"2023-11-28T13:00:17.633198+0000","last_clean":"2023-11-28T13:00:17.633198+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:00:17.633198+0000","last_undegraded":"2023-11-28T13:00:17.633198+0000","last_fullsized":"2023-11-28T13:00:17.633198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215108+0000","last_change":"2023-11-28T13:00:17.646149+0000","last_active":"2023-11-28T13:00:19.215108+0000","last_peered":"2023-11-28T13:00:19.215108+0000","last_clean":"2023-11-28T13:00:19.215108+0000","last_became_active":"2023-11-28T13:00:17.645893+0000","last_became_peered":"2023-11-28T13:00:17.645893+0000","last_unstale":"2023-11-28T13:00:19.215108+0000","last_undegraded":"2023-11-28T13:00:19.215108+0000","last_fullsized":"2023-11-28T13:00:19.215108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:24.466207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.226618+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:00:17.226618+0000","last_peered":"2023-11-28T13:00:17.226618+0000","last_clean":"2023-11-28T13:00:17.226618+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:00:17.226618+0000","last_undegraded":"2023-11-28T13:00:17.226618+0000","last_fullsized":"2023-11-28T13:00:17.226618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245166+0000","last_change":"2023-11-28T13:00:17.240379+0000","last_active":"2023-11-28T13:00:17.245166+0000","last_peered":"2023-11-28T13:00:17.245166+0000","last_clean":"2023-11-28T13:00:17.245166+0000","last_became_active":"2023-11-28T13:00:17.239505+0000","last_became_peered":"2023-11-28T13:00:17.239505+0000","last_unstale":"2023-11-28T13:00:17.245166+0000","last_undegraded":"2023-11-28T13:00:17.245166+0000","last_fullsized":"2023-11-28T13:00:17.245166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655975+0000","last_change":"2023-11-28T13:00:17.646003+0000","last_active":"2023-11-28T13:00:17.655975+0000","last_peered":"2023-11-28T13:00:17.655975+0000","last_clean":"2023-11-28T13:00:17.655975+0000","last_became_active":"2023-11-28T13:00:17.645727+0000","last_became_peered":"2023-11-28T13:00:17.645727+0000","last_unstale":"2023-11-28T13:00:17.655975+0000","last_undegraded":"2023-11-28T13:00:17.655975+0000","last_fullsized":"2023-11-28T13:00:17.655975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:30.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253113+0000","last_change":"2023-11-28T13:00:17.251398+0000","last_active":"2023-11-28T13:00:17.253113+0000","last_peered":"2023-11-28T13:00:17.253113+0000","last_clean":"2023-11-28T13:00:17.253113+0000","last_became_active":"2023-11-28T13:00:17.251054+0000","last_became_peered":"2023-11-28T13:00:17.251054+0000","last_unstale":"2023-11-28T13:00:17.253113+0000","last_undegraded":"2023-11-28T13:00:17.253113+0000","last_fullsized":"2023-11-28T13:00:17.253113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217848+0000","last_change":"2023-11-28T13:00:17.641355+0000","last_active":"2023-11-28T13:00:19.217848+0000","last_peered":"2023-11-28T13:00:19.217848+0000","last_clean":"2023-11-28T13:00:19.217848+0000","last_became_active":"2023-11-28T13:00:17.641093+0000","last_became_peered":"2023-11-28T13:00:17.641093+0000","last_unstale":"2023-11-28T13:00:19.217848+0000","last_undegraded":"2023-11-28T13:00:19.217848+0000","last_fullsized":"2023-11-28T13:00:19.217848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:41.816686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655479+0000","last_change":"2023-11-28T13:00:17.246896+0000","last_active":"2023-11-28T13:00:17.655479+0000","last_peered":"2023-11-28T13:00:17.655479+0000","last_clean":"2023-11-28T13:00:17.655479+0000","last_became_active":"2023-11-28T13:00:17.246604+0000","last_became_peered":"2023-11-28T13:00:17.246604+0000","last_unstale":"2023-11-28T13:00:17.655479+0000","last_undegraded":"2023-11-28T13:00:17.655479+0000","last_fullsized":"2023-11-28T13:00:17.655479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656231+0000","last_change":"2023-11-28T13:00:17.648973+0000","last_active":"2023-11-28T13:00:17.656231+0000","last_peered":"2023-11-28T13:00:17.656231+0000","last_clean":"2023-11-28T13:00:17.656231+0000","last_became_active":"2023-11-28T13:00:17.648323+0000","last_became_peered":"2023-11-28T13:00:17.648323+0000","last_unstale":"2023-11-28T13:00:17.656231+0000","last_undegraded":"2023-11-28T13:00:17.656231+0000","last_fullsized":"2023-11-28T13:00:17.656231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654699+0000","last_change":"2023-11-28T13:00:17.247013+0000","last_active":"2023-11-28T13:00:17.654699+0000","last_peered":"2023-11-28T13:00:17.654699+0000","last_clean":"2023-11-28T13:00:17.654699+0000","last_became_active":"2023-11-28T13:00:17.246645+0000","last_became_peered":"2023-11-28T13:00:17.246645+0000","last_unstale":"2023-11-28T13:00:17.654699+0000","last_undegraded":"2023-11-28T13:00:17.654699+0000","last_fullsized":"2023-11-28T13:00:17.654699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656116+0000","last_change":"2023-11-28T13:00:17.649078+0000","last_active":"2023-11-28T13:00:17.656116+0000","last_peered":"2023-11-28T13:00:17.656116+0000","last_clean":"2023-11-28T13:00:17.656116+0000","last_became_active":"2023-11-28T13:00:17.648432+0000","last_became_peered":"2023-11-28T13:00:17.648432+0000","last_unstale":"2023-11-28T13:00:17.656116+0000","last_undegraded":"2023-11-28T13:00:17.656116+0000","last_fullsized":"2023-11-28T13:00:17.656116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215495+0000","last_change":"2023-11-28T13:00:17.234074+0000","last_active":"2023-11-28T13:00:19.215495+0000","last_peered":"2023-11-28T13:00:19.215495+0000","last_clean":"2023-11-28T13:00:19.215495+0000","last_became_active":"2023-11-28T13:00:17.233732+0000","last_became_peered":"2023-11-28T13:00:17.233732+0000","last_unstale":"2023-11-28T13:00:19.215495+0000","last_undegraded":"2023-11-28T13:00:19.215495+0000","last_fullsized":"2023-11-28T13:00:19.215495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:36.950864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215231+0000","last_change":"2023-11-28T13:00:17.641105+0000","last_active":"2023-11-28T13:00:19.215231+0000","last_peered":"2023-11-28T13:00:19.215231+0000","last_clean":"2023-11-28T13:00:19.215231+0000","last_became_active":"2023-11-28T13:00:17.640854+0000","last_became_peered":"2023-11-28T13:00:17.640854+0000","last_unstale":"2023-11-28T13:00:19.215231+0000","last_undegraded":"2023-11-28T13:00:19.215231+0000","last_fullsized":"2023-11-28T13:00:19.215231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247814+0000","last_change":"2023-11-28T13:00:17.242508+0000","last_active":"2023-11-28T13:00:17.247814+0000","last_peered":"2023-11-28T13:00:17.247814+0000","last_clean":"2023-11-28T13:00:17.247814+0000","last_became_active":"2023-11-28T13:00:17.242230+0000","last_became_peered":"2023-11-28T13:00:17.242230+0000","last_unstale":"2023-11-28T13:00:17.247814+0000","last_undegraded":"2023-11-28T13:00:17.247814+0000","last_fullsized":"2023-11-28T13:00:17.247814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656224+0000","last_change":"2023-11-28T13:00:17.650868+0000","last_active":"2023-11-28T13:00:17.656224+0000","last_peered":"2023-11-28T13:00:17.656224+0000","last_clean":"2023-11-28T13:00:17.656224+0000","last_became_active":"2023-11-28T13:00:17.649842+0000","last_became_peered":"2023-11-28T13:00:17.649842+0000","last_unstale":"2023-11-28T13:00:17.656224+0000","last_undegraded":"2023-11-28T13:00:17.656224+0000","last_fullsized":"2023-11-28T13:00:17.656224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645743+0000","last_change":"2023-11-28T13:00:17.641330+0000","last_active":"2023-11-28T13:00:17.645743+0000","last_peered":"2023-11-28T13:00:17.645743+0000","last_clean":"2023-11-28T13:00:17.645743+0000","last_became_active":"2023-11-28T13:00:17.641029+0000","last_became_peered":"2023-11-28T13:00:17.641029+0000","last_unstale":"2023-11-28T13:00:17.645743+0000","last_undegraded":"2023-11-28T13:00:17.645743+0000","last_fullsized":"2023-11-28T13:00:17.645743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:28.607875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215202+0000","last_change":"2023-11-28T13:00:17.643816+0000","last_active":"2023-11-28T13:00:19.215202+0000","last_peered":"2023-11-28T13:00:19.215202+0000","last_clean":"2023-11-28T13:00:19.215202+0000","last_became_active":"2023-11-28T13:00:17.643260+0000","last_became_peered":"2023-11-28T13:00:17.643260+0000","last_unstale":"2023-11-28T13:00:19.215202+0000","last_undegraded":"2023-11-28T13:00:19.215202+0000","last_fullsized":"2023-11-28T13:00:19.215202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656140+0000","last_change":"2023-11-28T13:00:17.649431+0000","last_active":"2023-11-28T13:00:17.656140+0000","last_peered":"2023-11-28T13:00:17.656140+0000","last_clean":"2023-11-28T13:00:17.656140+0000","last_became_active":"2023-11-28T13:00:17.648763+0000","last_became_peered":"2023-11-28T13:00:17.648763+0000","last_unstale":"2023-11-28T13:00:17.656140+0000","last_undegraded":"2023-11-28T13:00:17.656140+0000","last_fullsized":"2023-11-28T13:00:17.656140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:10.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215472+0000","last_change":"2023-11-28T13:00:17.649148+0000","last_active":"2023-11-28T13:00:19.215472+0000","last_peered":"2023-11-28T13:00:19.215472+0000","last_clean":"2023-11-28T13:00:19.215472+0000","last_became_active":"2023-11-28T13:00:17.648925+0000","last_became_peered":"2023-11-28T13:00:17.648925+0000","last_unstale":"2023-11-28T13:00:19.215472+0000","last_undegraded":"2023-11-28T13:00:19.215472+0000","last_fullsized":"2023-11-28T13:00:19.215472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636138+0000","last_change":"2023-11-28T13:00:17.648409+0000","last_active":"2023-11-28T13:00:18.636138+0000","last_peered":"2023-11-28T13:00:18.636138+0000","last_clean":"2023-11-28T13:00:18.636138+0000","last_became_active":"2023-11-28T13:00:17.648164+0000","last_became_peered":"2023-11-28T13:00:17.648164+0000","last_unstale":"2023-11-28T13:00:18.636138+0000","last_undegraded":"2023-11-28T13:00:18.636138+0000","last_fullsized":"2023-11-28T13:00:18.636138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":96947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7940,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363900,"statfs":{"total":96636764160,"available":96628633600,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":91808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8928,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362912,"statfs":{"total":96636764160,"available":96627621888,"internally_reserved":0,"allocated":811008,"data_stored":666055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9112,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362728,"statfs":{"total":96636764160,"available":96627433472,"internally_reserved":0,"allocated":868352,"data_stored":682336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8000,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363840,"statfs":{"total":96636764160,"available":96628572160,"internally_reserved":0,"allocated":253952,"data_stored":90840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7992,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363848,"statfs":{"total":96636764160,"available":96628580352,"internally_reserved":0,"allocated":245760,"data_stored":90204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8068,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363772,"statfs":{"total":96636764160,"available":96628502528,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:22.082 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:22.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.236+0000 7f470f9c4700 1 Processor -- start 2023-11-28T13:00:22.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.236+0000 7f470f9c4700 1 -- start start 2023-11-28T13:00:22.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470f9c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708071040 0x7f4708071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470f9c4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4708160540 0x7f47081609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470f9c4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4708071950 con 0x7f470806dee0 2023-11-28T13:00:22.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470f9c4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470815d060 con 0x7f4708160540 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470f9c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470815d340 con 0x7f4708071040 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4708160540 0x7f47081609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4708160540 0x7f47081609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38350/0 (socket says 172.21.15.153:38350) 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 -- 172.21.15.153:0/772239683 learned_addr learned my addr 172.21.15.153:0/772239683 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470d760700 1 --2- 172.21.15.153:0/772239683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708071040 0x7f4708071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 -- 172.21.15.153:0/772239683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708071040 msgr2=0x7f4708071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 --2- 172.21.15.153:0/772239683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708071040 0x7f4708071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 -- 172.21.15.153:0/772239683 >> v1:172.21.15.177:6789/0 conn(0x7f470806dee0 legacy=0x7f470806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 -- 172.21.15.153:0/772239683 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470815d620 con 0x7f4708160540 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.237+0000 7f470df61700 1 --2- 172.21.15.153:0/772239683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708160540 0x7f47081609c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4700009e50 tx=0x7f470000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27189d2ce58d281 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f46fffff700 1 -- 172.21.15.153:0/772239683 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000e070 con 0x7f4708160540 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f46fffff700 1 -- 172.21.15.153:0/772239683 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470000b9c0 con 0x7f4708160540 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f46fffff700 1 -- 172.21.15.153:0/772239683 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000bd70 con 0x7f4708160540 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 -- 172.21.15.153:0/772239683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708160540 msgr2=0x7f47081609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 --2- 172.21.15.153:0/772239683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708160540 0x7f47081609c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4700009e50 tx=0x7f470000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 -- 172.21.15.153:0/772239683 shutdown_connections 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 --2- 172.21.15.153:0/772239683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708160540 0x7f47081609c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 --2- 172.21.15.153:0/772239683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708071040 0x7f4708071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 -- 172.21.15.153:0/772239683 >> 172.21.15.153:0/772239683 conn(0x7f47080d0000 msgr2=0x7f47080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 -- 172.21.15.153:0/772239683 shutdown_connections 2023-11-28T13:00:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.238+0000 7f470f9c4700 1 -- 172.21.15.153:0/772239683 wait complete. 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.239+0000 7f470f9c4700 1 Processor -- start 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.239+0000 7f470f9c4700 1 -- start start 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470f9c4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470806dee0 0x7f470814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470f9c4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708071040 0x7f47080c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470f9c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708160540 0x7f4708159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470f9c4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708156eb0 con 0x7f4708071040 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470f9c4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47080b4ee0 con 0x7f470806dee0 2023-11-28T13:00:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470f9c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470814d880 con 0x7f4708160540 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708071040 0x7f47080c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708071040 0x7f47080c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38354/0 (socket says 172.21.15.153:38354) 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 -- 172.21.15.153:0/1946760106 learned_addr learned my addr 172.21.15.153:0/1946760106 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470df61700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708160540 0x7f4708159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470d760700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470806dee0 0x7f470814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 -- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708160540 msgr2=0x7f4708159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708160540 0x7f4708159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 -- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470806dee0 msgr2=0x7f470814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470806dee0 0x7f470814d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.240+0000 7f470cf5f700 1 -- 172.21.15.153:0/1946760106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700009b30 con 0x7f4708071040 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.241+0000 7f470cf5f700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708071040 0x7f47080c6320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c450 tx=0x7f46f800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.241+0000 7f46fdffb700 1 -- 172.21.15.153:0/1946760106 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80052d0 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.241+0000 7f46fdffb700 1 -- 172.21.15.153:0/1946760106 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8005430 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.241+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470815ccd0 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.241+0000 7f46fdffb700 1 -- 172.21.15.153:0/1946760106 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80038c0 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.241+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470815d900 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.242+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470806a270 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.242+0000 7f46fdffb700 1 -- 172.21.15.153:0/1946760106 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46f8019030 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.242+0000 7f46fdffb700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46ec0682f0 0x7f46ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.243+0000 7f46fdffb700 1 -- 172.21.15.153:0/1946760106 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46f8085000 con 0x7f4708071040 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.243+0000 7f470d760700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46ec0682f0 0x7f46ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.243+0000 7f470d760700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46ec0682f0 0x7f46ec06a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4704009770 tx=0x7f4704008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.247+0000 7f46fdffb700 1 -- 172.21.15.153:0/1946760106 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46f80516e0 con 0x7f4708071040 2023-11-28T13:00:22.428 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.426+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f470806f440 con 0x7f46ec0682f0 2023-11-28T13:00:22.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.434+0000 7f46fdffb700 1 -- 172.21.15.153:0/1946760106 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286273 (secure 0 0 0) 0x7f470806f440 con 0x7f46ec0682f0 2023-11-28T13:00:22.439 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46ec0682f0 msgr2=0x7f46ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46ec0682f0 0x7f46ec06a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4704009770 tx=0x7f4704008040 comp rx=0 tx=0).stop 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708071040 msgr2=0x7f47080c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708071040 0x7f47080c6320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c450 tx=0x7f46f800c810 comp rx=0 tx=0).stop 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 shutdown_connections 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4708160540 0x7f4708159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470806dee0 0x7f470814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46ec0682f0 0x7f46ec06a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 --2- 172.21.15.153:0/1946760106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4708071040 0x7f47080c6320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 >> 172.21.15.153:0/1946760106 conn(0x7f47080d0000 msgr2=0x7f470815f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:22.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 shutdown_connections 2023-11-28T13:00:22.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.438+0000 7f470f9c4700 1 -- 172.21.15.153:0/1946760106 wait complete. 2023-11-28T13:00:22.443 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:00:22.451 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-28T13:00:22.136295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592132,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66188,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908532,"statfs":{"total":773094113280,"available":773026336768,"internally_reserved":0,"allocated":3223552,"data_stored":1911904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.910352"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217825+0000","last_change":"2023-11-28T13:00:17.635428+0000","last_active":"2023-11-28T13:00:19.217825+0000","last_peered":"2023-11-28T13:00:19.217825+0000","last_clean":"2023-11-28T13:00:19.217825+0000","last_became_active":"2023-11-28T13:00:17.635148+0000","last_became_peered":"2023-11-28T13:00:17.635148+0000","last_unstale":"2023-11-28T13:00:19.217825+0000","last_undegraded":"2023-11-28T13:00:19.217825+0000","last_fullsized":"2023-11-28T13:00:19.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648693+0000","last_change":"2023-11-28T13:00:17.645017+0000","last_active":"2023-11-28T13:00:17.648693+0000","last_peered":"2023-11-28T13:00:17.648693+0000","last_clean":"2023-11-28T13:00:17.648693+0000","last_became_active":"2023-11-28T13:00:17.644524+0000","last_became_peered":"2023-11-28T13:00:17.644524+0000","last_unstale":"2023-11-28T13:00:17.648693+0000","last_undegraded":"2023-11-28T13:00:17.648693+0000","last_fullsized":"2023-11-28T13:00:17.648693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656057+0000","last_change":"2023-11-28T13:00:17.648662+0000","last_active":"2023-11-28T13:00:17.656057+0000","last_peered":"2023-11-28T13:00:17.656057+0000","last_clean":"2023-11-28T13:00:17.656057+0000","last_became_active":"2023-11-28T13:00:17.648271+0000","last_became_peered":"2023-11-28T13:00:17.648271+0000","last_unstale":"2023-11-28T13:00:17.656057+0000","last_undegraded":"2023-11-28T13:00:17.656057+0000","last_fullsized":"2023-11-28T13:00:17.656057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:37.103258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647723+0000","last_change":"2023-11-28T13:00:17.643733+0000","last_active":"2023-11-28T13:00:17.647723+0000","last_peered":"2023-11-28T13:00:17.647723+0000","last_clean":"2023-11-28T13:00:17.647723+0000","last_became_active":"2023-11-28T13:00:17.643428+0000","last_became_peered":"2023-11-28T13:00:17.643428+0000","last_unstale":"2023-11-28T13:00:17.647723+0000","last_undegraded":"2023-11-28T13:00:17.647723+0000","last_fullsized":"2023-11-28T13:00:17.647723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233853+0000","last_change":"2023-11-28T13:00:17.233853+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233853+0000","last_undegraded":"2023-11-28T13:00:17.233853+0000","last_fullsized":"2023-11-28T13:00:17.233853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658806+0000","last_change":"2023-11-28T13:00:17.652679+0000","last_active":"2023-11-28T13:00:17.658806+0000","last_peered":"2023-11-28T13:00:17.658806+0000","last_clean":"2023-11-28T13:00:17.658806+0000","last_became_active":"2023-11-28T13:00:17.648169+0000","last_became_peered":"2023-11-28T13:00:17.648169+0000","last_unstale":"2023-11-28T13:00:17.658806+0000","last_undegraded":"2023-11-28T13:00:17.658806+0000","last_fullsized":"2023-11-28T13:00:17.658806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:58.762522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649103+0000","last_change":"2023-11-28T13:00:17.647769+0000","last_active":"2023-11-28T13:00:17.649103+0000","last_peered":"2023-11-28T13:00:17.649103+0000","last_clean":"2023-11-28T13:00:17.649103+0000","last_became_active":"2023-11-28T13:00:17.647476+0000","last_became_peered":"2023-11-28T13:00:17.647476+0000","last_unstale":"2023-11-28T13:00:17.649103+0000","last_undegraded":"2023-11-28T13:00:17.649103+0000","last_fullsized":"2023-11-28T13:00:17.649103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653892+0000","last_change":"2023-11-28T13:00:17.653233+0000","last_active":"2023-11-28T13:00:17.653892+0000","last_peered":"2023-11-28T13:00:17.653892+0000","last_clean":"2023-11-28T13:00:17.653892+0000","last_became_active":"2023-11-28T13:00:17.652957+0000","last_became_peered":"2023-11-28T13:00:17.652957+0000","last_unstale":"2023-11-28T13:00:17.653892+0000","last_undegraded":"2023-11-28T13:00:17.653892+0000","last_fullsized":"2023-11-28T13:00:17.653892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231803+0000","last_change":"2023-11-28T13:00:17.231803+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231803+0000","last_undegraded":"2023-11-28T13:00:17.231803+0000","last_fullsized":"2023-11-28T13:00:17.231803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645112+0000","last_change":"2023-11-28T13:00:17.642144+0000","last_active":"2023-11-28T13:00:17.645112+0000","last_peered":"2023-11-28T13:00:17.645112+0000","last_clean":"2023-11-28T13:00:17.645112+0000","last_became_active":"2023-11-28T13:00:17.641814+0000","last_became_peered":"2023-11-28T13:00:17.641814+0000","last_unstale":"2023-11-28T13:00:17.645112+0000","last_undegraded":"2023-11-28T13:00:17.645112+0000","last_fullsized":"2023-11-28T13:00:17.645112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653974+0000","last_change":"2023-11-28T13:00:17.643190+0000","last_active":"2023-11-28T13:00:17.653974+0000","last_peered":"2023-11-28T13:00:17.653974+0000","last_clean":"2023-11-28T13:00:17.653974+0000","last_became_active":"2023-11-28T13:00:17.642741+0000","last_became_peered":"2023-11-28T13:00:17.642741+0000","last_unstale":"2023-11-28T13:00:17.653974+0000","last_undegraded":"2023-11-28T13:00:17.653974+0000","last_fullsized":"2023-11-28T13:00:17.653974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:33.241895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217739+0000","last_change":"2023-11-28T13:00:17.654503+0000","last_active":"2023-11-28T13:00:19.217739+0000","last_peered":"2023-11-28T13:00:19.217739+0000","last_clean":"2023-11-28T13:00:19.217739+0000","last_became_active":"2023-11-28T13:00:17.654204+0000","last_became_peered":"2023-11-28T13:00:17.654204+0000","last_unstale":"2023-11-28T13:00:19.217739+0000","last_undegraded":"2023-11-28T13:00:19.217739+0000","last_fullsized":"2023-11-28T13:00:19.217739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:27.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217678+0000","last_change":"2023-11-28T13:00:17.634718+0000","last_active":"2023-11-28T13:00:19.217678+0000","last_peered":"2023-11-28T13:00:19.217678+0000","last_clean":"2023-11-28T13:00:19.217678+0000","last_became_active":"2023-11-28T13:00:17.634420+0000","last_became_peered":"2023-11-28T13:00:17.634420+0000","last_unstale":"2023-11-28T13:00:19.217678+0000","last_undegraded":"2023-11-28T13:00:19.217678+0000","last_fullsized":"2023-11-28T13:00:19.217678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.567102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645127+0000","last_change":"2023-11-28T13:00:17.641361+0000","last_active":"2023-11-28T13:00:17.645127+0000","last_peered":"2023-11-28T13:00:17.645127+0000","last_clean":"2023-11-28T13:00:17.645127+0000","last_became_active":"2023-11-28T13:00:17.641078+0000","last_became_peered":"2023-11-28T13:00:17.641078+0000","last_unstale":"2023-11-28T13:00:17.645127+0000","last_undegraded":"2023-11-28T13:00:17.645127+0000","last_fullsized":"2023-11-28T13:00:17.645127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:38.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649123+0000","last_change":"2023-11-28T13:00:17.647800+0000","last_active":"2023-11-28T13:00:17.649123+0000","last_peered":"2023-11-28T13:00:17.649123+0000","last_clean":"2023-11-28T13:00:17.649123+0000","last_became_active":"2023-11-28T13:00:17.647519+0000","last_became_peered":"2023-11-28T13:00:17.647519+0000","last_unstale":"2023-11-28T13:00:17.649123+0000","last_undegraded":"2023-11-28T13:00:17.649123+0000","last_fullsized":"2023-11-28T13:00:17.649123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656251+0000","last_change":"2023-11-28T13:00:17.651090+0000","last_active":"2023-11-28T13:00:17.656251+0000","last_peered":"2023-11-28T13:00:17.656251+0000","last_clean":"2023-11-28T13:00:17.656251+0000","last_became_active":"2023-11-28T13:00:17.649174+0000","last_became_peered":"2023-11-28T13:00:17.649174+0000","last_unstale":"2023-11-28T13:00:17.656251+0000","last_undegraded":"2023-11-28T13:00:17.656251+0000","last_fullsized":"2023-11-28T13:00:17.656251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:32.070272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231312+0000","last_change":"2023-11-28T13:00:17.231312+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231312+0000","last_undegraded":"2023-11-28T13:00:17.231312+0000","last_fullsized":"2023-11-28T13:00:17.231312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656166+0000","last_change":"2023-11-28T13:00:17.648771+0000","last_active":"2023-11-28T13:00:17.656166+0000","last_peered":"2023-11-28T13:00:17.656166+0000","last_clean":"2023-11-28T13:00:17.656166+0000","last_became_active":"2023-11-28T13:00:17.646367+0000","last_became_peered":"2023-11-28T13:00:17.646367+0000","last_unstale":"2023-11-28T13:00:17.656166+0000","last_undegraded":"2023-11-28T13:00:17.656166+0000","last_fullsized":"2023-11-28T13:00:17.656166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:33.549492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217793+0000","last_change":"2023-11-28T13:00:17.654191+0000","last_active":"2023-11-28T13:00:19.217793+0000","last_peered":"2023-11-28T13:00:19.217793+0000","last_clean":"2023-11-28T13:00:19.217793+0000","last_became_active":"2023-11-28T13:00:17.653922+0000","last_became_peered":"2023-11-28T13:00:17.653922+0000","last_unstale":"2023-11-28T13:00:19.217793+0000","last_undegraded":"2023-11-28T13:00:19.217793+0000","last_fullsized":"2023-11-28T13:00:19.217793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.066202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658937+0000","last_change":"2023-11-28T13:00:17.653882+0000","last_active":"2023-11-28T13:00:17.658937+0000","last_peered":"2023-11-28T13:00:17.658937+0000","last_clean":"2023-11-28T13:00:17.658937+0000","last_became_active":"2023-11-28T13:00:17.653497+0000","last_became_peered":"2023-11-28T13:00:17.653497+0000","last_unstale":"2023-11-28T13:00:17.658937+0000","last_undegraded":"2023-11-28T13:00:17.658937+0000","last_fullsized":"2023-11-28T13:00:17.658937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:59.591407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654270+0000","last_change":"2023-11-28T13:00:17.652105+0000","last_active":"2023-11-28T13:00:17.654270+0000","last_peered":"2023-11-28T13:00:17.654270+0000","last_clean":"2023-11-28T13:00:17.654270+0000","last_became_active":"2023-11-28T13:00:17.651804+0000","last_became_peered":"2023-11-28T13:00:17.651804+0000","last_unstale":"2023-11-28T13:00:17.654270+0000","last_undegraded":"2023-11-28T13:00:17.654270+0000","last_fullsized":"2023-11-28T13:00:17.654270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232480+0000","last_change":"2023-11-28T13:00:17.232480+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232480+0000","last_undegraded":"2023-11-28T13:00:17.232480+0000","last_fullsized":"2023-11-28T13:00:17.232480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215336+0000","last_change":"2023-11-28T13:00:17.645773+0000","last_active":"2023-11-28T13:00:19.215336+0000","last_peered":"2023-11-28T13:00:19.215336+0000","last_clean":"2023-11-28T13:00:19.215336+0000","last_became_active":"2023-11-28T13:00:17.643545+0000","last_became_peered":"2023-11-28T13:00:17.643545+0000","last_unstale":"2023-11-28T13:00:19.215336+0000","last_undegraded":"2023-11-28T13:00:19.215336+0000","last_fullsized":"2023-11-28T13:00:19.215336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:36.504863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654082+0000","last_change":"2023-11-28T13:00:17.651625+0000","last_active":"2023-11-28T13:00:17.654082+0000","last_peered":"2023-11-28T13:00:17.654082+0000","last_clean":"2023-11-28T13:00:17.654082+0000","last_became_active":"2023-11-28T13:00:17.651210+0000","last_became_peered":"2023-11-28T13:00:17.651210+0000","last_unstale":"2023-11-28T13:00:17.654082+0000","last_undegraded":"2023-11-28T13:00:17.654082+0000","last_fullsized":"2023-11-28T13:00:17.654082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635880+0000","last_change":"2023-11-28T13:00:17.646690+0000","last_active":"2023-11-28T13:00:18.635880+0000","last_peered":"2023-11-28T13:00:18.635880+0000","last_clean":"2023-11-28T13:00:18.635880+0000","last_became_active":"2023-11-28T13:00:17.646438+0000","last_became_peered":"2023-11-28T13:00:17.646438+0000","last_unstale":"2023-11-28T13:00:18.635880+0000","last_undegraded":"2023-11-28T13:00:18.635880+0000","last_fullsized":"2023-11-28T13:00:18.635880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:52.870009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636047+0000","last_change":"2023-11-28T13:00:17.641404+0000","last_active":"2023-11-28T13:00:18.636047+0000","last_peered":"2023-11-28T13:00:18.636047+0000","last_clean":"2023-11-28T13:00:18.636047+0000","last_became_active":"2023-11-28T13:00:17.641140+0000","last_became_peered":"2023-11-28T13:00:17.641140+0000","last_unstale":"2023-11-28T13:00:18.636047+0000","last_undegraded":"2023-11-28T13:00:18.636047+0000","last_fullsized":"2023-11-28T13:00:18.636047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648907+0000","last_change":"2023-11-28T13:00:17.645292+0000","last_active":"2023-11-28T13:00:17.648907+0000","last_peered":"2023-11-28T13:00:17.648907+0000","last_clean":"2023-11-28T13:00:17.648907+0000","last_became_active":"2023-11-28T13:00:17.645001+0000","last_became_peered":"2023-11-28T13:00:17.645001+0000","last_unstale":"2023-11-28T13:00:17.648907+0000","last_undegraded":"2023-11-28T13:00:17.648907+0000","last_fullsized":"2023-11-28T13:00:17.648907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652791+0000","last_change":"2023-11-28T13:00:17.245858+0000","last_active":"2023-11-28T13:00:17.652791+0000","last_peered":"2023-11-28T13:00:17.652791+0000","last_clean":"2023-11-28T13:00:17.652791+0000","last_became_active":"2023-11-28T13:00:17.245568+0000","last_became_peered":"2023-11-28T13:00:17.245568+0000","last_unstale":"2023-11-28T13:00:17.652791+0000","last_undegraded":"2023-11-28T13:00:17.652791+0000","last_fullsized":"2023-11-28T13:00:17.652791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215475+0000","last_change":"2023-11-28T13:00:17.644504+0000","last_active":"2023-11-28T13:00:19.215475+0000","last_peered":"2023-11-28T13:00:19.215475+0000","last_clean":"2023-11-28T13:00:19.215475+0000","last_became_active":"2023-11-28T13:00:17.644099+0000","last_became_peered":"2023-11-28T13:00:17.644099+0000","last_unstale":"2023-11-28T13:00:19.215475+0000","last_undegraded":"2023-11-28T13:00:19.215475+0000","last_fullsized":"2023-11-28T13:00:19.215475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635846+0000","last_change":"2023-11-28T13:00:17.236124+0000","last_active":"2023-11-28T13:00:18.635846+0000","last_peered":"2023-11-28T13:00:18.635846+0000","last_clean":"2023-11-28T13:00:18.635846+0000","last_became_active":"2023-11-28T13:00:17.235500+0000","last_became_peered":"2023-11-28T13:00:17.235500+0000","last_unstale":"2023-11-28T13:00:18.635846+0000","last_undegraded":"2023-11-28T13:00:18.635846+0000","last_fullsized":"2023-11-28T13:00:18.635846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:19.252999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.234634+0000","last_change":"2023-11-28T13:00:17.234634+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.234634+0000","last_undegraded":"2023-11-28T13:00:17.234634+0000","last_fullsized":"2023-11-28T13:00:17.234634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215390+0000","last_change":"2023-11-28T13:00:17.240445+0000","last_active":"2023-11-28T13:00:19.215390+0000","last_peered":"2023-11-28T13:00:19.215390+0000","last_clean":"2023-11-28T13:00:19.215390+0000","last_became_active":"2023-11-28T13:00:17.240095+0000","last_became_peered":"2023-11-28T13:00:17.240095+0000","last_unstale":"2023-11-28T13:00:19.215390+0000","last_undegraded":"2023-11-28T13:00:19.215390+0000","last_fullsized":"2023-11-28T13:00:19.215390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648526+0000","last_change":"2023-11-28T13:00:17.644329+0000","last_active":"2023-11-28T13:00:17.648526+0000","last_peered":"2023-11-28T13:00:17.648526+0000","last_clean":"2023-11-28T13:00:17.648526+0000","last_became_active":"2023-11-28T13:00:17.644046+0000","last_became_peered":"2023-11-28T13:00:17.644046+0000","last_unstale":"2023-11-28T13:00:17.648526+0000","last_undegraded":"2023-11-28T13:00:17.648526+0000","last_fullsized":"2023-11-28T13:00:17.648526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.849042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247832+0000","last_change":"2023-11-28T13:00:17.242514+0000","last_active":"2023-11-28T13:00:17.247832+0000","last_peered":"2023-11-28T13:00:17.247832+0000","last_clean":"2023-11-28T13:00:17.247832+0000","last_became_active":"2023-11-28T13:00:17.242258+0000","last_became_peered":"2023-11-28T13:00:17.242258+0000","last_unstale":"2023-11-28T13:00:17.247832+0000","last_undegraded":"2023-11-28T13:00:17.247832+0000","last_fullsized":"2023-11-28T13:00:17.247832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215295+0000","last_change":"2023-11-28T13:00:17.642442+0000","last_active":"2023-11-28T13:00:19.215295+0000","last_peered":"2023-11-28T13:00:19.215295+0000","last_clean":"2023-11-28T13:00:19.215295+0000","last_became_active":"2023-11-28T13:00:17.642162+0000","last_became_peered":"2023-11-28T13:00:17.642162+0000","last_unstale":"2023-11-28T13:00:19.215295+0000","last_undegraded":"2023-11-28T13:00:19.215295+0000","last_fullsized":"2023-11-28T13:00:19.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.635989+0000","last_change":"2023-11-28T13:00:17.238867+0000","last_active":"2023-11-28T13:00:17.635989+0000","last_peered":"2023-11-28T13:00:17.635989+0000","last_clean":"2023-11-28T13:00:17.635989+0000","last_became_active":"2023-11-28T13:00:17.238545+0000","last_became_peered":"2023-11-28T13:00:17.238545+0000","last_unstale":"2023-11-28T13:00:17.635989+0000","last_undegraded":"2023-11-28T13:00:17.635989+0000","last_fullsized":"2023-11-28T13:00:17.635989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215335+0000","last_change":"2023-11-28T13:00:17.645527+0000","last_active":"2023-11-28T13:00:19.215335+0000","last_peered":"2023-11-28T13:00:19.215335+0000","last_clean":"2023-11-28T13:00:19.215335+0000","last_became_active":"2023-11-28T13:00:17.644057+0000","last_became_peered":"2023-11-28T13:00:17.644057+0000","last_unstale":"2023-11-28T13:00:19.215335+0000","last_undegraded":"2023-11-28T13:00:19.215335+0000","last_fullsized":"2023-11-28T13:00:19.215335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:55.013485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651526+0000","last_change":"2023-11-28T13:00:17.242375+0000","last_active":"2023-11-28T13:00:17.651526+0000","last_peered":"2023-11-28T13:00:17.651526+0000","last_clean":"2023-11-28T13:00:17.651526+0000","last_became_active":"2023-11-28T13:00:17.242010+0000","last_became_peered":"2023-11-28T13:00:17.242010+0000","last_unstale":"2023-11-28T13:00:17.651526+0000","last_undegraded":"2023-11-28T13:00:17.651526+0000","last_fullsized":"2023-11-28T13:00:17.651526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.522601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635882+0000","last_change":"2023-11-28T13:00:17.647427+0000","last_active":"2023-11-28T13:00:18.635882+0000","last_peered":"2023-11-28T13:00:18.635882+0000","last_clean":"2023-11-28T13:00:18.635882+0000","last_became_active":"2023-11-28T13:00:17.646853+0000","last_became_peered":"2023-11-28T13:00:17.646853+0000","last_unstale":"2023-11-28T13:00:18.635882+0000","last_undegraded":"2023-11-28T13:00:18.635882+0000","last_fullsized":"2023-11-28T13:00:18.635882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.661600+0000","last_change":"2023-11-28T13:00:17.243823+0000","last_active":"2023-11-28T13:00:17.661600+0000","last_peered":"2023-11-28T13:00:17.661600+0000","last_clean":"2023-11-28T13:00:17.661600+0000","last_became_active":"2023-11-28T13:00:17.241238+0000","last_became_peered":"2023-11-28T13:00:17.241238+0000","last_unstale":"2023-11-28T13:00:17.661600+0000","last_undegraded":"2023-11-28T13:00:17.661600+0000","last_fullsized":"2023-11-28T13:00:17.661600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656243+0000","last_change":"2023-11-28T13:00:17.649986+0000","last_active":"2023-11-28T13:00:17.656243+0000","last_peered":"2023-11-28T13:00:17.656243+0000","last_clean":"2023-11-28T13:00:17.656243+0000","last_became_active":"2023-11-28T13:00:17.649402+0000","last_became_peered":"2023-11-28T13:00:17.649402+0000","last_unstale":"2023-11-28T13:00:17.656243+0000","last_undegraded":"2023-11-28T13:00:17.656243+0000","last_fullsized":"2023-11-28T13:00:17.656243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:15.471790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656508+0000","last_change":"2023-11-28T13:00:17.247047+0000","last_active":"2023-11-28T13:00:17.656508+0000","last_peered":"2023-11-28T13:00:17.656508+0000","last_clean":"2023-11-28T13:00:17.656508+0000","last_became_active":"2023-11-28T13:00:17.246742+0000","last_became_peered":"2023-11-28T13:00:17.246742+0000","last_unstale":"2023-11-28T13:00:17.656508+0000","last_undegraded":"2023-11-28T13:00:17.656508+0000","last_fullsized":"2023-11-28T13:00:17.656508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:50.559956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215527+0000","last_change":"2023-11-28T13:00:17.641605+0000","last_active":"2023-11-28T13:00:19.215527+0000","last_peered":"2023-11-28T13:00:19.215527+0000","last_clean":"2023-11-28T13:00:19.215527+0000","last_became_active":"2023-11-28T13:00:17.641350+0000","last_became_peered":"2023-11-28T13:00:17.641350+0000","last_unstale":"2023-11-28T13:00:19.215527+0000","last_undegraded":"2023-11-28T13:00:19.215527+0000","last_fullsized":"2023-11-28T13:00:19.215527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245165+0000","last_change":"2023-11-28T13:00:17.240432+0000","last_active":"2023-11-28T13:00:17.245165+0000","last_peered":"2023-11-28T13:00:17.245165+0000","last_clean":"2023-11-28T13:00:17.245165+0000","last_became_active":"2023-11-28T13:00:17.239192+0000","last_became_peered":"2023-11-28T13:00:17.239192+0000","last_unstale":"2023-11-28T13:00:17.245165+0000","last_undegraded":"2023-11-28T13:00:17.245165+0000","last_fullsized":"2023-11-28T13:00:17.245165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651770+0000","last_change":"2023-11-28T13:00:17.648774+0000","last_active":"2023-11-28T13:00:17.651770+0000","last_peered":"2023-11-28T13:00:17.651770+0000","last_clean":"2023-11-28T13:00:17.651770+0000","last_became_active":"2023-11-28T13:00:17.647403+0000","last_became_peered":"2023-11-28T13:00:17.647403+0000","last_unstale":"2023-11-28T13:00:17.651770+0000","last_undegraded":"2023-11-28T13:00:17.651770+0000","last_fullsized":"2023-11-28T13:00:17.651770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:17.592744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633096+0000","last_change":"2023-11-28T13:00:17.245977+0000","last_active":"2023-11-28T13:00:17.633096+0000","last_peered":"2023-11-28T13:00:17.633096+0000","last_clean":"2023-11-28T13:00:17.633096+0000","last_became_active":"2023-11-28T13:00:17.245651+0000","last_became_peered":"2023-11-28T13:00:17.245651+0000","last_unstale":"2023-11-28T13:00:17.633096+0000","last_undegraded":"2023-11-28T13:00:17.633096+0000","last_fullsized":"2023-11-28T13:00:17.633096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:30.760573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.235374+0000","last_change":"2023-11-28T13:00:17.235374+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.235374+0000","last_undegraded":"2023-11-28T13:00:17.235374+0000","last_fullsized":"2023-11-28T13:00:17.235374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650468+0000","last_change":"2023-11-28T13:00:17.240716+0000","last_active":"2023-11-28T13:00:17.650468+0000","last_peered":"2023-11-28T13:00:17.650468+0000","last_clean":"2023-11-28T13:00:17.650468+0000","last_became_active":"2023-11-28T13:00:17.239928+0000","last_became_peered":"2023-11-28T13:00:17.239928+0000","last_unstale":"2023-11-28T13:00:17.650468+0000","last_undegraded":"2023-11-28T13:00:17.650468+0000","last_fullsized":"2023-11-28T13:00:17.650468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:32.086174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215680+0000","last_change":"2023-11-28T13:00:17.643923+0000","last_active":"2023-11-28T13:00:19.215680+0000","last_peered":"2023-11-28T13:00:19.215680+0000","last_clean":"2023-11-28T13:00:19.215680+0000","last_became_active":"2023-11-28T13:00:17.643579+0000","last_became_peered":"2023-11-28T13:00:17.643579+0000","last_unstale":"2023-11-28T13:00:19.215680+0000","last_undegraded":"2023-11-28T13:00:19.215680+0000","last_fullsized":"2023-11-28T13:00:19.215680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654531+0000","last_change":"2023-11-28T13:00:17.247259+0000","last_active":"2023-11-28T13:00:17.654531+0000","last_peered":"2023-11-28T13:00:17.654531+0000","last_clean":"2023-11-28T13:00:17.654531+0000","last_became_active":"2023-11-28T13:00:17.246943+0000","last_became_peered":"2023-11-28T13:00:17.246943+0000","last_unstale":"2023-11-28T13:00:17.654531+0000","last_undegraded":"2023-11-28T13:00:17.654531+0000","last_fullsized":"2023-11-28T13:00:17.654531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:02.643388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647850+0000","last_change":"2023-11-28T13:00:17.644322+0000","last_active":"2023-11-28T13:00:17.647850+0000","last_peered":"2023-11-28T13:00:17.647850+0000","last_clean":"2023-11-28T13:00:17.647850+0000","last_became_active":"2023-11-28T13:00:17.644054+0000","last_became_peered":"2023-11-28T13:00:17.644054+0000","last_unstale":"2023-11-28T13:00:17.647850+0000","last_undegraded":"2023-11-28T13:00:17.647850+0000","last_fullsized":"2023-11-28T13:00:17.647850+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:46.098419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656474+0000","last_change":"2023-11-28T13:00:17.247274+0000","last_active":"2023-11-28T13:00:17.656474+0000","last_peered":"2023-11-28T13:00:17.656474+0000","last_clean":"2023-11-28T13:00:17.656474+0000","last_became_active":"2023-11-28T13:00:17.246851+0000","last_became_peered":"2023-11-28T13:00:17.246851+0000","last_unstale":"2023-11-28T13:00:17.656474+0000","last_undegraded":"2023-11-28T13:00:17.656474+0000","last_fullsized":"2023-11-28T13:00:17.656474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:17.621082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652526+0000","last_change":"2023-11-28T13:00:17.640395+0000","last_active":"2023-11-28T13:00:17.652526+0000","last_peered":"2023-11-28T13:00:17.652526+0000","last_clean":"2023-11-28T13:00:17.652526+0000","last_became_active":"2023-11-28T13:00:17.640195+0000","last_became_peered":"2023-11-28T13:00:17.640195+0000","last_unstale":"2023-11-28T13:00:17.652526+0000","last_undegraded":"2023-11-28T13:00:17.652526+0000","last_fullsized":"2023-11-28T13:00:17.652526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:14.878366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.646780+0000","last_change":"2023-11-28T13:00:17.240698+0000","last_active":"2023-11-28T13:00:17.646780+0000","last_peered":"2023-11-28T13:00:17.646780+0000","last_clean":"2023-11-28T13:00:17.646780+0000","last_became_active":"2023-11-28T13:00:17.240387+0000","last_became_peered":"2023-11-28T13:00:17.240387+0000","last_unstale":"2023-11-28T13:00:17.646780+0000","last_undegraded":"2023-11-28T13:00:17.646780+0000","last_fullsized":"2023-11-28T13:00:17.646780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:05.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648509+0000","last_change":"2023-11-28T13:00:17.647318+0000","last_active":"2023-11-28T13:00:17.648509+0000","last_peered":"2023-11-28T13:00:17.648509+0000","last_clean":"2023-11-28T13:00:17.648509+0000","last_became_active":"2023-11-28T13:00:17.647043+0000","last_became_peered":"2023-11-28T13:00:17.647043+0000","last_unstale":"2023-11-28T13:00:17.648509+0000","last_undegraded":"2023-11-28T13:00:17.648509+0000","last_fullsized":"2023-11-28T13:00:17.648509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:38.321116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634419+0000","last_change":"2023-11-28T13:00:17.235843+0000","last_active":"2023-11-28T13:00:17.634419+0000","last_peered":"2023-11-28T13:00:17.634419+0000","last_clean":"2023-11-28T13:00:17.634419+0000","last_became_active":"2023-11-28T13:00:17.235495+0000","last_became_peered":"2023-11-28T13:00:17.235495+0000","last_unstale":"2023-11-28T13:00:17.634419+0000","last_undegraded":"2023-11-28T13:00:17.634419+0000","last_fullsized":"2023-11-28T13:00:17.634419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232454+0000","last_change":"2023-11-28T13:00:17.232454+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232454+0000","last_undegraded":"2023-11-28T13:00:17.232454+0000","last_fullsized":"2023-11-28T13:00:17.232454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215456+0000","last_change":"2023-11-28T13:00:17.240709+0000","last_active":"2023-11-28T13:00:19.215456+0000","last_peered":"2023-11-28T13:00:19.215456+0000","last_clean":"2023-11-28T13:00:19.215456+0000","last_became_active":"2023-11-28T13:00:17.239893+0000","last_became_peered":"2023-11-28T13:00:17.239893+0000","last_unstale":"2023-11-28T13:00:19.215456+0000","last_undegraded":"2023-11-28T13:00:19.215456+0000","last_fullsized":"2023-11-28T13:00:19.215456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658839+0000","last_change":"2023-11-28T13:00:17.653235+0000","last_active":"2023-11-28T13:00:17.658839+0000","last_peered":"2023-11-28T13:00:17.658839+0000","last_clean":"2023-11-28T13:00:17.658839+0000","last_became_active":"2023-11-28T13:00:17.650249+0000","last_became_peered":"2023-11-28T13:00:17.650249+0000","last_unstale":"2023-11-28T13:00:17.658839+0000","last_undegraded":"2023-11-28T13:00:17.658839+0000","last_fullsized":"2023-11-28T13:00:17.658839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631372+0000","last_change":"2023-11-28T13:00:17.245224+0000","last_active":"2023-11-28T13:00:17.631372+0000","last_peered":"2023-11-28T13:00:17.631372+0000","last_clean":"2023-11-28T13:00:17.631372+0000","last_became_active":"2023-11-28T13:00:17.244913+0000","last_became_peered":"2023-11-28T13:00:17.244913+0000","last_unstale":"2023-11-28T13:00:17.631372+0000","last_undegraded":"2023-11-28T13:00:17.631372+0000","last_fullsized":"2023-11-28T13:00:17.631372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653954+0000","last_change":"2023-11-28T13:00:17.643015+0000","last_active":"2023-11-28T13:00:17.653954+0000","last_peered":"2023-11-28T13:00:17.653954+0000","last_clean":"2023-11-28T13:00:17.653954+0000","last_became_active":"2023-11-28T13:00:17.642470+0000","last_became_peered":"2023-11-28T13:00:17.642470+0000","last_unstale":"2023-11-28T13:00:17.653954+0000","last_undegraded":"2023-11-28T13:00:17.653954+0000","last_fullsized":"2023-11-28T13:00:17.653954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:41.679867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.639452+0000","last_change":"2023-11-28T13:00:17.244207+0000","last_active":"2023-11-28T13:00:17.639452+0000","last_peered":"2023-11-28T13:00:17.639452+0000","last_clean":"2023-11-28T13:00:17.639452+0000","last_became_active":"2023-11-28T13:00:17.243861+0000","last_became_peered":"2023-11-28T13:00:17.243861+0000","last_unstale":"2023-11-28T13:00:17.639452+0000","last_undegraded":"2023-11-28T13:00:17.639452+0000","last_fullsized":"2023-11-28T13:00:17.639452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656262+0000","last_change":"2023-11-28T13:00:17.651269+0000","last_active":"2023-11-28T13:00:17.656262+0000","last_peered":"2023-11-28T13:00:17.656262+0000","last_clean":"2023-11-28T13:00:17.656262+0000","last_became_active":"2023-11-28T13:00:17.650883+0000","last_became_peered":"2023-11-28T13:00:17.650883+0000","last_unstale":"2023-11-28T13:00:17.656262+0000","last_undegraded":"2023-11-28T13:00:17.656262+0000","last_fullsized":"2023-11-28T13:00:17.656262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215191+0000","last_change":"2023-11-28T13:00:17.238617+0000","last_active":"2023-11-28T13:00:19.215191+0000","last_peered":"2023-11-28T13:00:19.215191+0000","last_clean":"2023-11-28T13:00:19.215191+0000","last_became_active":"2023-11-28T13:00:17.238344+0000","last_became_peered":"2023-11-28T13:00:17.238344+0000","last_unstale":"2023-11-28T13:00:19.215191+0000","last_undegraded":"2023-11-28T13:00:19.215191+0000","last_fullsized":"2023-11-28T13:00:19.215191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:46.225565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636017+0000","last_change":"2023-11-28T13:00:17.649564+0000","last_active":"2023-11-28T13:00:18.636017+0000","last_peered":"2023-11-28T13:00:18.636017+0000","last_clean":"2023-11-28T13:00:18.636017+0000","last_became_active":"2023-11-28T13:00:17.649312+0000","last_became_peered":"2023-11-28T13:00:17.649312+0000","last_unstale":"2023-11-28T13:00:18.636017+0000","last_undegraded":"2023-11-28T13:00:18.636017+0000","last_fullsized":"2023-11-28T13:00:18.636017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634764+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:00:17.634764+0000","last_peered":"2023-11-28T13:00:17.634764+0000","last_clean":"2023-11-28T13:00:17.634764+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:00:17.634764+0000","last_undegraded":"2023-11-28T13:00:17.634764+0000","last_fullsized":"2023-11-28T13:00:17.634764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.636391+0000","last_change":"2023-11-28T13:00:17.240503+0000","last_active":"2023-11-28T13:00:17.636391+0000","last_peered":"2023-11-28T13:00:17.636391+0000","last_clean":"2023-11-28T13:00:17.636391+0000","last_became_active":"2023-11-28T13:00:17.240256+0000","last_became_peered":"2023-11-28T13:00:17.240256+0000","last_unstale":"2023-11-28T13:00:17.636391+0000","last_undegraded":"2023-11-28T13:00:17.636391+0000","last_fullsized":"2023-11-28T13:00:17.636391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635946+0000","last_change":"2023-11-28T13:00:17.645964+0000","last_active":"2023-11-28T13:00:18.635946+0000","last_peered":"2023-11-28T13:00:18.635946+0000","last_clean":"2023-11-28T13:00:18.635946+0000","last_became_active":"2023-11-28T13:00:17.645384+0000","last_became_peered":"2023-11-28T13:00:17.645384+0000","last_unstale":"2023-11-28T13:00:18.635946+0000","last_undegraded":"2023-11-28T13:00:18.635946+0000","last_fullsized":"2023-11-28T13:00:18.635946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234232+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:00:17.234232+0000","last_peered":"2023-11-28T13:00:17.234232+0000","last_clean":"2023-11-28T13:00:17.234232+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:00:17.234232+0000","last_undegraded":"2023-11-28T13:00:17.234232+0000","last_fullsized":"2023-11-28T13:00:17.234232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.640096+0000","last_change":"2023-11-28T13:00:17.245773+0000","last_active":"2023-11-28T13:00:17.640096+0000","last_peered":"2023-11-28T13:00:17.640096+0000","last_clean":"2023-11-28T13:00:17.640096+0000","last_became_active":"2023-11-28T13:00:17.245409+0000","last_became_peered":"2023-11-28T13:00:17.245409+0000","last_unstale":"2023-11-28T13:00:17.640096+0000","last_undegraded":"2023-11-28T13:00:17.640096+0000","last_fullsized":"2023-11-28T13:00:17.640096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:00.519142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636216+0000","last_change":"2023-11-28T13:00:17.641395+0000","last_active":"2023-11-28T13:00:18.636216+0000","last_peered":"2023-11-28T13:00:18.636216+0000","last_clean":"2023-11-28T13:00:18.636216+0000","last_became_active":"2023-11-28T13:00:17.641149+0000","last_became_peered":"2023-11-28T13:00:17.641149+0000","last_unstale":"2023-11-28T13:00:18.636216+0000","last_undegraded":"2023-11-28T13:00:18.636216+0000","last_fullsized":"2023-11-28T13:00:18.636216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.628022+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:00:17.628022+0000","last_peered":"2023-11-28T13:00:17.628022+0000","last_clean":"2023-11-28T13:00:17.628022+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:00:17.628022+0000","last_undegraded":"2023-11-28T13:00:17.628022+0000","last_fullsized":"2023-11-28T13:00:17.628022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635610+0000","last_change":"2023-11-28T13:00:17.237802+0000","last_active":"2023-11-28T13:00:18.635610+0000","last_peered":"2023-11-28T13:00:18.635610+0000","last_clean":"2023-11-28T13:00:18.635610+0000","last_became_active":"2023-11-28T13:00:17.237487+0000","last_became_peered":"2023-11-28T13:00:17.237487+0000","last_unstale":"2023-11-28T13:00:18.635610+0000","last_undegraded":"2023-11-28T13:00:18.635610+0000","last_fullsized":"2023-11-28T13:00:18.635610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635877+0000","last_change":"2023-11-28T13:00:17.644511+0000","last_active":"2023-11-28T13:00:18.635877+0000","last_peered":"2023-11-28T13:00:18.635877+0000","last_clean":"2023-11-28T13:00:18.635877+0000","last_became_active":"2023-11-28T13:00:17.644095+0000","last_became_peered":"2023-11-28T13:00:17.644095+0000","last_unstale":"2023-11-28T13:00:18.635877+0000","last_undegraded":"2023-11-28T13:00:18.635877+0000","last_fullsized":"2023-11-28T13:00:18.635877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234538+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:00:17.234538+0000","last_peered":"2023-11-28T13:00:17.234538+0000","last_clean":"2023-11-28T13:00:17.234538+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:00:17.234538+0000","last_undegraded":"2023-11-28T13:00:17.234538+0000","last_fullsized":"2023-11-28T13:00:17.234538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235309+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:00:17.235309+0000","last_peered":"2023-11-28T13:00:17.235309+0000","last_clean":"2023-11-28T13:00:17.235309+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:00:17.235309+0000","last_undegraded":"2023-11-28T13:00:17.235309+0000","last_fullsized":"2023-11-28T13:00:17.235309+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215311+0000","last_change":"2023-11-28T13:00:17.239373+0000","last_active":"2023-11-28T13:00:19.215311+0000","last_peered":"2023-11-28T13:00:19.215311+0000","last_clean":"2023-11-28T13:00:19.215311+0000","last_became_active":"2023-11-28T13:00:17.238864+0000","last_became_peered":"2023-11-28T13:00:17.238864+0000","last_unstale":"2023-11-28T13:00:19.215311+0000","last_undegraded":"2023-11-28T13:00:19.215311+0000","last_fullsized":"2023-11-28T13:00:19.215311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:17.763933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653117+0000","last_change":"2023-11-28T13:00:17.649771+0000","last_active":"2023-11-28T13:00:17.653117+0000","last_peered":"2023-11-28T13:00:17.653117+0000","last_clean":"2023-11-28T13:00:17.653117+0000","last_became_active":"2023-11-28T13:00:17.649475+0000","last_became_peered":"2023-11-28T13:00:17.649475+0000","last_unstale":"2023-11-28T13:00:17.653117+0000","last_undegraded":"2023-11-28T13:00:17.653117+0000","last_fullsized":"2023-11-28T13:00:17.653117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235498+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:00:17.235498+0000","last_peered":"2023-11-28T13:00:17.235498+0000","last_clean":"2023-11-28T13:00:17.235498+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:00:17.235498+0000","last_undegraded":"2023-11-28T13:00:17.235498+0000","last_fullsized":"2023-11-28T13:00:17.235498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253107+0000","last_change":"2023-11-28T13:00:17.251262+0000","last_active":"2023-11-28T13:00:17.253107+0000","last_peered":"2023-11-28T13:00:17.253107+0000","last_clean":"2023-11-28T13:00:17.253107+0000","last_became_active":"2023-11-28T13:00:17.250948+0000","last_became_peered":"2023-11-28T13:00:17.250948+0000","last_unstale":"2023-11-28T13:00:17.253107+0000","last_undegraded":"2023-11-28T13:00:17.253107+0000","last_fullsized":"2023-11-28T13:00:17.253107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:37.311866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233919+0000","last_change":"2023-11-28T13:00:17.233919+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233919+0000","last_undegraded":"2023-11-28T13:00:17.233919+0000","last_fullsized":"2023-11-28T13:00:17.233919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.236149+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:00:17.236149+0000","last_peered":"2023-11-28T13:00:17.236149+0000","last_clean":"2023-11-28T13:00:17.236149+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:00:17.236149+0000","last_undegraded":"2023-11-28T13:00:17.236149+0000","last_fullsized":"2023-11-28T13:00:17.236149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636224+0000","last_change":"2023-11-28T13:00:17.237999+0000","last_active":"2023-11-28T13:00:18.636224+0000","last_peered":"2023-11-28T13:00:18.636224+0000","last_clean":"2023-11-28T13:00:18.636224+0000","last_became_active":"2023-11-28T13:00:17.237625+0000","last_became_peered":"2023-11-28T13:00:17.237625+0000","last_unstale":"2023-11-28T13:00:18.636224+0000","last_undegraded":"2023-11-28T13:00:18.636224+0000","last_fullsized":"2023-11-28T13:00:18.636224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233381+0000","last_change":"2023-11-28T13:00:17.233381+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233381+0000","last_undegraded":"2023-11-28T13:00:17.233381+0000","last_fullsized":"2023-11-28T13:00:17.233381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215660+0000","last_change":"2023-11-28T13:00:17.239284+0000","last_active":"2023-11-28T13:00:19.215660+0000","last_peered":"2023-11-28T13:00:19.215660+0000","last_clean":"2023-11-28T13:00:19.215660+0000","last_became_active":"2023-11-28T13:00:17.238997+0000","last_became_peered":"2023-11-28T13:00:17.238997+0000","last_unstale":"2023-11-28T13:00:19.215660+0000","last_undegraded":"2023-11-28T13:00:19.215660+0000","last_fullsized":"2023-11-28T13:00:19.215660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:55.423382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633198+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:00:17.633198+0000","last_peered":"2023-11-28T13:00:17.633198+0000","last_clean":"2023-11-28T13:00:17.633198+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:00:17.633198+0000","last_undegraded":"2023-11-28T13:00:17.633198+0000","last_fullsized":"2023-11-28T13:00:17.633198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215108+0000","last_change":"2023-11-28T13:00:17.646149+0000","last_active":"2023-11-28T13:00:19.215108+0000","last_peered":"2023-11-28T13:00:19.215108+0000","last_clean":"2023-11-28T13:00:19.215108+0000","last_became_active":"2023-11-28T13:00:17.645893+0000","last_became_peered":"2023-11-28T13:00:17.645893+0000","last_unstale":"2023-11-28T13:00:19.215108+0000","last_undegraded":"2023-11-28T13:00:19.215108+0000","last_fullsized":"2023-11-28T13:00:19.215108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:24.466207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.226618+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:00:17.226618+0000","last_peered":"2023-11-28T13:00:17.226618+0000","last_clean":"2023-11-28T13:00:17.226618+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:00:17.226618+0000","last_undegraded":"2023-11-28T13:00:17.226618+0000","last_fullsized":"2023-11-28T13:00:17.226618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245166+0000","last_change":"2023-11-28T13:00:17.240379+0000","last_active":"2023-11-28T13:00:17.245166+0000","last_peered":"2023-11-28T13:00:17.245166+0000","last_clean":"2023-11-28T13:00:17.245166+0000","last_became_active":"2023-11-28T13:00:17.239505+0000","last_became_peered":"2023-11-28T13:00:17.239505+0000","last_unstale":"2023-11-28T13:00:17.245166+0000","last_undegraded":"2023-11-28T13:00:17.245166+0000","last_fullsized":"2023-11-28T13:00:17.245166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655975+0000","last_change":"2023-11-28T13:00:17.646003+0000","last_active":"2023-11-28T13:00:17.655975+0000","last_peered":"2023-11-28T13:00:17.655975+0000","last_clean":"2023-11-28T13:00:17.655975+0000","last_became_active":"2023-11-28T13:00:17.645727+0000","last_became_peered":"2023-11-28T13:00:17.645727+0000","last_unstale":"2023-11-28T13:00:17.655975+0000","last_undegraded":"2023-11-28T13:00:17.655975+0000","last_fullsized":"2023-11-28T13:00:17.655975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:30.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253113+0000","last_change":"2023-11-28T13:00:17.251398+0000","last_active":"2023-11-28T13:00:17.253113+0000","last_peered":"2023-11-28T13:00:17.253113+0000","last_clean":"2023-11-28T13:00:17.253113+0000","last_became_active":"2023-11-28T13:00:17.251054+0000","last_became_peered":"2023-11-28T13:00:17.251054+0000","last_unstale":"2023-11-28T13:00:17.253113+0000","last_undegraded":"2023-11-28T13:00:17.253113+0000","last_fullsized":"2023-11-28T13:00:17.253113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217848+0000","last_change":"2023-11-28T13:00:17.641355+0000","last_active":"2023-11-28T13:00:19.217848+0000","last_peered":"2023-11-28T13:00:19.217848+0000","last_clean":"2023-11-28T13:00:19.217848+0000","last_became_active":"2023-11-28T13:00:17.641093+0000","last_became_peered":"2023-11-28T13:00:17.641093+0000","last_unstale":"2023-11-28T13:00:19.217848+0000","last_undegraded":"2023-11-28T13:00:19.217848+0000","last_fullsized":"2023-11-28T13:00:19.217848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:41.816686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655479+0000","last_change":"2023-11-28T13:00:17.246896+0000","last_active":"2023-11-28T13:00:17.655479+0000","last_peered":"2023-11-28T13:00:17.655479+0000","last_clean":"2023-11-28T13:00:17.655479+0000","last_became_active":"2023-11-28T13:00:17.246604+0000","last_became_peered":"2023-11-28T13:00:17.246604+0000","last_unstale":"2023-11-28T13:00:17.655479+0000","last_undegraded":"2023-11-28T13:00:17.655479+0000","last_fullsized":"2023-11-28T13:00:17.655479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656231+0000","last_change":"2023-11-28T13:00:17.648973+0000","last_active":"2023-11-28T13:00:17.656231+0000","last_peered":"2023-11-28T13:00:17.656231+0000","last_clean":"2023-11-28T13:00:17.656231+0000","last_became_active":"2023-11-28T13:00:17.648323+0000","last_became_peered":"2023-11-28T13:00:17.648323+0000","last_unstale":"2023-11-28T13:00:17.656231+0000","last_undegraded":"2023-11-28T13:00:17.656231+0000","last_fullsized":"2023-11-28T13:00:17.656231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654699+0000","last_change":"2023-11-28T13:00:17.247013+0000","last_active":"2023-11-28T13:00:17.654699+0000","last_peered":"2023-11-28T13:00:17.654699+0000","last_clean":"2023-11-28T13:00:17.654699+0000","last_became_active":"2023-11-28T13:00:17.246645+0000","last_became_peered":"2023-11-28T13:00:17.246645+0000","last_unstale":"2023-11-28T13:00:17.654699+0000","last_undegraded":"2023-11-28T13:00:17.654699+0000","last_fullsized":"2023-11-28T13:00:17.654699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656116+0000","last_change":"2023-11-28T13:00:17.649078+0000","last_active":"2023-11-28T13:00:17.656116+0000","last_peered":"2023-11-28T13:00:17.656116+0000","last_clean":"2023-11-28T13:00:17.656116+0000","last_became_active":"2023-11-28T13:00:17.648432+0000","last_became_peered":"2023-11-28T13:00:17.648432+0000","last_unstale":"2023-11-28T13:00:17.656116+0000","last_undegraded":"2023-11-28T13:00:17.656116+0000","last_fullsized":"2023-11-28T13:00:17.656116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215495+0000","last_change":"2023-11-28T13:00:17.234074+0000","last_active":"2023-11-28T13:00:19.215495+0000","last_peered":"2023-11-28T13:00:19.215495+0000","last_clean":"2023-11-28T13:00:19.215495+0000","last_became_active":"2023-11-28T13:00:17.233732+0000","last_became_peered":"2023-11-28T13:00:17.233732+0000","last_unstale":"2023-11-28T13:00:19.215495+0000","last_undegraded":"2023-11-28T13:00:19.215495+0000","last_fullsized":"2023-11-28T13:00:19.215495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:36.950864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215231+0000","last_change":"2023-11-28T13:00:17.641105+0000","last_active":"2023-11-28T13:00:19.215231+0000","last_peered":"2023-11-28T13:00:19.215231+0000","last_clean":"2023-11-28T13:00:19.215231+0000","last_became_active":"2023-11-28T13:00:17.640854+0000","last_became_peered":"2023-11-28T13:00:17.640854+0000","last_unstale":"2023-11-28T13:00:19.215231+0000","last_undegraded":"2023-11-28T13:00:19.215231+0000","last_fullsized":"2023-11-28T13:00:19.215231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247814+0000","last_change":"2023-11-28T13:00:17.242508+0000","last_active":"2023-11-28T13:00:17.247814+0000","last_peered":"2023-11-28T13:00:17.247814+0000","last_clean":"2023-11-28T13:00:17.247814+0000","last_became_active":"2023-11-28T13:00:17.242230+0000","last_became_peered":"2023-11-28T13:00:17.242230+0000","last_unstale":"2023-11-28T13:00:17.247814+0000","last_undegraded":"2023-11-28T13:00:17.247814+0000","last_fullsized":"2023-11-28T13:00:17.247814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656224+0000","last_change":"2023-11-28T13:00:17.650868+0000","last_active":"2023-11-28T13:00:17.656224+0000","last_peered":"2023-11-28T13:00:17.656224+0000","last_clean":"2023-11-28T13:00:17.656224+0000","last_became_active":"2023-11-28T13:00:17.649842+0000","last_became_peered":"2023-11-28T13:00:17.649842+0000","last_unstale":"2023-11-28T13:00:17.656224+0000","last_undegraded":"2023-11-28T13:00:17.656224+0000","last_fullsized":"2023-11-28T13:00:17.656224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645743+0000","last_change":"2023-11-28T13:00:17.641330+0000","last_active":"2023-11-28T13:00:17.645743+0000","last_peered":"2023-11-28T13:00:17.645743+0000","last_clean":"2023-11-28T13:00:17.645743+0000","last_became_active":"2023-11-28T13:00:17.641029+0000","last_became_peered":"2023-11-28T13:00:17.641029+0000","last_unstale":"2023-11-28T13:00:17.645743+0000","last_undegraded":"2023-11-28T13:00:17.645743+0000","last_fullsized":"2023-11-28T13:00:17.645743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:28.607875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215202+0000","last_change":"2023-11-28T13:00:17.643816+0000","last_active":"2023-11-28T13:00:19.215202+0000","last_peered":"2023-11-28T13:00:19.215202+0000","last_clean":"2023-11-28T13:00:19.215202+0000","last_became_active":"2023-11-28T13:00:17.643260+0000","last_became_peered":"2023-11-28T13:00:17.643260+0000","last_unstale":"2023-11-28T13:00:19.215202+0000","last_undegraded":"2023-11-28T13:00:19.215202+0000","last_fullsized":"2023-11-28T13:00:19.215202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656140+0000","last_change":"2023-11-28T13:00:17.649431+0000","last_active":"2023-11-28T13:00:17.656140+0000","last_peered":"2023-11-28T13:00:17.656140+0000","last_clean":"2023-11-28T13:00:17.656140+0000","last_became_active":"2023-11-28T13:00:17.648763+0000","last_became_peered":"2023-11-28T13:00:17.648763+0000","last_unstale":"2023-11-28T13:00:17.656140+0000","last_undegraded":"2023-11-28T13:00:17.656140+0000","last_fullsized":"2023-11-28T13:00:17.656140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:10.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215472+0000","last_change":"2023-11-28T13:00:17.649148+0000","last_active":"2023-11-28T13:00:19.215472+0000","last_peered":"2023-11-28T13:00:19.215472+0000","last_clean":"2023-11-28T13:00:19.215472+0000","last_became_active":"2023-11-28T13:00:17.648925+0000","last_became_peered":"2023-11-28T13:00:17.648925+0000","last_unstale":"2023-11-28T13:00:19.215472+0000","last_undegraded":"2023-11-28T13:00:19.215472+0000","last_fullsized":"2023-11-28T13:00:19.215472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636138+0000","last_change":"2023-11-28T13:00:17.648409+0000","last_active":"2023-11-28T13:00:18.636138+0000","last_peered":"2023-11-28T13:00:18.636138+0000","last_clean":"2023-11-28T13:00:18.636138+0000","last_became_active":"2023-11-28T13:00:17.648164+0000","last_became_peered":"2023-11-28T13:00:17.648164+0000","last_unstale":"2023-11-28T13:00:18.636138+0000","last_undegraded":"2023-11-28T13:00:18.636138+0000","last_fullsized":"2023-11-28T13:00:18.636138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":96947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7940,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363900,"statfs":{"total":96636764160,"available":96628633600,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":91808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8928,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362912,"statfs":{"total":96636764160,"available":96627621888,"internally_reserved":0,"allocated":811008,"data_stored":666055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9112,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362728,"statfs":{"total":96636764160,"available":96627433472,"internally_reserved":0,"allocated":868352,"data_stored":682336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8000,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363840,"statfs":{"total":96636764160,"available":96628572160,"internally_reserved":0,"allocated":253952,"data_stored":90840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7992,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363848,"statfs":{"total":96636764160,"available":96628580352,"internally_reserved":0,"allocated":245760,"data_stored":90204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8068,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363772,"statfs":{"total":96636764160,"available":96628502528,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:22.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2023-11-28T13:00:22.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.233853+0000', 'last_change': '2023-11-28T13:00:17.233853+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.233853+0000', 'last_undegraded': '2023-11-28T13:00:17.233853+0000', 'last_fullsized': '2023-11-28T13:00:17.233853+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2023-11-28T13:00:22.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.231803+0000', 'last_change': '2023-11-28T13:00:17.231803+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.231803+0000', 'last_undegraded': '2023-11-28T13:00:17.231803+0000', 'last_fullsized': '2023-11-28T13:00:17.231803+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2023-11-28T13:00:22.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.231312+0000', 'last_change': '2023-11-28T13:00:17.231312+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.231312+0000', 'last_undegraded': '2023-11-28T13:00:17.231312+0000', 'last_fullsized': '2023-11-28T13:00:17.231312+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.232480+0000', 'last_change': '2023-11-28T13:00:17.232480+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.232480+0000', 'last_undegraded': '2023-11-28T13:00:17.232480+0000', 'last_fullsized': '2023-11-28T13:00:17.232480+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.234634+0000', 'last_change': '2023-11-28T13:00:17.234634+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.234634+0000', 'last_undegraded': '2023-11-28T13:00:17.234634+0000', 'last_fullsized': '2023-11-28T13:00:17.234634+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.235374+0000', 'last_change': '2023-11-28T13:00:17.235374+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.235374+0000', 'last_undegraded': '2023-11-28T13:00:17.235374+0000', 'last_fullsized': '2023-11-28T13:00:17.235374+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.232454+0000', 'last_change': '2023-11-28T13:00:17.232454+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.232454+0000', 'last_undegraded': '2023-11-28T13:00:17.232454+0000', 'last_fullsized': '2023-11-28T13:00:17.232454+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-11-28T13:00:22.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.233919+0000', 'last_change': '2023-11-28T13:00:17.233919+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.233919+0000', 'last_undegraded': '2023-11-28T13:00:17.233919+0000', 'last_fullsized': '2023-11-28T13:00:17.233919+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-11-28T13:00:22.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:00:17.233381+0000', 'last_change': '2023-11-28T13:00:17.233381+0000', 'last_active': '2023-11-28T13:00:17.215040+0000', 'last_peered': '2023-11-28T13:00:17.215040+0000', 'last_clean': '2023-11-28T13:00:17.215040+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:00:17.233381+0000', 'last_undegraded': '2023-11-28T13:00:17.233381+0000', 'last_fullsized': '2023-11-28T13:00:17.233381+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'last_clean_scrub_stamp': '2023-11-28T13:00:17.215040+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:00:22.458 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T13:00:22.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.609+0000 7f754e6e8700 1 Processor -- start 2023-11-28T13:00:22.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.609+0000 7f754e6e8700 1 -- start start 2023-11-28T13:00:22.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754e6e8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f75480cd8c0 0x7f7548140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754e6e8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75480b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754e6e8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f75480b2e70 0x7f754806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754e6e8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754813ff00 con 0x7f75480b2e70 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754e6e8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75481404c0 con 0x7f75480cd8c0 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754e6e8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754815aa80 con 0x7f75480cdc90 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754cc85700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f75480b2e70 0x7f754806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754cc85700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f75480b2e70 0x7f754806f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38364/0 (socket says 172.21.15.153:38364) 2023-11-28T13:00:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.610+0000 7f754cc85700 1 -- 172.21.15.153:0/3321520853 learned_addr learned my addr 172.21.15.153:0/3321520853 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f7547fff700 1 --2- 172.21.15.153:0/3321520853 >> v2:172.21.15.177:3300/0 conn(0x7f75480cd8c0 0x7f7548140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754cc85700 1 -- 172.21.15.153:0/3321520853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 msgr2=0x7f75480b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f75477fe700 1 --2- 172.21.15.153:0/3321520853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75480b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754cc85700 1 --2- 172.21.15.153:0/3321520853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75480b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754cc85700 1 -- 172.21.15.153:0/3321520853 >> v2:172.21.15.177:3300/0 conn(0x7f75480cd8c0 msgr2=0x7f7548140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754cc85700 1 --2- 172.21.15.153:0/3321520853 >> v2:172.21.15.177:3300/0 conn(0x7f75480cd8c0 0x7f7548140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754cc85700 1 -- 172.21.15.153:0/3321520853 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480b3310 con 0x7f75480b2e70 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f7547fff700 1 --2- 172.21.15.153:0/3321520853 >> v2:172.21.15.177:3300/0 conn(0x7f75480cd8c0 0x7f7548140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754cc85700 1 --2- 172.21.15.153:0/3321520853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f754806f850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7548070a20 tx=0x7f754000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef055360d138845a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f7546ffd700 1 -- 172.21.15.153:0/3321520853 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754000e070 con 0x7f75480b2e70 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f7546ffd700 1 -- 172.21.15.153:0/3321520853 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754000b9c0 con 0x7f75480b2e70 2023-11-28T13:00:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f7546ffd700 1 -- 172.21.15.153:0/3321520853 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754000bd70 con 0x7f75480b2e70 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754e6e8700 1 -- 172.21.15.153:0/3321520853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 msgr2=0x7f754806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.611+0000 7f754e6e8700 1 --2- 172.21.15.153:0/3321520853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f754806f850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7548070a20 tx=0x7f754000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.612+0000 7f754e6e8700 1 -- 172.21.15.153:0/3321520853 shutdown_connections 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.612+0000 7f754e6e8700 1 --2- 172.21.15.153:0/3321520853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f754806f850 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.612+0000 7f754e6e8700 1 --2- 172.21.15.153:0/3321520853 >> v2:172.21.15.177:3300/0 conn(0x7f75480cd8c0 0x7f7548140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.612+0000 7f754e6e8700 1 --2- 172.21.15.153:0/3321520853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75480b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.612+0000 7f754e6e8700 1 -- 172.21.15.153:0/3321520853 >> 172.21.15.153:0/3321520853 conn(0x7f754806aed0 msgr2=0x7f754806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.612+0000 7f754e6e8700 1 -- 172.21.15.153:0/3321520853 shutdown_connections 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.612+0000 7f754e6e8700 1 -- 172.21.15.153:0/3321520853 wait complete. 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.613+0000 7f754e6e8700 1 Processor -- start 2023-11-28T13:00:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.613+0000 7f754e6e8700 1 -- start start 2023-11-28T13:00:22.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f75480b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75480cd8c0 0x7f75480b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75481f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75480b5f50 con 0x7f75480b2e70 2023-11-28T13:00:22.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75480b6230 con 0x7f75480cd8c0 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75480b6510 con 0x7f75480cdc90 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f75480b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f75480b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38366/0 (socket says 172.21.15.153:38366) 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 -- 172.21.15.153:0/2867848505 learned_addr learned my addr 172.21.15.153:0/2867848505 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f75477fe700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75480cd8c0 0x7f75480b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754cc85700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75481f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 -- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 msgr2=0x7f75481f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75481f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 -- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75480cd8c0 msgr2=0x7f75480b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75480cd8c0 0x7f75480b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 -- 172.21.15.153:0/2867848505 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7540009b30 con 0x7f75480b2e70 2023-11-28T13:00:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7547fff700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f75480b5950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f753800c7f0 tx=0x7f753800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7544ff9700 1 -- 172.21.15.153:0/2867848505 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538005c20 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7544ff9700 1 -- 172.21.15.153:0/2867848505 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7538005d80 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7548140150 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f7544ff9700 1 -- 172.21.15.153:0/2867848505 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538011820 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.614+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75480b35f0 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.615+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75480ce100 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.616+0000 7f7544ff9700 1 -- 172.21.15.153:0/2867848505 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7538020070 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.616+0000 7f7544ff9700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75340682f0 0x7f753406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.616+0000 7f7544ff9700 1 -- 172.21.15.153:0/2867848505 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7538085e10 con 0x7f75480b2e70 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.616+0000 7f75477fe700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75340682f0 0x7f753406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.617+0000 7f75477fe700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75340682f0 0x7f753406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f754806ee20 tx=0x7f753c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.620+0000 7f7544ff9700 1 -- 172.21.15.153:0/2867848505 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7538089030 con 0x7f75480b2e70 2023-11-28T13:00:22.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.877+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f754806b380 con 0x7f75480b2e70 2023-11-28T13:00:22.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.877+0000 7f7544ff9700 1 -- 172.21.15.153:0/2867848505 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1096 (secure 0 0 0) 0x7f7538055fb0 con 0x7f75480b2e70 2023-11-28T13:00:22.879 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:22.879 INFO:teuthology.orchestra.run.smithi153.stdout:{"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":25,"num_osds":8,"num_up_osds":8,"osd_up_since":1701176403,"num_in_osds":8,"osd_in_since":1701176395,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":96},{"state_name":"creating+peering","count":9}],"num_pgs":105,"num_pools":4,"num_objects":21,"data_bytes":592132,"bytes_used":67776512,"bytes_avail":773026336768,"bytes_total":773094113280,"inactive_pgs_ratio":0.085714288055896759,"write_bytes_sec":2252,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4300}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-11-28T12:59:52.562980+0000","services":{}},"progress_events":{}} 2023-11-28T13:00:22.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75340682f0 msgr2=0x7f753406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75340682f0 0x7f753406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f754806ee20 tx=0x7f753c008040 comp rx=0 tx=0).stop 2023-11-28T13:00:22.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 msgr2=0x7f75480b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f75480b5950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f753800c7f0 tx=0x7f753800cb00 comp rx=0 tx=0).stop 2023-11-28T13:00:22.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 shutdown_connections 2023-11-28T13:00:22.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f75340682f0 0x7f753406a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f75480b2e70 0x7f75480b5950 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75480cdc90 0x7f75481f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 --2- 172.21.15.153:0/2867848505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75480cd8c0 0x7f75480b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 >> 172.21.15.153:0/2867848505 conn(0x7f754806aed0 msgr2=0x7f75480cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:22.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 shutdown_connections 2023-11-28T13:00:22.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:22.879+0000 7f754e6e8700 1 -- 172.21.15.153:0/2867848505 wait complete. 2023-11-28T13:00:22.893 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T13:00:22.893 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:23.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.012+0000 7fe9f7b37700 1 Processor -- start 2023-11-28T13:00:23.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.012+0000 7fe9f7b37700 1 -- start start 2023-11-28T13:00:23.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.013+0000 7fe9f7b37700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe9f0071040 0x7fe9f0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:23.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.013+0000 7fe9f7b37700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f0071950 con 0x7fe9f006dee0 2023-11-28T13:00:23.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.013+0000 7fe9f7b37700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f015d060 con 0x7fe9f0160540 2023-11-28T13:00:23.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.013+0000 7fe9f7b37700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f015d340 con 0x7fe9f0071040 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.013+0000 7fe9f58d3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe9f0071040 0x7fe9f0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.013+0000 7fe9f50d2700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe9f006dee0 0x7fe9f006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56538/0 (socket says 172.21.15.153:56538) 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.013+0000 7fe9f50d2700 1 -- 172.21.15.153:0/4163605846 learned_addr learned my addr 172.21.15.153:0/4163605846 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f58d3700 1 -- 172.21.15.153:0/4163605846 >> v1:172.21.15.177:6789/0 conn(0x7fe9f0160540 legacy=0x7fe9f0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f58d3700 1 -- 172.21.15.153:0/4163605846 >> v1:172.21.15.153:6789/0 conn(0x7fe9f006dee0 legacy=0x7fe9f006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f58d3700 1 -- 172.21.15.153:0/4163605846 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f015d620 con 0x7fe9f0071040 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f48d1700 1 -- 172.21.15.153:0/4163605846 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529900189 0 0) 0x7fe9f0071950 con 0x7fe9f006dee0 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f58d3700 1 --2- 172.21.15.153:0/4163605846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f0071040 0x7fe9f0071410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009840 tx=0x7fe9e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548e38f1b841a949 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f48d1700 1 -- 172.21.15.153:0/4163605846 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0004230 con 0x7fe9f0071040 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f48d1700 1 -- 172.21.15.153:0/4163605846 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e0004390 con 0x7fe9f0071040 2023-11-28T13:00:23.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f48d1700 1 -- 172.21.15.153:0/4163605846 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e00036a0 con 0x7fe9f0071040 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f7b37700 1 -- 172.21.15.153:0/4163605846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f0071040 msgr2=0x7fe9f0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.014+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/4163605846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f0071040 0x7fe9f0071410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009840 tx=0x7fe9e0009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.015+0000 7fe9f7b37700 1 -- 172.21.15.153:0/4163605846 shutdown_connections 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.015+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/4163605846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f0071040 0x7fe9f0071410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.015+0000 7fe9f7b37700 1 -- 172.21.15.153:0/4163605846 >> 172.21.15.153:0/4163605846 conn(0x7fe9f00d0000 msgr2=0x7fe9f00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.015+0000 7fe9f7b37700 1 -- 172.21.15.153:0/4163605846 shutdown_connections 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.015+0000 7fe9f7b37700 1 -- 172.21.15.153:0/4163605846 wait complete. 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.016+0000 7fe9f7b37700 1 Processor -- start 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.016+0000 7fe9f7b37700 1 -- start start 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f7b37700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f006dee0 0x7fe9f01f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f7b37700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f0071040 0x7fe9f00b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f7b37700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f0160540 0x7fe9f00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f7b37700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01f6c10 con 0x7fe9f006dee0 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f7b37700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00b4340 con 0x7fe9f0160540 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f7b37700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00b32a0 con 0x7fe9f0071040 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f006dee0 0x7fe9f01f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f006dee0 0x7fe9f01f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38380/0 (socket says 172.21.15.153:38380) 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 -- 172.21.15.153:0/3731322865 learned_addr learned my addr 172.21.15.153:0/3731322865 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 -- 172.21.15.153:0/3731322865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f0071040 msgr2=0x7fe9f00b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f0071040 0x7fe9f00b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f60d4700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f0160540 0x7fe9f00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:23.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 -- 172.21.15.153:0/3731322865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f0160540 msgr2=0x7fe9f00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f0160540 0x7fe9f00b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.017+0000 7fe9f58d3700 1 -- 172.21.15.153:0/3731322865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00094a0 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.018+0000 7fe9f58d3700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f006dee0 0x7fe9f01f66d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009b90 tx=0x7fe9e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.018+0000 7fe9e67fc700 1 -- 172.21.15.153:0/3731322865 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0011630 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.018+0000 7fe9e67fc700 1 -- 172.21.15.153:0/3731322865 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e0011790 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.018+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f015ccd0 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.018+0000 7fe9e67fc700 1 -- 172.21.15.153:0/3731322865 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0011b20 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.018+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f015d900 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.019+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f006a270 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.019+0000 7fe9e67fc700 1 -- 172.21.15.153:0/3731322865 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe9e0017020 con 0x7fe9f006dee0 2023-11-28T13:00:23.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.019+0000 7fe9e67fc700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9d40682f0 0x7fe9d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:23.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.020+0000 7fe9e67fc700 1 -- 172.21.15.153:0/3731322865 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9e0087480 con 0x7fe9f006dee0 2023-11-28T13:00:23.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.020+0000 7fe9f50d2700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9d40682f0 0x7fe9d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:23.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.020+0000 7fe9f50d2700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9d40682f0 0x7fe9d406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0158580 tx=0x7fe9ec007a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:23.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.024+0000 7fe9e67fc700 1 -- 172.21.15.153:0/3731322865 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe9e0053ce0 con 0x7fe9f006dee0 2023-11-28T13:00:23.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.204+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9f00720a0 con 0x7fe9d40682f0 2023-11-28T13:00:23.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.211+0000 7fe9e67fc700 1 -- 172.21.15.153:0/3731322865 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286273 (secure 0 0 0) 0x7fe9f00720a0 con 0x7fe9d40682f0 2023-11-28T13:00:23.216 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9d40682f0 msgr2=0x7fe9d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9d40682f0 0x7fe9d406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0158580 tx=0x7fe9ec007a10 comp rx=0 tx=0).stop 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f006dee0 msgr2=0x7fe9f01f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f006dee0 0x7fe9f01f66d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009b90 tx=0x7fe9e00047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 shutdown_connections 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9d40682f0 0x7fe9d406a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe9f006dee0 0x7fe9f01f66d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f0160540 0x7fe9f00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 --2- 172.21.15.153:0/3731322865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f0071040 0x7fe9f00b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:23.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 >> 172.21.15.153:0/3731322865 conn(0x7fe9f00d0000 msgr2=0x7fe9f015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:23.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 shutdown_connections 2023-11-28T13:00:23.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:23.215+0000 7fe9f7b37700 1 -- 172.21.15.153:0/3731322865 wait complete. 2023-11-28T13:00:23.220 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:00:23.228 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-11-28T13:00:22.136295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592132,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66188,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908532,"statfs":{"total":773094113280,"available":773026336768,"internally_reserved":0,"allocated":3223552,"data_stored":1911904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.910352"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217825+0000","last_change":"2023-11-28T13:00:17.635428+0000","last_active":"2023-11-28T13:00:19.217825+0000","last_peered":"2023-11-28T13:00:19.217825+0000","last_clean":"2023-11-28T13:00:19.217825+0000","last_became_active":"2023-11-28T13:00:17.635148+0000","last_became_peered":"2023-11-28T13:00:17.635148+0000","last_unstale":"2023-11-28T13:00:19.217825+0000","last_undegraded":"2023-11-28T13:00:19.217825+0000","last_fullsized":"2023-11-28T13:00:19.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648693+0000","last_change":"2023-11-28T13:00:17.645017+0000","last_active":"2023-11-28T13:00:17.648693+0000","last_peered":"2023-11-28T13:00:17.648693+0000","last_clean":"2023-11-28T13:00:17.648693+0000","last_became_active":"2023-11-28T13:00:17.644524+0000","last_became_peered":"2023-11-28T13:00:17.644524+0000","last_unstale":"2023-11-28T13:00:17.648693+0000","last_undegraded":"2023-11-28T13:00:17.648693+0000","last_fullsized":"2023-11-28T13:00:17.648693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656057+0000","last_change":"2023-11-28T13:00:17.648662+0000","last_active":"2023-11-28T13:00:17.656057+0000","last_peered":"2023-11-28T13:00:17.656057+0000","last_clean":"2023-11-28T13:00:17.656057+0000","last_became_active":"2023-11-28T13:00:17.648271+0000","last_became_peered":"2023-11-28T13:00:17.648271+0000","last_unstale":"2023-11-28T13:00:17.656057+0000","last_undegraded":"2023-11-28T13:00:17.656057+0000","last_fullsized":"2023-11-28T13:00:17.656057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:37.103258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647723+0000","last_change":"2023-11-28T13:00:17.643733+0000","last_active":"2023-11-28T13:00:17.647723+0000","last_peered":"2023-11-28T13:00:17.647723+0000","last_clean":"2023-11-28T13:00:17.647723+0000","last_became_active":"2023-11-28T13:00:17.643428+0000","last_became_peered":"2023-11-28T13:00:17.643428+0000","last_unstale":"2023-11-28T13:00:17.647723+0000","last_undegraded":"2023-11-28T13:00:17.647723+0000","last_fullsized":"2023-11-28T13:00:17.647723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233853+0000","last_change":"2023-11-28T13:00:17.233853+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233853+0000","last_undegraded":"2023-11-28T13:00:17.233853+0000","last_fullsized":"2023-11-28T13:00:17.233853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658806+0000","last_change":"2023-11-28T13:00:17.652679+0000","last_active":"2023-11-28T13:00:17.658806+0000","last_peered":"2023-11-28T13:00:17.658806+0000","last_clean":"2023-11-28T13:00:17.658806+0000","last_became_active":"2023-11-28T13:00:17.648169+0000","last_became_peered":"2023-11-28T13:00:17.648169+0000","last_unstale":"2023-11-28T13:00:17.658806+0000","last_undegraded":"2023-11-28T13:00:17.658806+0000","last_fullsized":"2023-11-28T13:00:17.658806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:58.762522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649103+0000","last_change":"2023-11-28T13:00:17.647769+0000","last_active":"2023-11-28T13:00:17.649103+0000","last_peered":"2023-11-28T13:00:17.649103+0000","last_clean":"2023-11-28T13:00:17.649103+0000","last_became_active":"2023-11-28T13:00:17.647476+0000","last_became_peered":"2023-11-28T13:00:17.647476+0000","last_unstale":"2023-11-28T13:00:17.649103+0000","last_undegraded":"2023-11-28T13:00:17.649103+0000","last_fullsized":"2023-11-28T13:00:17.649103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653892+0000","last_change":"2023-11-28T13:00:17.653233+0000","last_active":"2023-11-28T13:00:17.653892+0000","last_peered":"2023-11-28T13:00:17.653892+0000","last_clean":"2023-11-28T13:00:17.653892+0000","last_became_active":"2023-11-28T13:00:17.652957+0000","last_became_peered":"2023-11-28T13:00:17.652957+0000","last_unstale":"2023-11-28T13:00:17.653892+0000","last_undegraded":"2023-11-28T13:00:17.653892+0000","last_fullsized":"2023-11-28T13:00:17.653892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231803+0000","last_change":"2023-11-28T13:00:17.231803+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231803+0000","last_undegraded":"2023-11-28T13:00:17.231803+0000","last_fullsized":"2023-11-28T13:00:17.231803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645112+0000","last_change":"2023-11-28T13:00:17.642144+0000","last_active":"2023-11-28T13:00:17.645112+0000","last_peered":"2023-11-28T13:00:17.645112+0000","last_clean":"2023-11-28T13:00:17.645112+0000","last_became_active":"2023-11-28T13:00:17.641814+0000","last_became_peered":"2023-11-28T13:00:17.641814+0000","last_unstale":"2023-11-28T13:00:17.645112+0000","last_undegraded":"2023-11-28T13:00:17.645112+0000","last_fullsized":"2023-11-28T13:00:17.645112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653974+0000","last_change":"2023-11-28T13:00:17.643190+0000","last_active":"2023-11-28T13:00:17.653974+0000","last_peered":"2023-11-28T13:00:17.653974+0000","last_clean":"2023-11-28T13:00:17.653974+0000","last_became_active":"2023-11-28T13:00:17.642741+0000","last_became_peered":"2023-11-28T13:00:17.642741+0000","last_unstale":"2023-11-28T13:00:17.653974+0000","last_undegraded":"2023-11-28T13:00:17.653974+0000","last_fullsized":"2023-11-28T13:00:17.653974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:33.241895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217739+0000","last_change":"2023-11-28T13:00:17.654503+0000","last_active":"2023-11-28T13:00:19.217739+0000","last_peered":"2023-11-28T13:00:19.217739+0000","last_clean":"2023-11-28T13:00:19.217739+0000","last_became_active":"2023-11-28T13:00:17.654204+0000","last_became_peered":"2023-11-28T13:00:17.654204+0000","last_unstale":"2023-11-28T13:00:19.217739+0000","last_undegraded":"2023-11-28T13:00:19.217739+0000","last_fullsized":"2023-11-28T13:00:19.217739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:27.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217678+0000","last_change":"2023-11-28T13:00:17.634718+0000","last_active":"2023-11-28T13:00:19.217678+0000","last_peered":"2023-11-28T13:00:19.217678+0000","last_clean":"2023-11-28T13:00:19.217678+0000","last_became_active":"2023-11-28T13:00:17.634420+0000","last_became_peered":"2023-11-28T13:00:17.634420+0000","last_unstale":"2023-11-28T13:00:19.217678+0000","last_undegraded":"2023-11-28T13:00:19.217678+0000","last_fullsized":"2023-11-28T13:00:19.217678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.567102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645127+0000","last_change":"2023-11-28T13:00:17.641361+0000","last_active":"2023-11-28T13:00:17.645127+0000","last_peered":"2023-11-28T13:00:17.645127+0000","last_clean":"2023-11-28T13:00:17.645127+0000","last_became_active":"2023-11-28T13:00:17.641078+0000","last_became_peered":"2023-11-28T13:00:17.641078+0000","last_unstale":"2023-11-28T13:00:17.645127+0000","last_undegraded":"2023-11-28T13:00:17.645127+0000","last_fullsized":"2023-11-28T13:00:17.645127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:38.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649123+0000","last_change":"2023-11-28T13:00:17.647800+0000","last_active":"2023-11-28T13:00:17.649123+0000","last_peered":"2023-11-28T13:00:17.649123+0000","last_clean":"2023-11-28T13:00:17.649123+0000","last_became_active":"2023-11-28T13:00:17.647519+0000","last_became_peered":"2023-11-28T13:00:17.647519+0000","last_unstale":"2023-11-28T13:00:17.649123+0000","last_undegraded":"2023-11-28T13:00:17.649123+0000","last_fullsized":"2023-11-28T13:00:17.649123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656251+0000","last_change":"2023-11-28T13:00:17.651090+0000","last_active":"2023-11-28T13:00:17.656251+0000","last_peered":"2023-11-28T13:00:17.656251+0000","last_clean":"2023-11-28T13:00:17.656251+0000","last_became_active":"2023-11-28T13:00:17.649174+0000","last_became_peered":"2023-11-28T13:00:17.649174+0000","last_unstale":"2023-11-28T13:00:17.656251+0000","last_undegraded":"2023-11-28T13:00:17.656251+0000","last_fullsized":"2023-11-28T13:00:17.656251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:32.070272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.231312+0000","last_change":"2023-11-28T13:00:17.231312+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.231312+0000","last_undegraded":"2023-11-28T13:00:17.231312+0000","last_fullsized":"2023-11-28T13:00:17.231312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656166+0000","last_change":"2023-11-28T13:00:17.648771+0000","last_active":"2023-11-28T13:00:17.656166+0000","last_peered":"2023-11-28T13:00:17.656166+0000","last_clean":"2023-11-28T13:00:17.656166+0000","last_became_active":"2023-11-28T13:00:17.646367+0000","last_became_peered":"2023-11-28T13:00:17.646367+0000","last_unstale":"2023-11-28T13:00:17.656166+0000","last_undegraded":"2023-11-28T13:00:17.656166+0000","last_fullsized":"2023-11-28T13:00:17.656166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:33.549492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217793+0000","last_change":"2023-11-28T13:00:17.654191+0000","last_active":"2023-11-28T13:00:19.217793+0000","last_peered":"2023-11-28T13:00:19.217793+0000","last_clean":"2023-11-28T13:00:19.217793+0000","last_became_active":"2023-11-28T13:00:17.653922+0000","last_became_peered":"2023-11-28T13:00:17.653922+0000","last_unstale":"2023-11-28T13:00:19.217793+0000","last_undegraded":"2023-11-28T13:00:19.217793+0000","last_fullsized":"2023-11-28T13:00:19.217793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.066202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658937+0000","last_change":"2023-11-28T13:00:17.653882+0000","last_active":"2023-11-28T13:00:17.658937+0000","last_peered":"2023-11-28T13:00:17.658937+0000","last_clean":"2023-11-28T13:00:17.658937+0000","last_became_active":"2023-11-28T13:00:17.653497+0000","last_became_peered":"2023-11-28T13:00:17.653497+0000","last_unstale":"2023-11-28T13:00:17.658937+0000","last_undegraded":"2023-11-28T13:00:17.658937+0000","last_fullsized":"2023-11-28T13:00:17.658937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:59.591407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654270+0000","last_change":"2023-11-28T13:00:17.652105+0000","last_active":"2023-11-28T13:00:17.654270+0000","last_peered":"2023-11-28T13:00:17.654270+0000","last_clean":"2023-11-28T13:00:17.654270+0000","last_became_active":"2023-11-28T13:00:17.651804+0000","last_became_peered":"2023-11-28T13:00:17.651804+0000","last_unstale":"2023-11-28T13:00:17.654270+0000","last_undegraded":"2023-11-28T13:00:17.654270+0000","last_fullsized":"2023-11-28T13:00:17.654270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232480+0000","last_change":"2023-11-28T13:00:17.232480+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232480+0000","last_undegraded":"2023-11-28T13:00:17.232480+0000","last_fullsized":"2023-11-28T13:00:17.232480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215336+0000","last_change":"2023-11-28T13:00:17.645773+0000","last_active":"2023-11-28T13:00:19.215336+0000","last_peered":"2023-11-28T13:00:19.215336+0000","last_clean":"2023-11-28T13:00:19.215336+0000","last_became_active":"2023-11-28T13:00:17.643545+0000","last_became_peered":"2023-11-28T13:00:17.643545+0000","last_unstale":"2023-11-28T13:00:19.215336+0000","last_undegraded":"2023-11-28T13:00:19.215336+0000","last_fullsized":"2023-11-28T13:00:19.215336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:36.504863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654082+0000","last_change":"2023-11-28T13:00:17.651625+0000","last_active":"2023-11-28T13:00:17.654082+0000","last_peered":"2023-11-28T13:00:17.654082+0000","last_clean":"2023-11-28T13:00:17.654082+0000","last_became_active":"2023-11-28T13:00:17.651210+0000","last_became_peered":"2023-11-28T13:00:17.651210+0000","last_unstale":"2023-11-28T13:00:17.654082+0000","last_undegraded":"2023-11-28T13:00:17.654082+0000","last_fullsized":"2023-11-28T13:00:17.654082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635880+0000","last_change":"2023-11-28T13:00:17.646690+0000","last_active":"2023-11-28T13:00:18.635880+0000","last_peered":"2023-11-28T13:00:18.635880+0000","last_clean":"2023-11-28T13:00:18.635880+0000","last_became_active":"2023-11-28T13:00:17.646438+0000","last_became_peered":"2023-11-28T13:00:17.646438+0000","last_unstale":"2023-11-28T13:00:18.635880+0000","last_undegraded":"2023-11-28T13:00:18.635880+0000","last_fullsized":"2023-11-28T13:00:18.635880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:52.870009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636047+0000","last_change":"2023-11-28T13:00:17.641404+0000","last_active":"2023-11-28T13:00:18.636047+0000","last_peered":"2023-11-28T13:00:18.636047+0000","last_clean":"2023-11-28T13:00:18.636047+0000","last_became_active":"2023-11-28T13:00:17.641140+0000","last_became_peered":"2023-11-28T13:00:17.641140+0000","last_unstale":"2023-11-28T13:00:18.636047+0000","last_undegraded":"2023-11-28T13:00:18.636047+0000","last_fullsized":"2023-11-28T13:00:18.636047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648907+0000","last_change":"2023-11-28T13:00:17.645292+0000","last_active":"2023-11-28T13:00:17.648907+0000","last_peered":"2023-11-28T13:00:17.648907+0000","last_clean":"2023-11-28T13:00:17.648907+0000","last_became_active":"2023-11-28T13:00:17.645001+0000","last_became_peered":"2023-11-28T13:00:17.645001+0000","last_unstale":"2023-11-28T13:00:17.648907+0000","last_undegraded":"2023-11-28T13:00:17.648907+0000","last_fullsized":"2023-11-28T13:00:17.648907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652791+0000","last_change":"2023-11-28T13:00:17.245858+0000","last_active":"2023-11-28T13:00:17.652791+0000","last_peered":"2023-11-28T13:00:17.652791+0000","last_clean":"2023-11-28T13:00:17.652791+0000","last_became_active":"2023-11-28T13:00:17.245568+0000","last_became_peered":"2023-11-28T13:00:17.245568+0000","last_unstale":"2023-11-28T13:00:17.652791+0000","last_undegraded":"2023-11-28T13:00:17.652791+0000","last_fullsized":"2023-11-28T13:00:17.652791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215475+0000","last_change":"2023-11-28T13:00:17.644504+0000","last_active":"2023-11-28T13:00:19.215475+0000","last_peered":"2023-11-28T13:00:19.215475+0000","last_clean":"2023-11-28T13:00:19.215475+0000","last_became_active":"2023-11-28T13:00:17.644099+0000","last_became_peered":"2023-11-28T13:00:17.644099+0000","last_unstale":"2023-11-28T13:00:19.215475+0000","last_undegraded":"2023-11-28T13:00:19.215475+0000","last_fullsized":"2023-11-28T13:00:19.215475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635846+0000","last_change":"2023-11-28T13:00:17.236124+0000","last_active":"2023-11-28T13:00:18.635846+0000","last_peered":"2023-11-28T13:00:18.635846+0000","last_clean":"2023-11-28T13:00:18.635846+0000","last_became_active":"2023-11-28T13:00:17.235500+0000","last_became_peered":"2023-11-28T13:00:17.235500+0000","last_unstale":"2023-11-28T13:00:18.635846+0000","last_undegraded":"2023-11-28T13:00:18.635846+0000","last_fullsized":"2023-11-28T13:00:18.635846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:19.252999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.234634+0000","last_change":"2023-11-28T13:00:17.234634+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.234634+0000","last_undegraded":"2023-11-28T13:00:17.234634+0000","last_fullsized":"2023-11-28T13:00:17.234634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215390+0000","last_change":"2023-11-28T13:00:17.240445+0000","last_active":"2023-11-28T13:00:19.215390+0000","last_peered":"2023-11-28T13:00:19.215390+0000","last_clean":"2023-11-28T13:00:19.215390+0000","last_became_active":"2023-11-28T13:00:17.240095+0000","last_became_peered":"2023-11-28T13:00:17.240095+0000","last_unstale":"2023-11-28T13:00:19.215390+0000","last_undegraded":"2023-11-28T13:00:19.215390+0000","last_fullsized":"2023-11-28T13:00:19.215390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648526+0000","last_change":"2023-11-28T13:00:17.644329+0000","last_active":"2023-11-28T13:00:17.648526+0000","last_peered":"2023-11-28T13:00:17.648526+0000","last_clean":"2023-11-28T13:00:17.648526+0000","last_became_active":"2023-11-28T13:00:17.644046+0000","last_became_peered":"2023-11-28T13:00:17.644046+0000","last_unstale":"2023-11-28T13:00:17.648526+0000","last_undegraded":"2023-11-28T13:00:17.648526+0000","last_fullsized":"2023-11-28T13:00:17.648526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.849042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247832+0000","last_change":"2023-11-28T13:00:17.242514+0000","last_active":"2023-11-28T13:00:17.247832+0000","last_peered":"2023-11-28T13:00:17.247832+0000","last_clean":"2023-11-28T13:00:17.247832+0000","last_became_active":"2023-11-28T13:00:17.242258+0000","last_became_peered":"2023-11-28T13:00:17.242258+0000","last_unstale":"2023-11-28T13:00:17.247832+0000","last_undegraded":"2023-11-28T13:00:17.247832+0000","last_fullsized":"2023-11-28T13:00:17.247832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215295+0000","last_change":"2023-11-28T13:00:17.642442+0000","last_active":"2023-11-28T13:00:19.215295+0000","last_peered":"2023-11-28T13:00:19.215295+0000","last_clean":"2023-11-28T13:00:19.215295+0000","last_became_active":"2023-11-28T13:00:17.642162+0000","last_became_peered":"2023-11-28T13:00:17.642162+0000","last_unstale":"2023-11-28T13:00:19.215295+0000","last_undegraded":"2023-11-28T13:00:19.215295+0000","last_fullsized":"2023-11-28T13:00:19.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.635989+0000","last_change":"2023-11-28T13:00:17.238867+0000","last_active":"2023-11-28T13:00:17.635989+0000","last_peered":"2023-11-28T13:00:17.635989+0000","last_clean":"2023-11-28T13:00:17.635989+0000","last_became_active":"2023-11-28T13:00:17.238545+0000","last_became_peered":"2023-11-28T13:00:17.238545+0000","last_unstale":"2023-11-28T13:00:17.635989+0000","last_undegraded":"2023-11-28T13:00:17.635989+0000","last_fullsized":"2023-11-28T13:00:17.635989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215335+0000","last_change":"2023-11-28T13:00:17.645527+0000","last_active":"2023-11-28T13:00:19.215335+0000","last_peered":"2023-11-28T13:00:19.215335+0000","last_clean":"2023-11-28T13:00:19.215335+0000","last_became_active":"2023-11-28T13:00:17.644057+0000","last_became_peered":"2023-11-28T13:00:17.644057+0000","last_unstale":"2023-11-28T13:00:19.215335+0000","last_undegraded":"2023-11-28T13:00:19.215335+0000","last_fullsized":"2023-11-28T13:00:19.215335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:55.013485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651526+0000","last_change":"2023-11-28T13:00:17.242375+0000","last_active":"2023-11-28T13:00:17.651526+0000","last_peered":"2023-11-28T13:00:17.651526+0000","last_clean":"2023-11-28T13:00:17.651526+0000","last_became_active":"2023-11-28T13:00:17.242010+0000","last_became_peered":"2023-11-28T13:00:17.242010+0000","last_unstale":"2023-11-28T13:00:17.651526+0000","last_undegraded":"2023-11-28T13:00:17.651526+0000","last_fullsized":"2023-11-28T13:00:17.651526+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.522601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635882+0000","last_change":"2023-11-28T13:00:17.647427+0000","last_active":"2023-11-28T13:00:18.635882+0000","last_peered":"2023-11-28T13:00:18.635882+0000","last_clean":"2023-11-28T13:00:18.635882+0000","last_became_active":"2023-11-28T13:00:17.646853+0000","last_became_peered":"2023-11-28T13:00:17.646853+0000","last_unstale":"2023-11-28T13:00:18.635882+0000","last_undegraded":"2023-11-28T13:00:18.635882+0000","last_fullsized":"2023-11-28T13:00:18.635882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.661600+0000","last_change":"2023-11-28T13:00:17.243823+0000","last_active":"2023-11-28T13:00:17.661600+0000","last_peered":"2023-11-28T13:00:17.661600+0000","last_clean":"2023-11-28T13:00:17.661600+0000","last_became_active":"2023-11-28T13:00:17.241238+0000","last_became_peered":"2023-11-28T13:00:17.241238+0000","last_unstale":"2023-11-28T13:00:17.661600+0000","last_undegraded":"2023-11-28T13:00:17.661600+0000","last_fullsized":"2023-11-28T13:00:17.661600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656243+0000","last_change":"2023-11-28T13:00:17.649986+0000","last_active":"2023-11-28T13:00:17.656243+0000","last_peered":"2023-11-28T13:00:17.656243+0000","last_clean":"2023-11-28T13:00:17.656243+0000","last_became_active":"2023-11-28T13:00:17.649402+0000","last_became_peered":"2023-11-28T13:00:17.649402+0000","last_unstale":"2023-11-28T13:00:17.656243+0000","last_undegraded":"2023-11-28T13:00:17.656243+0000","last_fullsized":"2023-11-28T13:00:17.656243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:15.471790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656508+0000","last_change":"2023-11-28T13:00:17.247047+0000","last_active":"2023-11-28T13:00:17.656508+0000","last_peered":"2023-11-28T13:00:17.656508+0000","last_clean":"2023-11-28T13:00:17.656508+0000","last_became_active":"2023-11-28T13:00:17.246742+0000","last_became_peered":"2023-11-28T13:00:17.246742+0000","last_unstale":"2023-11-28T13:00:17.656508+0000","last_undegraded":"2023-11-28T13:00:17.656508+0000","last_fullsized":"2023-11-28T13:00:17.656508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:50.559956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215527+0000","last_change":"2023-11-28T13:00:17.641605+0000","last_active":"2023-11-28T13:00:19.215527+0000","last_peered":"2023-11-28T13:00:19.215527+0000","last_clean":"2023-11-28T13:00:19.215527+0000","last_became_active":"2023-11-28T13:00:17.641350+0000","last_became_peered":"2023-11-28T13:00:17.641350+0000","last_unstale":"2023-11-28T13:00:19.215527+0000","last_undegraded":"2023-11-28T13:00:19.215527+0000","last_fullsized":"2023-11-28T13:00:19.215527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245165+0000","last_change":"2023-11-28T13:00:17.240432+0000","last_active":"2023-11-28T13:00:17.245165+0000","last_peered":"2023-11-28T13:00:17.245165+0000","last_clean":"2023-11-28T13:00:17.245165+0000","last_became_active":"2023-11-28T13:00:17.239192+0000","last_became_peered":"2023-11-28T13:00:17.239192+0000","last_unstale":"2023-11-28T13:00:17.245165+0000","last_undegraded":"2023-11-28T13:00:17.245165+0000","last_fullsized":"2023-11-28T13:00:17.245165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.651770+0000","last_change":"2023-11-28T13:00:17.648774+0000","last_active":"2023-11-28T13:00:17.651770+0000","last_peered":"2023-11-28T13:00:17.651770+0000","last_clean":"2023-11-28T13:00:17.651770+0000","last_became_active":"2023-11-28T13:00:17.647403+0000","last_became_peered":"2023-11-28T13:00:17.647403+0000","last_unstale":"2023-11-28T13:00:17.651770+0000","last_undegraded":"2023-11-28T13:00:17.651770+0000","last_fullsized":"2023-11-28T13:00:17.651770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:17.592744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633096+0000","last_change":"2023-11-28T13:00:17.245977+0000","last_active":"2023-11-28T13:00:17.633096+0000","last_peered":"2023-11-28T13:00:17.633096+0000","last_clean":"2023-11-28T13:00:17.633096+0000","last_became_active":"2023-11-28T13:00:17.245651+0000","last_became_peered":"2023-11-28T13:00:17.245651+0000","last_unstale":"2023-11-28T13:00:17.633096+0000","last_undegraded":"2023-11-28T13:00:17.633096+0000","last_fullsized":"2023-11-28T13:00:17.633096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:30.760573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.235374+0000","last_change":"2023-11-28T13:00:17.235374+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.235374+0000","last_undegraded":"2023-11-28T13:00:17.235374+0000","last_fullsized":"2023-11-28T13:00:17.235374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650468+0000","last_change":"2023-11-28T13:00:17.240716+0000","last_active":"2023-11-28T13:00:17.650468+0000","last_peered":"2023-11-28T13:00:17.650468+0000","last_clean":"2023-11-28T13:00:17.650468+0000","last_became_active":"2023-11-28T13:00:17.239928+0000","last_became_peered":"2023-11-28T13:00:17.239928+0000","last_unstale":"2023-11-28T13:00:17.650468+0000","last_undegraded":"2023-11-28T13:00:17.650468+0000","last_fullsized":"2023-11-28T13:00:17.650468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:32.086174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215680+0000","last_change":"2023-11-28T13:00:17.643923+0000","last_active":"2023-11-28T13:00:19.215680+0000","last_peered":"2023-11-28T13:00:19.215680+0000","last_clean":"2023-11-28T13:00:19.215680+0000","last_became_active":"2023-11-28T13:00:17.643579+0000","last_became_peered":"2023-11-28T13:00:17.643579+0000","last_unstale":"2023-11-28T13:00:19.215680+0000","last_undegraded":"2023-11-28T13:00:19.215680+0000","last_fullsized":"2023-11-28T13:00:19.215680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654531+0000","last_change":"2023-11-28T13:00:17.247259+0000","last_active":"2023-11-28T13:00:17.654531+0000","last_peered":"2023-11-28T13:00:17.654531+0000","last_clean":"2023-11-28T13:00:17.654531+0000","last_became_active":"2023-11-28T13:00:17.246943+0000","last_became_peered":"2023-11-28T13:00:17.246943+0000","last_unstale":"2023-11-28T13:00:17.654531+0000","last_undegraded":"2023-11-28T13:00:17.654531+0000","last_fullsized":"2023-11-28T13:00:17.654531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:02.643388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.647850+0000","last_change":"2023-11-28T13:00:17.644322+0000","last_active":"2023-11-28T13:00:17.647850+0000","last_peered":"2023-11-28T13:00:17.647850+0000","last_clean":"2023-11-28T13:00:17.647850+0000","last_became_active":"2023-11-28T13:00:17.644054+0000","last_became_peered":"2023-11-28T13:00:17.644054+0000","last_unstale":"2023-11-28T13:00:17.647850+0000","last_undegraded":"2023-11-28T13:00:17.647850+0000","last_fullsized":"2023-11-28T13:00:17.647850+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:46.098419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656474+0000","last_change":"2023-11-28T13:00:17.247274+0000","last_active":"2023-11-28T13:00:17.656474+0000","last_peered":"2023-11-28T13:00:17.656474+0000","last_clean":"2023-11-28T13:00:17.656474+0000","last_became_active":"2023-11-28T13:00:17.246851+0000","last_became_peered":"2023-11-28T13:00:17.246851+0000","last_unstale":"2023-11-28T13:00:17.656474+0000","last_undegraded":"2023-11-28T13:00:17.656474+0000","last_fullsized":"2023-11-28T13:00:17.656474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:17.621082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652526+0000","last_change":"2023-11-28T13:00:17.640395+0000","last_active":"2023-11-28T13:00:17.652526+0000","last_peered":"2023-11-28T13:00:17.652526+0000","last_clean":"2023-11-28T13:00:17.652526+0000","last_became_active":"2023-11-28T13:00:17.640195+0000","last_became_peered":"2023-11-28T13:00:17.640195+0000","last_unstale":"2023-11-28T13:00:17.652526+0000","last_undegraded":"2023-11-28T13:00:17.652526+0000","last_fullsized":"2023-11-28T13:00:17.652526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:14.878366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.646780+0000","last_change":"2023-11-28T13:00:17.240698+0000","last_active":"2023-11-28T13:00:17.646780+0000","last_peered":"2023-11-28T13:00:17.646780+0000","last_clean":"2023-11-28T13:00:17.646780+0000","last_became_active":"2023-11-28T13:00:17.240387+0000","last_became_peered":"2023-11-28T13:00:17.240387+0000","last_unstale":"2023-11-28T13:00:17.646780+0000","last_undegraded":"2023-11-28T13:00:17.646780+0000","last_fullsized":"2023-11-28T13:00:17.646780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:05.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648509+0000","last_change":"2023-11-28T13:00:17.647318+0000","last_active":"2023-11-28T13:00:17.648509+0000","last_peered":"2023-11-28T13:00:17.648509+0000","last_clean":"2023-11-28T13:00:17.648509+0000","last_became_active":"2023-11-28T13:00:17.647043+0000","last_became_peered":"2023-11-28T13:00:17.647043+0000","last_unstale":"2023-11-28T13:00:17.648509+0000","last_undegraded":"2023-11-28T13:00:17.648509+0000","last_fullsized":"2023-11-28T13:00:17.648509+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:38.321116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634419+0000","last_change":"2023-11-28T13:00:17.235843+0000","last_active":"2023-11-28T13:00:17.634419+0000","last_peered":"2023-11-28T13:00:17.634419+0000","last_clean":"2023-11-28T13:00:17.634419+0000","last_became_active":"2023-11-28T13:00:17.235495+0000","last_became_peered":"2023-11-28T13:00:17.235495+0000","last_unstale":"2023-11-28T13:00:17.634419+0000","last_undegraded":"2023-11-28T13:00:17.634419+0000","last_fullsized":"2023-11-28T13:00:17.634419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.232454+0000","last_change":"2023-11-28T13:00:17.232454+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.232454+0000","last_undegraded":"2023-11-28T13:00:17.232454+0000","last_fullsized":"2023-11-28T13:00:17.232454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215456+0000","last_change":"2023-11-28T13:00:17.240709+0000","last_active":"2023-11-28T13:00:19.215456+0000","last_peered":"2023-11-28T13:00:19.215456+0000","last_clean":"2023-11-28T13:00:19.215456+0000","last_became_active":"2023-11-28T13:00:17.239893+0000","last_became_peered":"2023-11-28T13:00:17.239893+0000","last_unstale":"2023-11-28T13:00:19.215456+0000","last_undegraded":"2023-11-28T13:00:19.215456+0000","last_fullsized":"2023-11-28T13:00:19.215456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.658839+0000","last_change":"2023-11-28T13:00:17.653235+0000","last_active":"2023-11-28T13:00:17.658839+0000","last_peered":"2023-11-28T13:00:17.658839+0000","last_clean":"2023-11-28T13:00:17.658839+0000","last_became_active":"2023-11-28T13:00:17.650249+0000","last_became_peered":"2023-11-28T13:00:17.650249+0000","last_unstale":"2023-11-28T13:00:17.658839+0000","last_undegraded":"2023-11-28T13:00:17.658839+0000","last_fullsized":"2023-11-28T13:00:17.658839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631372+0000","last_change":"2023-11-28T13:00:17.245224+0000","last_active":"2023-11-28T13:00:17.631372+0000","last_peered":"2023-11-28T13:00:17.631372+0000","last_clean":"2023-11-28T13:00:17.631372+0000","last_became_active":"2023-11-28T13:00:17.244913+0000","last_became_peered":"2023-11-28T13:00:17.244913+0000","last_unstale":"2023-11-28T13:00:17.631372+0000","last_undegraded":"2023-11-28T13:00:17.631372+0000","last_fullsized":"2023-11-28T13:00:17.631372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653954+0000","last_change":"2023-11-28T13:00:17.643015+0000","last_active":"2023-11-28T13:00:17.653954+0000","last_peered":"2023-11-28T13:00:17.653954+0000","last_clean":"2023-11-28T13:00:17.653954+0000","last_became_active":"2023-11-28T13:00:17.642470+0000","last_became_peered":"2023-11-28T13:00:17.642470+0000","last_unstale":"2023-11-28T13:00:17.653954+0000","last_undegraded":"2023-11-28T13:00:17.653954+0000","last_fullsized":"2023-11-28T13:00:17.653954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:41.679867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.639452+0000","last_change":"2023-11-28T13:00:17.244207+0000","last_active":"2023-11-28T13:00:17.639452+0000","last_peered":"2023-11-28T13:00:17.639452+0000","last_clean":"2023-11-28T13:00:17.639452+0000","last_became_active":"2023-11-28T13:00:17.243861+0000","last_became_peered":"2023-11-28T13:00:17.243861+0000","last_unstale":"2023-11-28T13:00:17.639452+0000","last_undegraded":"2023-11-28T13:00:17.639452+0000","last_fullsized":"2023-11-28T13:00:17.639452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656262+0000","last_change":"2023-11-28T13:00:17.651269+0000","last_active":"2023-11-28T13:00:17.656262+0000","last_peered":"2023-11-28T13:00:17.656262+0000","last_clean":"2023-11-28T13:00:17.656262+0000","last_became_active":"2023-11-28T13:00:17.650883+0000","last_became_peered":"2023-11-28T13:00:17.650883+0000","last_unstale":"2023-11-28T13:00:17.656262+0000","last_undegraded":"2023-11-28T13:00:17.656262+0000","last_fullsized":"2023-11-28T13:00:17.656262+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215191+0000","last_change":"2023-11-28T13:00:17.238617+0000","last_active":"2023-11-28T13:00:19.215191+0000","last_peered":"2023-11-28T13:00:19.215191+0000","last_clean":"2023-11-28T13:00:19.215191+0000","last_became_active":"2023-11-28T13:00:17.238344+0000","last_became_peered":"2023-11-28T13:00:17.238344+0000","last_unstale":"2023-11-28T13:00:19.215191+0000","last_undegraded":"2023-11-28T13:00:19.215191+0000","last_fullsized":"2023-11-28T13:00:19.215191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:46.225565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636017+0000","last_change":"2023-11-28T13:00:17.649564+0000","last_active":"2023-11-28T13:00:18.636017+0000","last_peered":"2023-11-28T13:00:18.636017+0000","last_clean":"2023-11-28T13:00:18.636017+0000","last_became_active":"2023-11-28T13:00:17.649312+0000","last_became_peered":"2023-11-28T13:00:17.649312+0000","last_unstale":"2023-11-28T13:00:18.636017+0000","last_undegraded":"2023-11-28T13:00:18.636017+0000","last_fullsized":"2023-11-28T13:00:18.636017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634764+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:00:17.634764+0000","last_peered":"2023-11-28T13:00:17.634764+0000","last_clean":"2023-11-28T13:00:17.634764+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:00:17.634764+0000","last_undegraded":"2023-11-28T13:00:17.634764+0000","last_fullsized":"2023-11-28T13:00:17.634764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.636391+0000","last_change":"2023-11-28T13:00:17.240503+0000","last_active":"2023-11-28T13:00:17.636391+0000","last_peered":"2023-11-28T13:00:17.636391+0000","last_clean":"2023-11-28T13:00:17.636391+0000","last_became_active":"2023-11-28T13:00:17.240256+0000","last_became_peered":"2023-11-28T13:00:17.240256+0000","last_unstale":"2023-11-28T13:00:17.636391+0000","last_undegraded":"2023-11-28T13:00:17.636391+0000","last_fullsized":"2023-11-28T13:00:17.636391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635946+0000","last_change":"2023-11-28T13:00:17.645964+0000","last_active":"2023-11-28T13:00:18.635946+0000","last_peered":"2023-11-28T13:00:18.635946+0000","last_clean":"2023-11-28T13:00:18.635946+0000","last_became_active":"2023-11-28T13:00:17.645384+0000","last_became_peered":"2023-11-28T13:00:17.645384+0000","last_unstale":"2023-11-28T13:00:18.635946+0000","last_undegraded":"2023-11-28T13:00:18.635946+0000","last_fullsized":"2023-11-28T13:00:18.635946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234232+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:00:17.234232+0000","last_peered":"2023-11-28T13:00:17.234232+0000","last_clean":"2023-11-28T13:00:17.234232+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:00:17.234232+0000","last_undegraded":"2023-11-28T13:00:17.234232+0000","last_fullsized":"2023-11-28T13:00:17.234232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.640096+0000","last_change":"2023-11-28T13:00:17.245773+0000","last_active":"2023-11-28T13:00:17.640096+0000","last_peered":"2023-11-28T13:00:17.640096+0000","last_clean":"2023-11-28T13:00:17.640096+0000","last_became_active":"2023-11-28T13:00:17.245409+0000","last_became_peered":"2023-11-28T13:00:17.245409+0000","last_unstale":"2023-11-28T13:00:17.640096+0000","last_undegraded":"2023-11-28T13:00:17.640096+0000","last_fullsized":"2023-11-28T13:00:17.640096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:00.519142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636216+0000","last_change":"2023-11-28T13:00:17.641395+0000","last_active":"2023-11-28T13:00:18.636216+0000","last_peered":"2023-11-28T13:00:18.636216+0000","last_clean":"2023-11-28T13:00:18.636216+0000","last_became_active":"2023-11-28T13:00:17.641149+0000","last_became_peered":"2023-11-28T13:00:17.641149+0000","last_unstale":"2023-11-28T13:00:18.636216+0000","last_undegraded":"2023-11-28T13:00:18.636216+0000","last_fullsized":"2023-11-28T13:00:18.636216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.628022+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:00:17.628022+0000","last_peered":"2023-11-28T13:00:17.628022+0000","last_clean":"2023-11-28T13:00:17.628022+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:00:17.628022+0000","last_undegraded":"2023-11-28T13:00:17.628022+0000","last_fullsized":"2023-11-28T13:00:17.628022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635610+0000","last_change":"2023-11-28T13:00:17.237802+0000","last_active":"2023-11-28T13:00:18.635610+0000","last_peered":"2023-11-28T13:00:18.635610+0000","last_clean":"2023-11-28T13:00:18.635610+0000","last_became_active":"2023-11-28T13:00:17.237487+0000","last_became_peered":"2023-11-28T13:00:17.237487+0000","last_unstale":"2023-11-28T13:00:18.635610+0000","last_undegraded":"2023-11-28T13:00:18.635610+0000","last_fullsized":"2023-11-28T13:00:18.635610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635877+0000","last_change":"2023-11-28T13:00:17.644511+0000","last_active":"2023-11-28T13:00:18.635877+0000","last_peered":"2023-11-28T13:00:18.635877+0000","last_clean":"2023-11-28T13:00:18.635877+0000","last_became_active":"2023-11-28T13:00:17.644095+0000","last_became_peered":"2023-11-28T13:00:17.644095+0000","last_unstale":"2023-11-28T13:00:18.635877+0000","last_undegraded":"2023-11-28T13:00:18.635877+0000","last_fullsized":"2023-11-28T13:00:18.635877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.234538+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:00:17.234538+0000","last_peered":"2023-11-28T13:00:17.234538+0000","last_clean":"2023-11-28T13:00:17.234538+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:00:17.234538+0000","last_undegraded":"2023-11-28T13:00:17.234538+0000","last_fullsized":"2023-11-28T13:00:17.234538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235309+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:00:17.235309+0000","last_peered":"2023-11-28T13:00:17.235309+0000","last_clean":"2023-11-28T13:00:17.235309+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:00:17.235309+0000","last_undegraded":"2023-11-28T13:00:17.235309+0000","last_fullsized":"2023-11-28T13:00:17.235309+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215311+0000","last_change":"2023-11-28T13:00:17.239373+0000","last_active":"2023-11-28T13:00:19.215311+0000","last_peered":"2023-11-28T13:00:19.215311+0000","last_clean":"2023-11-28T13:00:19.215311+0000","last_became_active":"2023-11-28T13:00:17.238864+0000","last_became_peered":"2023-11-28T13:00:17.238864+0000","last_unstale":"2023-11-28T13:00:19.215311+0000","last_undegraded":"2023-11-28T13:00:19.215311+0000","last_fullsized":"2023-11-28T13:00:19.215311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:17.763933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.653117+0000","last_change":"2023-11-28T13:00:17.649771+0000","last_active":"2023-11-28T13:00:17.653117+0000","last_peered":"2023-11-28T13:00:17.653117+0000","last_clean":"2023-11-28T13:00:17.653117+0000","last_became_active":"2023-11-28T13:00:17.649475+0000","last_became_peered":"2023-11-28T13:00:17.649475+0000","last_unstale":"2023-11-28T13:00:17.653117+0000","last_undegraded":"2023-11-28T13:00:17.653117+0000","last_fullsized":"2023-11-28T13:00:17.653117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.235498+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:00:17.235498+0000","last_peered":"2023-11-28T13:00:17.235498+0000","last_clean":"2023-11-28T13:00:17.235498+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:00:17.235498+0000","last_undegraded":"2023-11-28T13:00:17.235498+0000","last_fullsized":"2023-11-28T13:00:17.235498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253107+0000","last_change":"2023-11-28T13:00:17.251262+0000","last_active":"2023-11-28T13:00:17.253107+0000","last_peered":"2023-11-28T13:00:17.253107+0000","last_clean":"2023-11-28T13:00:17.253107+0000","last_became_active":"2023-11-28T13:00:17.250948+0000","last_became_peered":"2023-11-28T13:00:17.250948+0000","last_unstale":"2023-11-28T13:00:17.253107+0000","last_undegraded":"2023-11-28T13:00:17.253107+0000","last_fullsized":"2023-11-28T13:00:17.253107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:37.311866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233919+0000","last_change":"2023-11-28T13:00:17.233919+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233919+0000","last_undegraded":"2023-11-28T13:00:17.233919+0000","last_fullsized":"2023-11-28T13:00:17.233919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.236149+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:00:17.236149+0000","last_peered":"2023-11-28T13:00:17.236149+0000","last_clean":"2023-11-28T13:00:17.236149+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:00:17.236149+0000","last_undegraded":"2023-11-28T13:00:17.236149+0000","last_fullsized":"2023-11-28T13:00:17.236149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636224+0000","last_change":"2023-11-28T13:00:17.237999+0000","last_active":"2023-11-28T13:00:18.636224+0000","last_peered":"2023-11-28T13:00:18.636224+0000","last_clean":"2023-11-28T13:00:18.636224+0000","last_became_active":"2023-11-28T13:00:17.237625+0000","last_became_peered":"2023-11-28T13:00:17.237625+0000","last_unstale":"2023-11-28T13:00:18.636224+0000","last_undegraded":"2023-11-28T13:00:18.636224+0000","last_fullsized":"2023-11-28T13:00:18.636224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2023-11-28T13:00:17.233381+0000","last_change":"2023-11-28T13:00:17.233381+0000","last_active":"2023-11-28T13:00:17.215040+0000","last_peered":"2023-11-28T13:00:17.215040+0000","last_clean":"2023-11-28T13:00:17.215040+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:00:17.233381+0000","last_undegraded":"2023-11-28T13:00:17.233381+0000","last_fullsized":"2023-11-28T13:00:17.233381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215660+0000","last_change":"2023-11-28T13:00:17.239284+0000","last_active":"2023-11-28T13:00:19.215660+0000","last_peered":"2023-11-28T13:00:19.215660+0000","last_clean":"2023-11-28T13:00:19.215660+0000","last_became_active":"2023-11-28T13:00:17.238997+0000","last_became_peered":"2023-11-28T13:00:17.238997+0000","last_unstale":"2023-11-28T13:00:19.215660+0000","last_undegraded":"2023-11-28T13:00:19.215660+0000","last_fullsized":"2023-11-28T13:00:19.215660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:55.423382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633198+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:00:17.633198+0000","last_peered":"2023-11-28T13:00:17.633198+0000","last_clean":"2023-11-28T13:00:17.633198+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:00:17.633198+0000","last_undegraded":"2023-11-28T13:00:17.633198+0000","last_fullsized":"2023-11-28T13:00:17.633198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215108+0000","last_change":"2023-11-28T13:00:17.646149+0000","last_active":"2023-11-28T13:00:19.215108+0000","last_peered":"2023-11-28T13:00:19.215108+0000","last_clean":"2023-11-28T13:00:19.215108+0000","last_became_active":"2023-11-28T13:00:17.645893+0000","last_became_peered":"2023-11-28T13:00:17.645893+0000","last_unstale":"2023-11-28T13:00:19.215108+0000","last_undegraded":"2023-11-28T13:00:19.215108+0000","last_fullsized":"2023-11-28T13:00:19.215108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:24.466207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.226618+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:00:17.226618+0000","last_peered":"2023-11-28T13:00:17.226618+0000","last_clean":"2023-11-28T13:00:17.226618+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:00:17.226618+0000","last_undegraded":"2023-11-28T13:00:17.226618+0000","last_fullsized":"2023-11-28T13:00:17.226618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.245166+0000","last_change":"2023-11-28T13:00:17.240379+0000","last_active":"2023-11-28T13:00:17.245166+0000","last_peered":"2023-11-28T13:00:17.245166+0000","last_clean":"2023-11-28T13:00:17.245166+0000","last_became_active":"2023-11-28T13:00:17.239505+0000","last_became_peered":"2023-11-28T13:00:17.239505+0000","last_unstale":"2023-11-28T13:00:17.245166+0000","last_undegraded":"2023-11-28T13:00:17.245166+0000","last_fullsized":"2023-11-28T13:00:17.245166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655975+0000","last_change":"2023-11-28T13:00:17.646003+0000","last_active":"2023-11-28T13:00:17.655975+0000","last_peered":"2023-11-28T13:00:17.655975+0000","last_clean":"2023-11-28T13:00:17.655975+0000","last_became_active":"2023-11-28T13:00:17.645727+0000","last_became_peered":"2023-11-28T13:00:17.645727+0000","last_unstale":"2023-11-28T13:00:17.655975+0000","last_undegraded":"2023-11-28T13:00:17.655975+0000","last_fullsized":"2023-11-28T13:00:17.655975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:30.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.253113+0000","last_change":"2023-11-28T13:00:17.251398+0000","last_active":"2023-11-28T13:00:17.253113+0000","last_peered":"2023-11-28T13:00:17.253113+0000","last_clean":"2023-11-28T13:00:17.253113+0000","last_became_active":"2023-11-28T13:00:17.251054+0000","last_became_peered":"2023-11-28T13:00:17.251054+0000","last_unstale":"2023-11-28T13:00:17.253113+0000","last_undegraded":"2023-11-28T13:00:17.253113+0000","last_fullsized":"2023-11-28T13:00:17.253113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217848+0000","last_change":"2023-11-28T13:00:17.641355+0000","last_active":"2023-11-28T13:00:19.217848+0000","last_peered":"2023-11-28T13:00:19.217848+0000","last_clean":"2023-11-28T13:00:19.217848+0000","last_became_active":"2023-11-28T13:00:17.641093+0000","last_became_peered":"2023-11-28T13:00:17.641093+0000","last_unstale":"2023-11-28T13:00:19.217848+0000","last_undegraded":"2023-11-28T13:00:19.217848+0000","last_fullsized":"2023-11-28T13:00:19.217848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:41.816686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.655479+0000","last_change":"2023-11-28T13:00:17.246896+0000","last_active":"2023-11-28T13:00:17.655479+0000","last_peered":"2023-11-28T13:00:17.655479+0000","last_clean":"2023-11-28T13:00:17.655479+0000","last_became_active":"2023-11-28T13:00:17.246604+0000","last_became_peered":"2023-11-28T13:00:17.246604+0000","last_unstale":"2023-11-28T13:00:17.655479+0000","last_undegraded":"2023-11-28T13:00:17.655479+0000","last_fullsized":"2023-11-28T13:00:17.655479+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656231+0000","last_change":"2023-11-28T13:00:17.648973+0000","last_active":"2023-11-28T13:00:17.656231+0000","last_peered":"2023-11-28T13:00:17.656231+0000","last_clean":"2023-11-28T13:00:17.656231+0000","last_became_active":"2023-11-28T13:00:17.648323+0000","last_became_peered":"2023-11-28T13:00:17.648323+0000","last_unstale":"2023-11-28T13:00:17.656231+0000","last_undegraded":"2023-11-28T13:00:17.656231+0000","last_fullsized":"2023-11-28T13:00:17.656231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.654699+0000","last_change":"2023-11-28T13:00:17.247013+0000","last_active":"2023-11-28T13:00:17.654699+0000","last_peered":"2023-11-28T13:00:17.654699+0000","last_clean":"2023-11-28T13:00:17.654699+0000","last_became_active":"2023-11-28T13:00:17.246645+0000","last_became_peered":"2023-11-28T13:00:17.246645+0000","last_unstale":"2023-11-28T13:00:17.654699+0000","last_undegraded":"2023-11-28T13:00:17.654699+0000","last_fullsized":"2023-11-28T13:00:17.654699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656116+0000","last_change":"2023-11-28T13:00:17.649078+0000","last_active":"2023-11-28T13:00:17.656116+0000","last_peered":"2023-11-28T13:00:17.656116+0000","last_clean":"2023-11-28T13:00:17.656116+0000","last_became_active":"2023-11-28T13:00:17.648432+0000","last_became_peered":"2023-11-28T13:00:17.648432+0000","last_unstale":"2023-11-28T13:00:17.656116+0000","last_undegraded":"2023-11-28T13:00:17.656116+0000","last_fullsized":"2023-11-28T13:00:17.656116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215495+0000","last_change":"2023-11-28T13:00:17.234074+0000","last_active":"2023-11-28T13:00:19.215495+0000","last_peered":"2023-11-28T13:00:19.215495+0000","last_clean":"2023-11-28T13:00:19.215495+0000","last_became_active":"2023-11-28T13:00:17.233732+0000","last_became_peered":"2023-11-28T13:00:17.233732+0000","last_unstale":"2023-11-28T13:00:19.215495+0000","last_undegraded":"2023-11-28T13:00:19.215495+0000","last_fullsized":"2023-11-28T13:00:19.215495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:36.950864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215231+0000","last_change":"2023-11-28T13:00:17.641105+0000","last_active":"2023-11-28T13:00:19.215231+0000","last_peered":"2023-11-28T13:00:19.215231+0000","last_clean":"2023-11-28T13:00:19.215231+0000","last_became_active":"2023-11-28T13:00:17.640854+0000","last_became_peered":"2023-11-28T13:00:17.640854+0000","last_unstale":"2023-11-28T13:00:19.215231+0000","last_undegraded":"2023-11-28T13:00:19.215231+0000","last_fullsized":"2023-11-28T13:00:19.215231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.247814+0000","last_change":"2023-11-28T13:00:17.242508+0000","last_active":"2023-11-28T13:00:17.247814+0000","last_peered":"2023-11-28T13:00:17.247814+0000","last_clean":"2023-11-28T13:00:17.247814+0000","last_became_active":"2023-11-28T13:00:17.242230+0000","last_became_peered":"2023-11-28T13:00:17.242230+0000","last_unstale":"2023-11-28T13:00:17.247814+0000","last_undegraded":"2023-11-28T13:00:17.247814+0000","last_fullsized":"2023-11-28T13:00:17.247814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656224+0000","last_change":"2023-11-28T13:00:17.650868+0000","last_active":"2023-11-28T13:00:17.656224+0000","last_peered":"2023-11-28T13:00:17.656224+0000","last_clean":"2023-11-28T13:00:17.656224+0000","last_became_active":"2023-11-28T13:00:17.649842+0000","last_became_peered":"2023-11-28T13:00:17.649842+0000","last_unstale":"2023-11-28T13:00:17.656224+0000","last_undegraded":"2023-11-28T13:00:17.656224+0000","last_fullsized":"2023-11-28T13:00:17.656224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.645743+0000","last_change":"2023-11-28T13:00:17.641330+0000","last_active":"2023-11-28T13:00:17.645743+0000","last_peered":"2023-11-28T13:00:17.645743+0000","last_clean":"2023-11-28T13:00:17.645743+0000","last_became_active":"2023-11-28T13:00:17.641029+0000","last_became_peered":"2023-11-28T13:00:17.641029+0000","last_unstale":"2023-11-28T13:00:17.645743+0000","last_undegraded":"2023-11-28T13:00:17.645743+0000","last_fullsized":"2023-11-28T13:00:17.645743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:28.607875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215202+0000","last_change":"2023-11-28T13:00:17.643816+0000","last_active":"2023-11-28T13:00:19.215202+0000","last_peered":"2023-11-28T13:00:19.215202+0000","last_clean":"2023-11-28T13:00:19.215202+0000","last_became_active":"2023-11-28T13:00:17.643260+0000","last_became_peered":"2023-11-28T13:00:17.643260+0000","last_unstale":"2023-11-28T13:00:19.215202+0000","last_undegraded":"2023-11-28T13:00:19.215202+0000","last_fullsized":"2023-11-28T13:00:19.215202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656140+0000","last_change":"2023-11-28T13:00:17.649431+0000","last_active":"2023-11-28T13:00:17.656140+0000","last_peered":"2023-11-28T13:00:17.656140+0000","last_clean":"2023-11-28T13:00:17.656140+0000","last_became_active":"2023-11-28T13:00:17.648763+0000","last_became_peered":"2023-11-28T13:00:17.648763+0000","last_unstale":"2023-11-28T13:00:17.656140+0000","last_undegraded":"2023-11-28T13:00:17.656140+0000","last_fullsized":"2023-11-28T13:00:17.656140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:10.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215472+0000","last_change":"2023-11-28T13:00:17.649148+0000","last_active":"2023-11-28T13:00:19.215472+0000","last_peered":"2023-11-28T13:00:19.215472+0000","last_clean":"2023-11-28T13:00:19.215472+0000","last_became_active":"2023-11-28T13:00:17.648925+0000","last_became_peered":"2023-11-28T13:00:17.648925+0000","last_unstale":"2023-11-28T13:00:19.215472+0000","last_undegraded":"2023-11-28T13:00:19.215472+0000","last_fullsized":"2023-11-28T13:00:19.215472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636138+0000","last_change":"2023-11-28T13:00:17.648409+0000","last_active":"2023-11-28T13:00:18.636138+0000","last_peered":"2023-11-28T13:00:18.636138+0000","last_clean":"2023-11-28T13:00:18.636138+0000","last_became_active":"2023-11-28T13:00:17.648164+0000","last_became_peered":"2023-11-28T13:00:17.648164+0000","last_unstale":"2023-11-28T13:00:18.636138+0000","last_undegraded":"2023-11-28T13:00:18.636138+0000","last_fullsized":"2023-11-28T13:00:18.636138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1764,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":96947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7940,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363900,"statfs":{"total":96636764160,"available":96628633600,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":91808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8928,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94362912,"statfs":{"total":96636764160,"available":96627621888,"internally_reserved":0,"allocated":811008,"data_stored":666055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9112,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362728,"statfs":{"total":96636764160,"available":96627433472,"internally_reserved":0,"allocated":868352,"data_stored":682336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8000,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363840,"statfs":{"total":96636764160,"available":96628572160,"internally_reserved":0,"allocated":253952,"data_stored":90840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7992,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363848,"statfs":{"total":96636764160,"available":96628580352,"internally_reserved":0,"allocated":245760,"data_stored":90204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8068,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363772,"statfs":{"total":96636764160,"available":96628502528,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:23.327 INFO:tasks.ceph.mds.b.smithi177.stderr:2023-11-28T13:00:23.327+0000 7fadb3441700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:00:26.234 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:26.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.367+0000 7f7fb579e700 1 Processor -- start 2023-11-28T13:00:26.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.367+0000 7f7fb579e700 1 -- start start 2023-11-28T13:00:26.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.367+0000 7f7fb579e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7fb00af0c0 0x7f7fb006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.367+0000 7f7fb579e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00b4960 0x7f7fb006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7fb579e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb006d170 con 0x7f7fb006a7b0 2023-11-28T13:00:26.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7fb579e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb015d900 con 0x7f7fb00af0c0 2023-11-28T13:00:26.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7fb579e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0071cb0 con 0x7f7fb00b4960 2023-11-28T13:00:26.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7fb00af0c0 0x7f7fb006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7fb4f9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7fb006a7b0 0x7f7fb0160340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56556/0 (socket says 172.21.15.153:56556) 2023-11-28T13:00:26.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7fb4f9d700 1 -- 172.21.15.153:0/3620315631 learned_addr learned my addr 172.21.15.153:0/3620315631 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:26.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faffff700 1 --2- 172.21.15.153:0/3620315631 >> v2:172.21.15.153:3300/0 conn(0x7f7fb00af0c0 0x7f7fb006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38388/0 (socket says 172.21.15.153:38388) 2023-11-28T13:00:26.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faf7fe700 1 --2- 172.21.15.153:0/3620315631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00b4960 0x7f7fb006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faffff700 1 -- 172.21.15.153:0/3620315631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00b4960 msgr2=0x7f7fb006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faffff700 1 --2- 172.21.15.153:0/3620315631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00b4960 0x7f7fb006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faffff700 1 -- 172.21.15.153:0/3620315631 >> v1:172.21.15.153:6789/0 conn(0x7f7fb006a7b0 legacy=0x7f7fb0160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faffff700 1 -- 172.21.15.153:0/3620315631 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb00b3fd0 con 0x7f7fb00af0c0 2023-11-28T13:00:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faeffd700 1 -- 172.21.15.153:0/3620315631 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741969675 0 0) 0x7f7fb006d170 con 0x7f7fb006a7b0 2023-11-28T13:00:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faffff700 1 --2- 172.21.15.153:0/3620315631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb00af0c0 0x7f7fb006cc30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009840 tx=0x7f7fa0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e5a410cd57e011 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faeffd700 1 -- 172.21.15.153:0/3620315631 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0011830 con 0x7f7fb00af0c0 2023-11-28T13:00:26.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faeffd700 1 -- 172.21.15.153:0/3620315631 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0011990 con 0x7f7fb00af0c0 2023-11-28T13:00:26.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.368+0000 7f7faeffd700 1 -- 172.21.15.153:0/3620315631 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0011d20 con 0x7f7fb00af0c0 2023-11-28T13:00:26.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 -- 172.21.15.153:0/3620315631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb00af0c0 msgr2=0x7f7fb006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 --2- 172.21.15.153:0/3620315631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb00af0c0 0x7f7fb006cc30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009840 tx=0x7f7fa0009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:26.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 -- 172.21.15.153:0/3620315631 shutdown_connections 2023-11-28T13:00:26.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 --2- 172.21.15.153:0/3620315631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb00af0c0 0x7f7fb006cc30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 --2- 172.21.15.153:0/3620315631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00b4960 0x7f7fb006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 -- 172.21.15.153:0/3620315631 >> 172.21.15.153:0/3620315631 conn(0x7f7fb006b380 msgr2=0x7f7fb006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 -- 172.21.15.153:0/3620315631 shutdown_connections 2023-11-28T13:00:26.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.369+0000 7f7fb579e700 1 -- 172.21.15.153:0/3620315631 wait complete. 2023-11-28T13:00:26.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.370+0000 7f7fb579e700 1 Processor -- start 2023-11-28T13:00:26.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.370+0000 7f7fb579e700 1 -- start start 2023-11-28T13:00:26.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7fb579e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb006a7b0 0x7f7fb00ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7fb579e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00af0c0 0x7f7fb0145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7fb579e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fb00b4960 0x7f7fb0148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7fb579e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01456c0 con 0x7f7fb006a7b0 2023-11-28T13:00:26.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7fb579e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01493a0 con 0x7f7fb00b4960 2023-11-28T13:00:26.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7fb579e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0155850 con 0x7f7fb00af0c0 2023-11-28T13:00:26.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7faffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb006a7b0 0x7f7fb00ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7faffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb006a7b0 0x7f7fb00ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38394/0 (socket says 172.21.15.153:38394) 2023-11-28T13:00:26.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7faffff700 1 -- 172.21.15.153:0/1642024950 learned_addr learned my addr 172.21.15.153:0/1642024950 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:26.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.371+0000 7f7faf7fe700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00af0c0 0x7f7fb0145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7fb4f9d700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fb00b4960 0x7f7fb0148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7faffff700 1 -- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00af0c0 msgr2=0x7f7fb0145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7faffff700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00af0c0 0x7f7fb0145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7faffff700 1 -- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fb00b4960 msgr2=0x7f7fb0148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7faffff700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fb00b4960 0x7f7fb0148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7faffff700 1 -- 172.21.15.153:0/1642024950 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00094a0 con 0x7f7fb006a7b0 2023-11-28T13:00:26.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7fb4f9d700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fb00b4960 0x7f7fb0148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:26.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7faffff700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb006a7b0 0x7f7fb00ae670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009fd0 tx=0x7f7fa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7facff9700 1 -- 172.21.15.153:0/1642024950 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00038a0 con 0x7f7fb006a7b0 2023-11-28T13:00:26.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7facff9700 1 -- 172.21.15.153:0/1642024950 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa001f070 con 0x7f7fb006a7b0 2023-11-28T13:00:26.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb006abd0 con 0x7f7fb006a7b0 2023-11-28T13:00:26.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7facff9700 1 -- 172.21.15.153:0/1642024950 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0003c40 con 0x7f7fb006a7b0 2023-11-28T13:00:26.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.372+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb015dd20 con 0x7f7fb006a7b0 2023-11-28T13:00:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.373+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb006d3a0 con 0x7f7fb006a7b0 2023-11-28T13:00:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.373+0000 7f7facff9700 1 -- 172.21.15.153:0/1642024950 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7fa000f050 con 0x7f7fb006a7b0 2023-11-28T13:00:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.374+0000 7f7facff9700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7f9c0682f0 0x7f7f9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.374+0000 7f7faf7fe700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7f9c0682f0 0x7f7f9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.374+0000 7f7facff9700 1 -- 172.21.15.153:0/1642024950 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fa0021070 con 0x7f7fb006a7b0 2023-11-28T13:00:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.374+0000 7f7faf7fe700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7f9c0682f0 0x7f7f9c06a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ad1c0 tx=0x7f7fa4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.378+0000 7f7facff9700 1 -- 172.21.15.153:0/1642024950 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7fa0053e00 con 0x7f7fb006a7b0 2023-11-28T13:00:26.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.558+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fb0155990 con 0x7f7f9c0682f0 2023-11-28T13:00:26.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.565+0000 7f7facff9700 1 -- 172.21.15.153:0/1642024950 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286665 (secure 0 0 0) 0x7f7fa4009660 con 0x7f7f9c0682f0 2023-11-28T13:00:26.571 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:26.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.568+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7f9c0682f0 msgr2=0x7f7f9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.568+0000 7f7fb579e700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7f9c0682f0 0x7f7f9c06a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ad1c0 tx=0x7f7fa4008040 comp rx=0 tx=0).stop 2023-11-28T13:00:26.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.568+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb006a7b0 msgr2=0x7f7fb00ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.568+0000 7f7fb579e700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb006a7b0 0x7f7fb00ae670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009fd0 tx=0x7f7fa0004750 comp rx=0 tx=0).stop 2023-11-28T13:00:26.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.568+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 shutdown_connections 2023-11-28T13:00:26.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.569+0000 7f7fb579e700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7f9c0682f0 0x7f7f9c06a7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.569+0000 7f7fb579e700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7fb006a7b0 0x7f7fb00ae670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.569+0000 7f7fb579e700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fb00b4960 0x7f7fb0148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.569+0000 7f7fb579e700 1 --2- 172.21.15.153:0/1642024950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fb00af0c0 0x7f7fb0145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.569+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 >> 172.21.15.153:0/1642024950 conn(0x7f7fb006b380 msgr2=0x7f7fb01602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.569+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 shutdown_connections 2023-11-28T13:00:26.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.569+0000 7f7fb579e700 1 -- 172.21.15.153:0/1642024950 wait complete. 2023-11-28T13:00:26.574 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:00:26.582 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-11-28T13:00:26.137205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66404,"kb_used_data":3236,"kb_used_omap":0,"kb_used_meta":63104,"kb_avail":754908316,"statfs":{"total":773094113280,"available":773026115584,"internally_reserved":0,"allocated":3313664,"data_stored":1959483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64618496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.510646"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217825+0000","last_change":"2023-11-28T13:00:17.635428+0000","last_active":"2023-11-28T13:00:19.217825+0000","last_peered":"2023-11-28T13:00:19.217825+0000","last_clean":"2023-11-28T13:00:19.217825+0000","last_became_active":"2023-11-28T13:00:17.635148+0000","last_became_peered":"2023-11-28T13:00:17.635148+0000","last_unstale":"2023-11-28T13:00:19.217825+0000","last_undegraded":"2023-11-28T13:00:19.217825+0000","last_fullsized":"2023-11-28T13:00:19.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338757+0000","last_change":"2023-11-28T13:00:17.645017+0000","last_active":"2023-11-28T13:00:18.338757+0000","last_peered":"2023-11-28T13:00:18.338757+0000","last_clean":"2023-11-28T13:00:18.338757+0000","last_became_active":"2023-11-28T13:00:17.644524+0000","last_became_peered":"2023-11-28T13:00:17.644524+0000","last_unstale":"2023-11-28T13:00:18.338757+0000","last_undegraded":"2023-11-28T13:00:18.338757+0000","last_fullsized":"2023-11-28T13:00:18.338757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497428+0000","last_change":"2023-11-28T13:00:17.648662+0000","last_active":"2023-11-28T13:00:18.497428+0000","last_peered":"2023-11-28T13:00:18.497428+0000","last_clean":"2023-11-28T13:00:18.497428+0000","last_became_active":"2023-11-28T13:00:17.648271+0000","last_became_peered":"2023-11-28T13:00:17.648271+0000","last_unstale":"2023-11-28T13:00:18.497428+0000","last_undegraded":"2023-11-28T13:00:18.497428+0000","last_fullsized":"2023-11-28T13:00:18.497428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:37.103258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330841+0000","last_change":"2023-11-28T13:00:17.643733+0000","last_active":"2023-11-28T13:00:18.330841+0000","last_peered":"2023-11-28T13:00:18.330841+0000","last_clean":"2023-11-28T13:00:18.330841+0000","last_became_active":"2023-11-28T13:00:17.643428+0000","last_became_peered":"2023-11-28T13:00:17.643428+0000","last_unstale":"2023-11-28T13:00:18.330841+0000","last_undegraded":"2023-11-28T13:00:18.330841+0000","last_fullsized":"2023-11-28T13:00:18.330841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652247+0000","last_change":"2023-11-28T13:00:17.650740+0000","last_active":"2023-11-28T13:00:17.652247+0000","last_peered":"2023-11-28T13:00:17.652247+0000","last_clean":"2023-11-28T13:00:17.652247+0000","last_became_active":"2023-11-28T13:00:17.650427+0000","last_became_peered":"2023-11-28T13:00:17.650427+0000","last_unstale":"2023-11-28T13:00:17.652247+0000","last_undegraded":"2023-11-28T13:00:17.652247+0000","last_fullsized":"2023-11-28T13:00:17.652247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497199+0000","last_change":"2023-11-28T13:00:17.652679+0000","last_active":"2023-11-28T13:00:18.497199+0000","last_peered":"2023-11-28T13:00:18.497199+0000","last_clean":"2023-11-28T13:00:18.497199+0000","last_became_active":"2023-11-28T13:00:17.648169+0000","last_became_peered":"2023-11-28T13:00:17.648169+0000","last_unstale":"2023-11-28T13:00:18.497199+0000","last_undegraded":"2023-11-28T13:00:18.497199+0000","last_fullsized":"2023-11-28T13:00:18.497199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:58.762522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331071+0000","last_change":"2023-11-28T13:00:17.647769+0000","last_active":"2023-11-28T13:00:18.331071+0000","last_peered":"2023-11-28T13:00:18.331071+0000","last_clean":"2023-11-28T13:00:18.331071+0000","last_became_active":"2023-11-28T13:00:17.647476+0000","last_became_peered":"2023-11-28T13:00:17.647476+0000","last_unstale":"2023-11-28T13:00:18.331071+0000","last_undegraded":"2023-11-28T13:00:18.331071+0000","last_fullsized":"2023-11-28T13:00:18.331071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338605+0000","last_change":"2023-11-28T13:00:17.653233+0000","last_active":"2023-11-28T13:00:18.338605+0000","last_peered":"2023-11-28T13:00:18.338605+0000","last_clean":"2023-11-28T13:00:18.338605+0000","last_became_active":"2023-11-28T13:00:17.652957+0000","last_became_peered":"2023-11-28T13:00:17.652957+0000","last_unstale":"2023-11-28T13:00:18.338605+0000","last_undegraded":"2023-11-28T13:00:18.338605+0000","last_fullsized":"2023-11-28T13:00:18.338605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652321+0000","last_change":"2023-11-28T13:00:17.650826+0000","last_active":"2023-11-28T13:00:17.652321+0000","last_peered":"2023-11-28T13:00:17.652321+0000","last_clean":"2023-11-28T13:00:17.652321+0000","last_became_active":"2023-11-28T13:00:17.650572+0000","last_became_peered":"2023-11-28T13:00:17.650572+0000","last_unstale":"2023-11-28T13:00:17.652321+0000","last_undegraded":"2023-11-28T13:00:17.652321+0000","last_fullsized":"2023-11-28T13:00:17.652321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338780+0000","last_change":"2023-11-28T13:00:17.642144+0000","last_active":"2023-11-28T13:00:18.338780+0000","last_peered":"2023-11-28T13:00:18.338780+0000","last_clean":"2023-11-28T13:00:18.338780+0000","last_became_active":"2023-11-28T13:00:17.641814+0000","last_became_peered":"2023-11-28T13:00:17.641814+0000","last_unstale":"2023-11-28T13:00:18.338780+0000","last_undegraded":"2023-11-28T13:00:18.338780+0000","last_fullsized":"2023-11-28T13:00:18.338780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497444+0000","last_change":"2023-11-28T13:00:17.643190+0000","last_active":"2023-11-28T13:00:18.497444+0000","last_peered":"2023-11-28T13:00:18.497444+0000","last_clean":"2023-11-28T13:00:18.497444+0000","last_became_active":"2023-11-28T13:00:17.642741+0000","last_became_peered":"2023-11-28T13:00:17.642741+0000","last_unstale":"2023-11-28T13:00:18.497444+0000","last_undegraded":"2023-11-28T13:00:18.497444+0000","last_fullsized":"2023-11-28T13:00:18.497444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:33.241895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217739+0000","last_change":"2023-11-28T13:00:17.654503+0000","last_active":"2023-11-28T13:00:19.217739+0000","last_peered":"2023-11-28T13:00:19.217739+0000","last_clean":"2023-11-28T13:00:19.217739+0000","last_became_active":"2023-11-28T13:00:17.654204+0000","last_became_peered":"2023-11-28T13:00:17.654204+0000","last_unstale":"2023-11-28T13:00:19.217739+0000","last_undegraded":"2023-11-28T13:00:19.217739+0000","last_fullsized":"2023-11-28T13:00:19.217739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:27.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217678+0000","last_change":"2023-11-28T13:00:17.634718+0000","last_active":"2023-11-28T13:00:19.217678+0000","last_peered":"2023-11-28T13:00:19.217678+0000","last_clean":"2023-11-28T13:00:19.217678+0000","last_became_active":"2023-11-28T13:00:17.634420+0000","last_became_peered":"2023-11-28T13:00:17.634420+0000","last_unstale":"2023-11-28T13:00:19.217678+0000","last_undegraded":"2023-11-28T13:00:19.217678+0000","last_fullsized":"2023-11-28T13:00:19.217678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.567102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338634+0000","last_change":"2023-11-28T13:00:17.641361+0000","last_active":"2023-11-28T13:00:18.338634+0000","last_peered":"2023-11-28T13:00:18.338634+0000","last_clean":"2023-11-28T13:00:18.338634+0000","last_became_active":"2023-11-28T13:00:17.641078+0000","last_became_peered":"2023-11-28T13:00:17.641078+0000","last_unstale":"2023-11-28T13:00:18.338634+0000","last_undegraded":"2023-11-28T13:00:18.338634+0000","last_fullsized":"2023-11-28T13:00:18.338634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:38.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331110+0000","last_change":"2023-11-28T13:00:17.647800+0000","last_active":"2023-11-28T13:00:18.331110+0000","last_peered":"2023-11-28T13:00:18.331110+0000","last_clean":"2023-11-28T13:00:18.331110+0000","last_became_active":"2023-11-28T13:00:17.647519+0000","last_became_peered":"2023-11-28T13:00:17.647519+0000","last_unstale":"2023-11-28T13:00:18.331110+0000","last_undegraded":"2023-11-28T13:00:18.331110+0000","last_fullsized":"2023-11-28T13:00:18.331110+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497110+0000","last_change":"2023-11-28T13:00:17.651090+0000","last_active":"2023-11-28T13:00:18.497110+0000","last_peered":"2023-11-28T13:00:18.497110+0000","last_clean":"2023-11-28T13:00:18.497110+0000","last_became_active":"2023-11-28T13:00:17.649174+0000","last_became_peered":"2023-11-28T13:00:17.649174+0000","last_unstale":"2023-11-28T13:00:18.497110+0000","last_undegraded":"2023-11-28T13:00:18.497110+0000","last_fullsized":"2023-11-28T13:00:18.497110+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:32.070272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652338+0000","last_change":"2023-11-28T13:00:17.650764+0000","last_active":"2023-11-28T13:00:17.652338+0000","last_peered":"2023-11-28T13:00:17.652338+0000","last_clean":"2023-11-28T13:00:17.652338+0000","last_became_active":"2023-11-28T13:00:17.650252+0000","last_became_peered":"2023-11-28T13:00:17.650252+0000","last_unstale":"2023-11-28T13:00:17.652338+0000","last_undegraded":"2023-11-28T13:00:17.652338+0000","last_fullsized":"2023-11-28T13:00:17.652338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497486+0000","last_change":"2023-11-28T13:00:17.648771+0000","last_active":"2023-11-28T13:00:18.497486+0000","last_peered":"2023-11-28T13:00:18.497486+0000","last_clean":"2023-11-28T13:00:18.497486+0000","last_became_active":"2023-11-28T13:00:17.646367+0000","last_became_peered":"2023-11-28T13:00:17.646367+0000","last_unstale":"2023-11-28T13:00:18.497486+0000","last_undegraded":"2023-11-28T13:00:18.497486+0000","last_fullsized":"2023-11-28T13:00:18.497486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:33.549492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217793+0000","last_change":"2023-11-28T13:00:17.654191+0000","last_active":"2023-11-28T13:00:19.217793+0000","last_peered":"2023-11-28T13:00:19.217793+0000","last_clean":"2023-11-28T13:00:19.217793+0000","last_became_active":"2023-11-28T13:00:17.653922+0000","last_became_peered":"2023-11-28T13:00:17.653922+0000","last_unstale":"2023-11-28T13:00:19.217793+0000","last_undegraded":"2023-11-28T13:00:19.217793+0000","last_fullsized":"2023-11-28T13:00:19.217793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.066202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497368+0000","last_change":"2023-11-28T13:00:17.653882+0000","last_active":"2023-11-28T13:00:18.497368+0000","last_peered":"2023-11-28T13:00:18.497368+0000","last_clean":"2023-11-28T13:00:18.497368+0000","last_became_active":"2023-11-28T13:00:17.653497+0000","last_became_peered":"2023-11-28T13:00:17.653497+0000","last_unstale":"2023-11-28T13:00:18.497368+0000","last_undegraded":"2023-11-28T13:00:18.497368+0000","last_fullsized":"2023-11-28T13:00:18.497368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:59.591407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331153+0000","last_change":"2023-11-28T13:00:17.652105+0000","last_active":"2023-11-28T13:00:18.331153+0000","last_peered":"2023-11-28T13:00:18.331153+0000","last_clean":"2023-11-28T13:00:18.331153+0000","last_became_active":"2023-11-28T13:00:17.651804+0000","last_became_peered":"2023-11-28T13:00:17.651804+0000","last_unstale":"2023-11-28T13:00:18.331153+0000","last_undegraded":"2023-11-28T13:00:18.331153+0000","last_fullsized":"2023-11-28T13:00:18.331153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650622+0000","last_change":"2023-11-28T13:00:17.647496+0000","last_active":"2023-11-28T13:00:17.650622+0000","last_peered":"2023-11-28T13:00:17.650622+0000","last_clean":"2023-11-28T13:00:17.650622+0000","last_became_active":"2023-11-28T13:00:17.647234+0000","last_became_peered":"2023-11-28T13:00:17.647234+0000","last_unstale":"2023-11-28T13:00:17.650622+0000","last_undegraded":"2023-11-28T13:00:17.650622+0000","last_fullsized":"2023-11-28T13:00:17.650622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215336+0000","last_change":"2023-11-28T13:00:17.645773+0000","last_active":"2023-11-28T13:00:19.215336+0000","last_peered":"2023-11-28T13:00:19.215336+0000","last_clean":"2023-11-28T13:00:19.215336+0000","last_became_active":"2023-11-28T13:00:17.643545+0000","last_became_peered":"2023-11-28T13:00:17.643545+0000","last_unstale":"2023-11-28T13:00:19.215336+0000","last_undegraded":"2023-11-28T13:00:19.215336+0000","last_fullsized":"2023-11-28T13:00:19.215336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:36.504863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331033+0000","last_change":"2023-11-28T13:00:17.651625+0000","last_active":"2023-11-28T13:00:18.331033+0000","last_peered":"2023-11-28T13:00:18.331033+0000","last_clean":"2023-11-28T13:00:18.331033+0000","last_became_active":"2023-11-28T13:00:17.651210+0000","last_became_peered":"2023-11-28T13:00:17.651210+0000","last_unstale":"2023-11-28T13:00:18.331033+0000","last_undegraded":"2023-11-28T13:00:18.331033+0000","last_fullsized":"2023-11-28T13:00:18.331033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635880+0000","last_change":"2023-11-28T13:00:17.646690+0000","last_active":"2023-11-28T13:00:18.635880+0000","last_peered":"2023-11-28T13:00:18.635880+0000","last_clean":"2023-11-28T13:00:18.635880+0000","last_became_active":"2023-11-28T13:00:17.646438+0000","last_became_peered":"2023-11-28T13:00:17.646438+0000","last_unstale":"2023-11-28T13:00:18.635880+0000","last_undegraded":"2023-11-28T13:00:18.635880+0000","last_fullsized":"2023-11-28T13:00:18.635880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:52.870009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636047+0000","last_change":"2023-11-28T13:00:17.641404+0000","last_active":"2023-11-28T13:00:18.636047+0000","last_peered":"2023-11-28T13:00:18.636047+0000","last_clean":"2023-11-28T13:00:18.636047+0000","last_became_active":"2023-11-28T13:00:17.641140+0000","last_became_peered":"2023-11-28T13:00:17.641140+0000","last_unstale":"2023-11-28T13:00:18.636047+0000","last_undegraded":"2023-11-28T13:00:18.636047+0000","last_fullsized":"2023-11-28T13:00:18.636047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338691+0000","last_change":"2023-11-28T13:00:17.645292+0000","last_active":"2023-11-28T13:00:18.338691+0000","last_peered":"2023-11-28T13:00:18.338691+0000","last_clean":"2023-11-28T13:00:18.338691+0000","last_became_active":"2023-11-28T13:00:17.645001+0000","last_became_peered":"2023-11-28T13:00:17.645001+0000","last_unstale":"2023-11-28T13:00:18.338691+0000","last_undegraded":"2023-11-28T13:00:18.338691+0000","last_fullsized":"2023-11-28T13:00:18.338691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497240+0000","last_change":"2023-11-28T13:00:17.245858+0000","last_active":"2023-11-28T13:00:18.497240+0000","last_peered":"2023-11-28T13:00:18.497240+0000","last_clean":"2023-11-28T13:00:18.497240+0000","last_became_active":"2023-11-28T13:00:17.245568+0000","last_became_peered":"2023-11-28T13:00:17.245568+0000","last_unstale":"2023-11-28T13:00:18.497240+0000","last_undegraded":"2023-11-28T13:00:18.497240+0000","last_fullsized":"2023-11-28T13:00:18.497240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215475+0000","last_change":"2023-11-28T13:00:17.644504+0000","last_active":"2023-11-28T13:00:19.215475+0000","last_peered":"2023-11-28T13:00:19.215475+0000","last_clean":"2023-11-28T13:00:19.215475+0000","last_became_active":"2023-11-28T13:00:17.644099+0000","last_became_peered":"2023-11-28T13:00:17.644099+0000","last_unstale":"2023-11-28T13:00:19.215475+0000","last_undegraded":"2023-11-28T13:00:19.215475+0000","last_fullsized":"2023-11-28T13:00:19.215475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635846+0000","last_change":"2023-11-28T13:00:17.236124+0000","last_active":"2023-11-28T13:00:18.635846+0000","last_peered":"2023-11-28T13:00:18.635846+0000","last_clean":"2023-11-28T13:00:18.635846+0000","last_became_active":"2023-11-28T13:00:17.235500+0000","last_became_peered":"2023-11-28T13:00:17.235500+0000","last_unstale":"2023-11-28T13:00:18.635846+0000","last_undegraded":"2023-11-28T13:00:18.635846+0000","last_fullsized":"2023-11-28T13:00:18.635846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:19.252999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648987+0000","last_change":"2023-11-28T13:00:17.642697+0000","last_active":"2023-11-28T13:00:17.648987+0000","last_peered":"2023-11-28T13:00:17.648987+0000","last_clean":"2023-11-28T13:00:17.648987+0000","last_became_active":"2023-11-28T13:00:17.642400+0000","last_became_peered":"2023-11-28T13:00:17.642400+0000","last_unstale":"2023-11-28T13:00:17.648987+0000","last_undegraded":"2023-11-28T13:00:17.648987+0000","last_fullsized":"2023-11-28T13:00:17.648987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215390+0000","last_change":"2023-11-28T13:00:17.240445+0000","last_active":"2023-11-28T13:00:19.215390+0000","last_peered":"2023-11-28T13:00:19.215390+0000","last_clean":"2023-11-28T13:00:19.215390+0000","last_became_active":"2023-11-28T13:00:17.240095+0000","last_became_peered":"2023-11-28T13:00:17.240095+0000","last_unstale":"2023-11-28T13:00:19.215390+0000","last_undegraded":"2023-11-28T13:00:19.215390+0000","last_fullsized":"2023-11-28T13:00:19.215390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330869+0000","last_change":"2023-11-28T13:00:17.644329+0000","last_active":"2023-11-28T13:00:18.330869+0000","last_peered":"2023-11-28T13:00:18.330869+0000","last_clean":"2023-11-28T13:00:18.330869+0000","last_became_active":"2023-11-28T13:00:17.644046+0000","last_became_peered":"2023-11-28T13:00:17.644046+0000","last_unstale":"2023-11-28T13:00:18.330869+0000","last_undegraded":"2023-11-28T13:00:18.330869+0000","last_fullsized":"2023-11-28T13:00:18.330869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.849042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631440+0000","last_change":"2023-11-28T13:00:17.242514+0000","last_active":"2023-11-28T13:00:17.631440+0000","last_peered":"2023-11-28T13:00:17.631440+0000","last_clean":"2023-11-28T13:00:17.631440+0000","last_became_active":"2023-11-28T13:00:17.242258+0000","last_became_peered":"2023-11-28T13:00:17.242258+0000","last_unstale":"2023-11-28T13:00:17.631440+0000","last_undegraded":"2023-11-28T13:00:17.631440+0000","last_fullsized":"2023-11-28T13:00:17.631440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215295+0000","last_change":"2023-11-28T13:00:17.642442+0000","last_active":"2023-11-28T13:00:19.215295+0000","last_peered":"2023-11-28T13:00:19.215295+0000","last_clean":"2023-11-28T13:00:19.215295+0000","last_became_active":"2023-11-28T13:00:17.642162+0000","last_became_peered":"2023-11-28T13:00:17.642162+0000","last_unstale":"2023-11-28T13:00:19.215295+0000","last_undegraded":"2023-11-28T13:00:19.215295+0000","last_fullsized":"2023-11-28T13:00:19.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338768+0000","last_change":"2023-11-28T13:00:17.238867+0000","last_active":"2023-11-28T13:00:18.338768+0000","last_peered":"2023-11-28T13:00:18.338768+0000","last_clean":"2023-11-28T13:00:18.338768+0000","last_became_active":"2023-11-28T13:00:17.238545+0000","last_became_peered":"2023-11-28T13:00:17.238545+0000","last_unstale":"2023-11-28T13:00:18.338768+0000","last_undegraded":"2023-11-28T13:00:18.338768+0000","last_fullsized":"2023-11-28T13:00:18.338768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215335+0000","last_change":"2023-11-28T13:00:17.645527+0000","last_active":"2023-11-28T13:00:19.215335+0000","last_peered":"2023-11-28T13:00:19.215335+0000","last_clean":"2023-11-28T13:00:19.215335+0000","last_became_active":"2023-11-28T13:00:17.644057+0000","last_became_peered":"2023-11-28T13:00:17.644057+0000","last_unstale":"2023-11-28T13:00:19.215335+0000","last_undegraded":"2023-11-28T13:00:19.215335+0000","last_fullsized":"2023-11-28T13:00:19.215335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:55.013485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331175+0000","last_change":"2023-11-28T13:00:17.242375+0000","last_active":"2023-11-28T13:00:18.331175+0000","last_peered":"2023-11-28T13:00:18.331175+0000","last_clean":"2023-11-28T13:00:18.331175+0000","last_became_active":"2023-11-28T13:00:17.242010+0000","last_became_peered":"2023-11-28T13:00:17.242010+0000","last_unstale":"2023-11-28T13:00:18.331175+0000","last_undegraded":"2023-11-28T13:00:18.331175+0000","last_fullsized":"2023-11-28T13:00:18.331175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.522601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635882+0000","last_change":"2023-11-28T13:00:17.647427+0000","last_active":"2023-11-28T13:00:18.635882+0000","last_peered":"2023-11-28T13:00:18.635882+0000","last_clean":"2023-11-28T13:00:18.635882+0000","last_became_active":"2023-11-28T13:00:17.646853+0000","last_became_peered":"2023-11-28T13:00:17.646853+0000","last_unstale":"2023-11-28T13:00:18.635882+0000","last_undegraded":"2023-11-28T13:00:18.635882+0000","last_fullsized":"2023-11-28T13:00:18.635882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497407+0000","last_change":"2023-11-28T13:00:17.243823+0000","last_active":"2023-11-28T13:00:18.497407+0000","last_peered":"2023-11-28T13:00:18.497407+0000","last_clean":"2023-11-28T13:00:18.497407+0000","last_became_active":"2023-11-28T13:00:17.241238+0000","last_became_peered":"2023-11-28T13:00:17.241238+0000","last_unstale":"2023-11-28T13:00:18.497407+0000","last_undegraded":"2023-11-28T13:00:18.497407+0000","last_fullsized":"2023-11-28T13:00:18.497407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497280+0000","last_change":"2023-11-28T13:00:17.649986+0000","last_active":"2023-11-28T13:00:18.497280+0000","last_peered":"2023-11-28T13:00:18.497280+0000","last_clean":"2023-11-28T13:00:18.497280+0000","last_became_active":"2023-11-28T13:00:17.649402+0000","last_became_peered":"2023-11-28T13:00:17.649402+0000","last_unstale":"2023-11-28T13:00:18.497280+0000","last_undegraded":"2023-11-28T13:00:18.497280+0000","last_fullsized":"2023-11-28T13:00:18.497280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:15.471790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331132+0000","last_change":"2023-11-28T13:00:17.247047+0000","last_active":"2023-11-28T13:00:18.331132+0000","last_peered":"2023-11-28T13:00:18.331132+0000","last_clean":"2023-11-28T13:00:18.331132+0000","last_became_active":"2023-11-28T13:00:17.246742+0000","last_became_peered":"2023-11-28T13:00:17.246742+0000","last_unstale":"2023-11-28T13:00:18.331132+0000","last_undegraded":"2023-11-28T13:00:18.331132+0000","last_fullsized":"2023-11-28T13:00:18.331132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:50.559956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215527+0000","last_change":"2023-11-28T13:00:17.641605+0000","last_active":"2023-11-28T13:00:19.215527+0000","last_peered":"2023-11-28T13:00:19.215527+0000","last_clean":"2023-11-28T13:00:19.215527+0000","last_became_active":"2023-11-28T13:00:17.641350+0000","last_became_peered":"2023-11-28T13:00:17.641350+0000","last_unstale":"2023-11-28T13:00:19.215527+0000","last_undegraded":"2023-11-28T13:00:19.215527+0000","last_fullsized":"2023-11-28T13:00:19.215527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.630517+0000","last_change":"2023-11-28T13:00:17.240432+0000","last_active":"2023-11-28T13:00:17.630517+0000","last_peered":"2023-11-28T13:00:17.630517+0000","last_clean":"2023-11-28T13:00:17.630517+0000","last_became_active":"2023-11-28T13:00:17.239192+0000","last_became_peered":"2023-11-28T13:00:17.239192+0000","last_unstale":"2023-11-28T13:00:17.630517+0000","last_undegraded":"2023-11-28T13:00:17.630517+0000","last_fullsized":"2023-11-28T13:00:17.630517+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338702+0000","last_change":"2023-11-28T13:00:17.648774+0000","last_active":"2023-11-28T13:00:18.338702+0000","last_peered":"2023-11-28T13:00:18.338702+0000","last_clean":"2023-11-28T13:00:18.338702+0000","last_became_active":"2023-11-28T13:00:17.647403+0000","last_became_peered":"2023-11-28T13:00:17.647403+0000","last_unstale":"2023-11-28T13:00:18.338702+0000","last_undegraded":"2023-11-28T13:00:18.338702+0000","last_fullsized":"2023-11-28T13:00:18.338702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:17.592744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331091+0000","last_change":"2023-11-28T13:00:17.245977+0000","last_active":"2023-11-28T13:00:18.331091+0000","last_peered":"2023-11-28T13:00:18.331091+0000","last_clean":"2023-11-28T13:00:18.331091+0000","last_became_active":"2023-11-28T13:00:17.245651+0000","last_became_peered":"2023-11-28T13:00:17.245651+0000","last_unstale":"2023-11-28T13:00:18.331091+0000","last_undegraded":"2023-11-28T13:00:18.331091+0000","last_fullsized":"2023-11-28T13:00:18.331091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:30.760573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649006+0000","last_change":"2023-11-28T13:00:17.644029+0000","last_active":"2023-11-28T13:00:17.649006+0000","last_peered":"2023-11-28T13:00:17.649006+0000","last_clean":"2023-11-28T13:00:17.649006+0000","last_became_active":"2023-11-28T13:00:17.643789+0000","last_became_peered":"2023-11-28T13:00:17.643789+0000","last_unstale":"2023-11-28T13:00:17.649006+0000","last_undegraded":"2023-11-28T13:00:17.649006+0000","last_fullsized":"2023-11-28T13:00:17.649006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330786+0000","last_change":"2023-11-28T13:00:17.240716+0000","last_active":"2023-11-28T13:00:18.330786+0000","last_peered":"2023-11-28T13:00:18.330786+0000","last_clean":"2023-11-28T13:00:18.330786+0000","last_became_active":"2023-11-28T13:00:17.239928+0000","last_became_peered":"2023-11-28T13:00:17.239928+0000","last_unstale":"2023-11-28T13:00:18.330786+0000","last_undegraded":"2023-11-28T13:00:18.330786+0000","last_fullsized":"2023-11-28T13:00:18.330786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:32.086174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215680+0000","last_change":"2023-11-28T13:00:17.643923+0000","last_active":"2023-11-28T13:00:19.215680+0000","last_peered":"2023-11-28T13:00:19.215680+0000","last_clean":"2023-11-28T13:00:19.215680+0000","last_became_active":"2023-11-28T13:00:17.643579+0000","last_became_peered":"2023-11-28T13:00:17.643579+0000","last_unstale":"2023-11-28T13:00:19.215680+0000","last_undegraded":"2023-11-28T13:00:19.215680+0000","last_fullsized":"2023-11-28T13:00:19.215680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331252+0000","last_change":"2023-11-28T13:00:17.247259+0000","last_active":"2023-11-28T13:00:18.331252+0000","last_peered":"2023-11-28T13:00:18.331252+0000","last_clean":"2023-11-28T13:00:18.331252+0000","last_became_active":"2023-11-28T13:00:17.246943+0000","last_became_peered":"2023-11-28T13:00:17.246943+0000","last_unstale":"2023-11-28T13:00:18.331252+0000","last_undegraded":"2023-11-28T13:00:18.331252+0000","last_fullsized":"2023-11-28T13:00:18.331252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:02.643388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338621+0000","last_change":"2023-11-28T13:00:17.644322+0000","last_active":"2023-11-28T13:00:18.338621+0000","last_peered":"2023-11-28T13:00:18.338621+0000","last_clean":"2023-11-28T13:00:18.338621+0000","last_became_active":"2023-11-28T13:00:17.644054+0000","last_became_peered":"2023-11-28T13:00:17.644054+0000","last_unstale":"2023-11-28T13:00:18.338621+0000","last_undegraded":"2023-11-28T13:00:18.338621+0000","last_fullsized":"2023-11-28T13:00:18.338621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:46.098419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331230+0000","last_change":"2023-11-28T13:00:17.247274+0000","last_active":"2023-11-28T13:00:18.331230+0000","last_peered":"2023-11-28T13:00:18.331230+0000","last_clean":"2023-11-28T13:00:18.331230+0000","last_became_active":"2023-11-28T13:00:17.246851+0000","last_became_peered":"2023-11-28T13:00:17.246851+0000","last_unstale":"2023-11-28T13:00:18.331230+0000","last_undegraded":"2023-11-28T13:00:18.331230+0000","last_fullsized":"2023-11-28T13:00:18.331230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:17.621082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497137+0000","last_change":"2023-11-28T13:00:17.640395+0000","last_active":"2023-11-28T13:00:18.497137+0000","last_peered":"2023-11-28T13:00:18.497137+0000","last_clean":"2023-11-28T13:00:18.497137+0000","last_became_active":"2023-11-28T13:00:17.640195+0000","last_became_peered":"2023-11-28T13:00:17.640195+0000","last_unstale":"2023-11-28T13:00:18.497137+0000","last_undegraded":"2023-11-28T13:00:18.497137+0000","last_fullsized":"2023-11-28T13:00:18.497137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:14.878366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331208+0000","last_change":"2023-11-28T13:00:17.240698+0000","last_active":"2023-11-28T13:00:18.331208+0000","last_peered":"2023-11-28T13:00:18.331208+0000","last_clean":"2023-11-28T13:00:18.331208+0000","last_became_active":"2023-11-28T13:00:17.240387+0000","last_became_peered":"2023-11-28T13:00:17.240387+0000","last_unstale":"2023-11-28T13:00:18.331208+0000","last_undegraded":"2023-11-28T13:00:18.331208+0000","last_fullsized":"2023-11-28T13:00:18.331208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:05.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330832+0000","last_change":"2023-11-28T13:00:17.647318+0000","last_active":"2023-11-28T13:00:18.330832+0000","last_peered":"2023-11-28T13:00:18.330832+0000","last_clean":"2023-11-28T13:00:18.330832+0000","last_became_active":"2023-11-28T13:00:17.647043+0000","last_became_peered":"2023-11-28T13:00:17.647043+0000","last_unstale":"2023-11-28T13:00:18.330832+0000","last_undegraded":"2023-11-28T13:00:18.330832+0000","last_fullsized":"2023-11-28T13:00:18.330832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:38.321116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338680+0000","last_change":"2023-11-28T13:00:17.235843+0000","last_active":"2023-11-28T13:00:18.338680+0000","last_peered":"2023-11-28T13:00:18.338680+0000","last_clean":"2023-11-28T13:00:18.338680+0000","last_became_active":"2023-11-28T13:00:17.235495+0000","last_became_peered":"2023-11-28T13:00:17.235495+0000","last_unstale":"2023-11-28T13:00:18.338680+0000","last_undegraded":"2023-11-28T13:00:18.338680+0000","last_fullsized":"2023-11-28T13:00:18.338680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648938+0000","last_change":"2023-11-28T13:00:17.640727+0000","last_active":"2023-11-28T13:00:17.648938+0000","last_peered":"2023-11-28T13:00:17.648938+0000","last_clean":"2023-11-28T13:00:17.648938+0000","last_became_active":"2023-11-28T13:00:17.640438+0000","last_became_peered":"2023-11-28T13:00:17.640438+0000","last_unstale":"2023-11-28T13:00:17.648938+0000","last_undegraded":"2023-11-28T13:00:17.648938+0000","last_fullsized":"2023-11-28T13:00:17.648938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215456+0000","last_change":"2023-11-28T13:00:17.240709+0000","last_active":"2023-11-28T13:00:19.215456+0000","last_peered":"2023-11-28T13:00:19.215456+0000","last_clean":"2023-11-28T13:00:19.215456+0000","last_became_active":"2023-11-28T13:00:17.239893+0000","last_became_peered":"2023-11-28T13:00:17.239893+0000","last_unstale":"2023-11-28T13:00:19.215456+0000","last_undegraded":"2023-11-28T13:00:19.215456+0000","last_fullsized":"2023-11-28T13:00:19.215456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497386+0000","last_change":"2023-11-28T13:00:17.653235+0000","last_active":"2023-11-28T13:00:18.497386+0000","last_peered":"2023-11-28T13:00:18.497386+0000","last_clean":"2023-11-28T13:00:18.497386+0000","last_became_active":"2023-11-28T13:00:17.650249+0000","last_became_peered":"2023-11-28T13:00:17.650249+0000","last_unstale":"2023-11-28T13:00:18.497386+0000","last_undegraded":"2023-11-28T13:00:18.497386+0000","last_fullsized":"2023-11-28T13:00:18.497386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330820+0000","last_change":"2023-11-28T13:00:17.245224+0000","last_active":"2023-11-28T13:00:18.330820+0000","last_peered":"2023-11-28T13:00:18.330820+0000","last_clean":"2023-11-28T13:00:18.330820+0000","last_became_active":"2023-11-28T13:00:17.244913+0000","last_became_peered":"2023-11-28T13:00:17.244913+0000","last_unstale":"2023-11-28T13:00:18.330820+0000","last_undegraded":"2023-11-28T13:00:18.330820+0000","last_fullsized":"2023-11-28T13:00:18.330820+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497463+0000","last_change":"2023-11-28T13:00:17.643015+0000","last_active":"2023-11-28T13:00:18.497463+0000","last_peered":"2023-11-28T13:00:18.497463+0000","last_clean":"2023-11-28T13:00:18.497463+0000","last_became_active":"2023-11-28T13:00:17.642470+0000","last_became_peered":"2023-11-28T13:00:17.642470+0000","last_unstale":"2023-11-28T13:00:18.497463+0000","last_undegraded":"2023-11-28T13:00:18.497463+0000","last_fullsized":"2023-11-28T13:00:18.497463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:41.679867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497180+0000","last_change":"2023-11-28T13:00:17.244207+0000","last_active":"2023-11-28T13:00:18.497180+0000","last_peered":"2023-11-28T13:00:18.497180+0000","last_clean":"2023-11-28T13:00:18.497180+0000","last_became_active":"2023-11-28T13:00:17.243861+0000","last_became_peered":"2023-11-28T13:00:17.243861+0000","last_unstale":"2023-11-28T13:00:18.497180+0000","last_undegraded":"2023-11-28T13:00:18.497180+0000","last_fullsized":"2023-11-28T13:00:18.497180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497504+0000","last_change":"2023-11-28T13:00:17.651269+0000","last_active":"2023-11-28T13:00:18.497504+0000","last_peered":"2023-11-28T13:00:18.497504+0000","last_clean":"2023-11-28T13:00:18.497504+0000","last_became_active":"2023-11-28T13:00:17.650883+0000","last_became_peered":"2023-11-28T13:00:17.650883+0000","last_unstale":"2023-11-28T13:00:18.497504+0000","last_undegraded":"2023-11-28T13:00:18.497504+0000","last_fullsized":"2023-11-28T13:00:18.497504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215191+0000","last_change":"2023-11-28T13:00:17.238617+0000","last_active":"2023-11-28T13:00:19.215191+0000","last_peered":"2023-11-28T13:00:19.215191+0000","last_clean":"2023-11-28T13:00:19.215191+0000","last_became_active":"2023-11-28T13:00:17.238344+0000","last_became_peered":"2023-11-28T13:00:17.238344+0000","last_unstale":"2023-11-28T13:00:19.215191+0000","last_undegraded":"2023-11-28T13:00:19.215191+0000","last_fullsized":"2023-11-28T13:00:19.215191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:46.225565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636017+0000","last_change":"2023-11-28T13:00:17.649564+0000","last_active":"2023-11-28T13:00:18.636017+0000","last_peered":"2023-11-28T13:00:18.636017+0000","last_clean":"2023-11-28T13:00:18.636017+0000","last_became_active":"2023-11-28T13:00:17.649312+0000","last_became_peered":"2023-11-28T13:00:17.649312+0000","last_unstale":"2023-11-28T13:00:18.636017+0000","last_undegraded":"2023-11-28T13:00:18.636017+0000","last_fullsized":"2023-11-28T13:00:18.636017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634764+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:00:17.634764+0000","last_peered":"2023-11-28T13:00:17.634764+0000","last_clean":"2023-11-28T13:00:17.634764+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:00:17.634764+0000","last_undegraded":"2023-11-28T13:00:17.634764+0000","last_fullsized":"2023-11-28T13:00:17.634764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338730+0000","last_change":"2023-11-28T13:00:17.240503+0000","last_active":"2023-11-28T13:00:18.338730+0000","last_peered":"2023-11-28T13:00:18.338730+0000","last_clean":"2023-11-28T13:00:18.338730+0000","last_became_active":"2023-11-28T13:00:17.240256+0000","last_became_peered":"2023-11-28T13:00:17.240256+0000","last_unstale":"2023-11-28T13:00:18.338730+0000","last_undegraded":"2023-11-28T13:00:18.338730+0000","last_fullsized":"2023-11-28T13:00:18.338730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635946+0000","last_change":"2023-11-28T13:00:17.645964+0000","last_active":"2023-11-28T13:00:18.635946+0000","last_peered":"2023-11-28T13:00:18.635946+0000","last_clean":"2023-11-28T13:00:18.635946+0000","last_became_active":"2023-11-28T13:00:17.645384+0000","last_became_peered":"2023-11-28T13:00:17.645384+0000","last_unstale":"2023-11-28T13:00:18.635946+0000","last_undegraded":"2023-11-28T13:00:18.635946+0000","last_fullsized":"2023-11-28T13:00:18.635946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634535+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:00:17.634535+0000","last_peered":"2023-11-28T13:00:17.634535+0000","last_clean":"2023-11-28T13:00:17.634535+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:00:17.634535+0000","last_undegraded":"2023-11-28T13:00:17.634535+0000","last_fullsized":"2023-11-28T13:00:17.634535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497313+0000","last_change":"2023-11-28T13:00:17.245773+0000","last_active":"2023-11-28T13:00:18.497313+0000","last_peered":"2023-11-28T13:00:18.497313+0000","last_clean":"2023-11-28T13:00:18.497313+0000","last_became_active":"2023-11-28T13:00:17.245409+0000","last_became_peered":"2023-11-28T13:00:17.245409+0000","last_unstale":"2023-11-28T13:00:18.497313+0000","last_undegraded":"2023-11-28T13:00:18.497313+0000","last_fullsized":"2023-11-28T13:00:18.497313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:00.519142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636216+0000","last_change":"2023-11-28T13:00:17.641395+0000","last_active":"2023-11-28T13:00:18.636216+0000","last_peered":"2023-11-28T13:00:18.636216+0000","last_clean":"2023-11-28T13:00:18.636216+0000","last_became_active":"2023-11-28T13:00:17.641149+0000","last_became_peered":"2023-11-28T13:00:17.641149+0000","last_unstale":"2023-11-28T13:00:18.636216+0000","last_undegraded":"2023-11-28T13:00:18.636216+0000","last_fullsized":"2023-11-28T13:00:18.636216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.628022+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:00:17.628022+0000","last_peered":"2023-11-28T13:00:17.628022+0000","last_clean":"2023-11-28T13:00:17.628022+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:00:17.628022+0000","last_undegraded":"2023-11-28T13:00:17.628022+0000","last_fullsized":"2023-11-28T13:00:17.628022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635610+0000","last_change":"2023-11-28T13:00:17.237802+0000","last_active":"2023-11-28T13:00:18.635610+0000","last_peered":"2023-11-28T13:00:18.635610+0000","last_clean":"2023-11-28T13:00:18.635610+0000","last_became_active":"2023-11-28T13:00:17.237487+0000","last_became_peered":"2023-11-28T13:00:17.237487+0000","last_unstale":"2023-11-28T13:00:18.635610+0000","last_undegraded":"2023-11-28T13:00:18.635610+0000","last_fullsized":"2023-11-28T13:00:18.635610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635877+0000","last_change":"2023-11-28T13:00:17.644511+0000","last_active":"2023-11-28T13:00:18.635877+0000","last_peered":"2023-11-28T13:00:18.635877+0000","last_clean":"2023-11-28T13:00:18.635877+0000","last_became_active":"2023-11-28T13:00:17.644095+0000","last_became_peered":"2023-11-28T13:00:17.644095+0000","last_unstale":"2023-11-28T13:00:18.635877+0000","last_undegraded":"2023-11-28T13:00:18.635877+0000","last_fullsized":"2023-11-28T13:00:18.635877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.636163+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:00:17.636163+0000","last_peered":"2023-11-28T13:00:17.636163+0000","last_clean":"2023-11-28T13:00:17.636163+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:00:17.636163+0000","last_undegraded":"2023-11-28T13:00:17.636163+0000","last_fullsized":"2023-11-28T13:00:17.636163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633158+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:00:17.633158+0000","last_peered":"2023-11-28T13:00:17.633158+0000","last_clean":"2023-11-28T13:00:17.633158+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:00:17.633158+0000","last_undegraded":"2023-11-28T13:00:17.633158+0000","last_fullsized":"2023-11-28T13:00:17.633158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215311+0000","last_change":"2023-11-28T13:00:17.239373+0000","last_active":"2023-11-28T13:00:19.215311+0000","last_peered":"2023-11-28T13:00:19.215311+0000","last_clean":"2023-11-28T13:00:19.215311+0000","last_became_active":"2023-11-28T13:00:17.238864+0000","last_became_peered":"2023-11-28T13:00:17.238864+0000","last_unstale":"2023-11-28T13:00:19.215311+0000","last_undegraded":"2023-11-28T13:00:19.215311+0000","last_fullsized":"2023-11-28T13:00:19.215311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:17.763933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330851+0000","last_change":"2023-11-28T13:00:17.649771+0000","last_active":"2023-11-28T13:00:18.330851+0000","last_peered":"2023-11-28T13:00:18.330851+0000","last_clean":"2023-11-28T13:00:18.330851+0000","last_became_active":"2023-11-28T13:00:17.649475+0000","last_became_peered":"2023-11-28T13:00:17.649475+0000","last_unstale":"2023-11-28T13:00:18.330851+0000","last_undegraded":"2023-11-28T13:00:18.330851+0000","last_fullsized":"2023-11-28T13:00:18.330851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633524+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:00:17.633524+0000","last_peered":"2023-11-28T13:00:17.633524+0000","last_clean":"2023-11-28T13:00:17.633524+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:00:17.633524+0000","last_undegraded":"2023-11-28T13:00:17.633524+0000","last_fullsized":"2023-11-28T13:00:17.633524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.630289+0000","last_change":"2023-11-28T13:00:17.251262+0000","last_active":"2023-11-28T13:00:17.630289+0000","last_peered":"2023-11-28T13:00:17.630289+0000","last_clean":"2023-11-28T13:00:17.630289+0000","last_became_active":"2023-11-28T13:00:17.250948+0000","last_became_peered":"2023-11-28T13:00:17.250948+0000","last_unstale":"2023-11-28T13:00:17.630289+0000","last_undegraded":"2023-11-28T13:00:17.630289+0000","last_fullsized":"2023-11-28T13:00:17.630289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:37.311866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648968+0000","last_change":"2023-11-28T13:00:17.640685+0000","last_active":"2023-11-28T13:00:17.648968+0000","last_peered":"2023-11-28T13:00:17.648968+0000","last_clean":"2023-11-28T13:00:17.648968+0000","last_became_active":"2023-11-28T13:00:17.640420+0000","last_became_peered":"2023-11-28T13:00:17.640420+0000","last_unstale":"2023-11-28T13:00:17.648968+0000","last_undegraded":"2023-11-28T13:00:17.648968+0000","last_fullsized":"2023-11-28T13:00:17.648968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.632566+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:00:17.632566+0000","last_peered":"2023-11-28T13:00:17.632566+0000","last_clean":"2023-11-28T13:00:17.632566+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:00:17.632566+0000","last_undegraded":"2023-11-28T13:00:17.632566+0000","last_fullsized":"2023-11-28T13:00:17.632566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636224+0000","last_change":"2023-11-28T13:00:17.237999+0000","last_active":"2023-11-28T13:00:18.636224+0000","last_peered":"2023-11-28T13:00:18.636224+0000","last_clean":"2023-11-28T13:00:18.636224+0000","last_became_active":"2023-11-28T13:00:17.237625+0000","last_became_peered":"2023-11-28T13:00:17.237625+0000","last_unstale":"2023-11-28T13:00:18.636224+0000","last_undegraded":"2023-11-28T13:00:18.636224+0000","last_fullsized":"2023-11-28T13:00:18.636224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650614+0000","last_change":"2023-11-28T13:00:17.644943+0000","last_active":"2023-11-28T13:00:17.650614+0000","last_peered":"2023-11-28T13:00:17.650614+0000","last_clean":"2023-11-28T13:00:17.650614+0000","last_became_active":"2023-11-28T13:00:17.644679+0000","last_became_peered":"2023-11-28T13:00:17.644679+0000","last_unstale":"2023-11-28T13:00:17.650614+0000","last_undegraded":"2023-11-28T13:00:17.650614+0000","last_fullsized":"2023-11-28T13:00:17.650614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215660+0000","last_change":"2023-11-28T13:00:17.239284+0000","last_active":"2023-11-28T13:00:19.215660+0000","last_peered":"2023-11-28T13:00:19.215660+0000","last_clean":"2023-11-28T13:00:19.215660+0000","last_became_active":"2023-11-28T13:00:17.238997+0000","last_became_peered":"2023-11-28T13:00:17.238997+0000","last_unstale":"2023-11-28T13:00:19.215660+0000","last_undegraded":"2023-11-28T13:00:19.215660+0000","last_fullsized":"2023-11-28T13:00:19.215660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:55.423382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633198+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:00:17.633198+0000","last_peered":"2023-11-28T13:00:17.633198+0000","last_clean":"2023-11-28T13:00:17.633198+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:00:17.633198+0000","last_undegraded":"2023-11-28T13:00:17.633198+0000","last_fullsized":"2023-11-28T13:00:17.633198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215108+0000","last_change":"2023-11-28T13:00:17.646149+0000","last_active":"2023-11-28T13:00:19.215108+0000","last_peered":"2023-11-28T13:00:19.215108+0000","last_clean":"2023-11-28T13:00:19.215108+0000","last_became_active":"2023-11-28T13:00:17.645893+0000","last_became_peered":"2023-11-28T13:00:17.645893+0000","last_unstale":"2023-11-28T13:00:19.215108+0000","last_undegraded":"2023-11-28T13:00:19.215108+0000","last_fullsized":"2023-11-28T13:00:19.215108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:24.466207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633216+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:00:17.633216+0000","last_peered":"2023-11-28T13:00:17.633216+0000","last_clean":"2023-11-28T13:00:17.633216+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:00:17.633216+0000","last_undegraded":"2023-11-28T13:00:17.633216+0000","last_fullsized":"2023-11-28T13:00:17.633216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631002+0000","last_change":"2023-11-28T13:00:17.240379+0000","last_active":"2023-11-28T13:00:17.631002+0000","last_peered":"2023-11-28T13:00:17.631002+0000","last_clean":"2023-11-28T13:00:17.631002+0000","last_became_active":"2023-11-28T13:00:17.239505+0000","last_became_peered":"2023-11-28T13:00:17.239505+0000","last_unstale":"2023-11-28T13:00:17.631002+0000","last_undegraded":"2023-11-28T13:00:17.631002+0000","last_fullsized":"2023-11-28T13:00:17.631002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497157+0000","last_change":"2023-11-28T13:00:17.646003+0000","last_active":"2023-11-28T13:00:18.497157+0000","last_peered":"2023-11-28T13:00:18.497157+0000","last_clean":"2023-11-28T13:00:18.497157+0000","last_became_active":"2023-11-28T13:00:17.645727+0000","last_became_peered":"2023-11-28T13:00:17.645727+0000","last_unstale":"2023-11-28T13:00:18.497157+0000","last_undegraded":"2023-11-28T13:00:18.497157+0000","last_fullsized":"2023-11-28T13:00:18.497157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:30.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656358+0000","last_change":"2023-11-28T13:00:17.251398+0000","last_active":"2023-11-28T13:00:17.656358+0000","last_peered":"2023-11-28T13:00:17.656358+0000","last_clean":"2023-11-28T13:00:17.656358+0000","last_became_active":"2023-11-28T13:00:17.251054+0000","last_became_peered":"2023-11-28T13:00:17.251054+0000","last_unstale":"2023-11-28T13:00:17.656358+0000","last_undegraded":"2023-11-28T13:00:17.656358+0000","last_fullsized":"2023-11-28T13:00:17.656358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217848+0000","last_change":"2023-11-28T13:00:17.641355+0000","last_active":"2023-11-28T13:00:19.217848+0000","last_peered":"2023-11-28T13:00:19.217848+0000","last_clean":"2023-11-28T13:00:19.217848+0000","last_became_active":"2023-11-28T13:00:17.641093+0000","last_became_peered":"2023-11-28T13:00:17.641093+0000","last_unstale":"2023-11-28T13:00:19.217848+0000","last_undegraded":"2023-11-28T13:00:19.217848+0000","last_fullsized":"2023-11-28T13:00:19.217848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:41.816686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338665+0000","last_change":"2023-11-28T13:00:17.246896+0000","last_active":"2023-11-28T13:00:18.338665+0000","last_peered":"2023-11-28T13:00:18.338665+0000","last_clean":"2023-11-28T13:00:18.338665+0000","last_became_active":"2023-11-28T13:00:17.246604+0000","last_became_peered":"2023-11-28T13:00:17.246604+0000","last_unstale":"2023-11-28T13:00:18.338665+0000","last_undegraded":"2023-11-28T13:00:18.338665+0000","last_fullsized":"2023-11-28T13:00:18.338665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497332+0000","last_change":"2023-11-28T13:00:17.648973+0000","last_active":"2023-11-28T13:00:18.497332+0000","last_peered":"2023-11-28T13:00:18.497332+0000","last_clean":"2023-11-28T13:00:18.497332+0000","last_became_active":"2023-11-28T13:00:17.648323+0000","last_became_peered":"2023-11-28T13:00:17.648323+0000","last_unstale":"2023-11-28T13:00:18.497332+0000","last_undegraded":"2023-11-28T13:00:18.497332+0000","last_fullsized":"2023-11-28T13:00:18.497332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338744+0000","last_change":"2023-11-28T13:00:17.247013+0000","last_active":"2023-11-28T13:00:18.338744+0000","last_peered":"2023-11-28T13:00:18.338744+0000","last_clean":"2023-11-28T13:00:18.338744+0000","last_became_active":"2023-11-28T13:00:17.246645+0000","last_became_peered":"2023-11-28T13:00:17.246645+0000","last_unstale":"2023-11-28T13:00:18.338744+0000","last_undegraded":"2023-11-28T13:00:18.338744+0000","last_fullsized":"2023-11-28T13:00:18.338744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497261+0000","last_change":"2023-11-28T13:00:17.649078+0000","last_active":"2023-11-28T13:00:18.497261+0000","last_peered":"2023-11-28T13:00:18.497261+0000","last_clean":"2023-11-28T13:00:18.497261+0000","last_became_active":"2023-11-28T13:00:17.648432+0000","last_became_peered":"2023-11-28T13:00:17.648432+0000","last_unstale":"2023-11-28T13:00:18.497261+0000","last_undegraded":"2023-11-28T13:00:18.497261+0000","last_fullsized":"2023-11-28T13:00:18.497261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215495+0000","last_change":"2023-11-28T13:00:17.234074+0000","last_active":"2023-11-28T13:00:19.215495+0000","last_peered":"2023-11-28T13:00:19.215495+0000","last_clean":"2023-11-28T13:00:19.215495+0000","last_became_active":"2023-11-28T13:00:17.233732+0000","last_became_peered":"2023-11-28T13:00:17.233732+0000","last_unstale":"2023-11-28T13:00:19.215495+0000","last_undegraded":"2023-11-28T13:00:19.215495+0000","last_fullsized":"2023-11-28T13:00:19.215495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:36.950864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215231+0000","last_change":"2023-11-28T13:00:17.641105+0000","last_active":"2023-11-28T13:00:19.215231+0000","last_peered":"2023-11-28T13:00:19.215231+0000","last_clean":"2023-11-28T13:00:19.215231+0000","last_became_active":"2023-11-28T13:00:17.640854+0000","last_became_peered":"2023-11-28T13:00:17.640854+0000","last_unstale":"2023-11-28T13:00:19.215231+0000","last_undegraded":"2023-11-28T13:00:19.215231+0000","last_fullsized":"2023-11-28T13:00:19.215231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652233+0000","last_change":"2023-11-28T13:00:17.242508+0000","last_active":"2023-11-28T13:00:17.652233+0000","last_peered":"2023-11-28T13:00:17.652233+0000","last_clean":"2023-11-28T13:00:17.652233+0000","last_became_active":"2023-11-28T13:00:17.242230+0000","last_became_peered":"2023-11-28T13:00:17.242230+0000","last_unstale":"2023-11-28T13:00:17.652233+0000","last_undegraded":"2023-11-28T13:00:17.652233+0000","last_fullsized":"2023-11-28T13:00:17.652233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497079+0000","last_change":"2023-11-28T13:00:17.650868+0000","last_active":"2023-11-28T13:00:18.497079+0000","last_peered":"2023-11-28T13:00:18.497079+0000","last_clean":"2023-11-28T13:00:18.497079+0000","last_became_active":"2023-11-28T13:00:17.649842+0000","last_became_peered":"2023-11-28T13:00:17.649842+0000","last_unstale":"2023-11-28T13:00:18.497079+0000","last_undegraded":"2023-11-28T13:00:18.497079+0000","last_fullsized":"2023-11-28T13:00:18.497079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330807+0000","last_change":"2023-11-28T13:00:17.641330+0000","last_active":"2023-11-28T13:00:18.330807+0000","last_peered":"2023-11-28T13:00:18.330807+0000","last_clean":"2023-11-28T13:00:18.330807+0000","last_became_active":"2023-11-28T13:00:17.641029+0000","last_became_peered":"2023-11-28T13:00:17.641029+0000","last_unstale":"2023-11-28T13:00:18.330807+0000","last_undegraded":"2023-11-28T13:00:18.330807+0000","last_fullsized":"2023-11-28T13:00:18.330807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:28.607875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215202+0000","last_change":"2023-11-28T13:00:17.643816+0000","last_active":"2023-11-28T13:00:19.215202+0000","last_peered":"2023-11-28T13:00:19.215202+0000","last_clean":"2023-11-28T13:00:19.215202+0000","last_became_active":"2023-11-28T13:00:17.643260+0000","last_became_peered":"2023-11-28T13:00:17.643260+0000","last_unstale":"2023-11-28T13:00:19.215202+0000","last_undegraded":"2023-11-28T13:00:19.215202+0000","last_fullsized":"2023-11-28T13:00:19.215202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497218+0000","last_change":"2023-11-28T13:00:17.649431+0000","last_active":"2023-11-28T13:00:18.497218+0000","last_peered":"2023-11-28T13:00:18.497218+0000","last_clean":"2023-11-28T13:00:18.497218+0000","last_became_active":"2023-11-28T13:00:17.648763+0000","last_became_peered":"2023-11-28T13:00:17.648763+0000","last_unstale":"2023-11-28T13:00:18.497218+0000","last_undegraded":"2023-11-28T13:00:18.497218+0000","last_fullsized":"2023-11-28T13:00:18.497218+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:10.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215472+0000","last_change":"2023-11-28T13:00:17.649148+0000","last_active":"2023-11-28T13:00:19.215472+0000","last_peered":"2023-11-28T13:00:19.215472+0000","last_clean":"2023-11-28T13:00:19.215472+0000","last_became_active":"2023-11-28T13:00:17.648925+0000","last_became_peered":"2023-11-28T13:00:17.648925+0000","last_unstale":"2023-11-28T13:00:19.215472+0000","last_undegraded":"2023-11-28T13:00:19.215472+0000","last_fullsized":"2023-11-28T13:00:19.215472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636138+0000","last_change":"2023-11-28T13:00:17.648409+0000","last_active":"2023-11-28T13:00:18.636138+0000","last_peered":"2023-11-28T13:00:18.636138+0000","last_clean":"2023-11-28T13:00:18.636138+0000","last_became_active":"2023-11-28T13:00:17.648164+0000","last_became_peered":"2023-11-28T13:00:17.648164+0000","last_unstale":"2023-11-28T13:00:18.636138+0000","last_undegraded":"2023-11-28T13:00:18.636138+0000","last_fullsized":"2023-11-28T13:00:18.636138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":96947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7940,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363900,"statfs":{"total":96636764160,"available":96628633600,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8088,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363752,"statfs":{"total":96636764160,"available":96628482048,"internally_reserved":0,"allocated":278528,"data_stored":98461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9096,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362744,"statfs":{"total":96636764160,"available":96627449856,"internally_reserved":0,"allocated":851968,"data_stored":687022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9124,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362716,"statfs":{"total":96636764160,"available":96627421184,"internally_reserved":0,"allocated":880640,"data_stored":688989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8012,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363828,"statfs":{"total":96636764160,"available":96628559872,"internally_reserved":0,"allocated":266240,"data_stored":97493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8004,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363836,"statfs":{"total":96636764160,"available":96628568064,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8068,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363772,"statfs":{"total":96636764160,"available":96628502528,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:26.587 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:00:26.587 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T13:00:26.587 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T13:00:26.588 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T13:00:26.588 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T13:00:26.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.741+0000 7f8d7c919700 1 Processor -- start 2023-11-28T13:00:26.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.741+0000 7f8d7c919700 1 -- start start 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d7c919700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7815cef0 0x7f8d780af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d7c919700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8d7806ba90 0x7f8d780ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d7c919700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d780cc5f0 con 0x7f8d780cbdf0 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d7c919700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7815d470 con 0x7f8d7806ba90 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d7c919700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7815d750 con 0x7f8d7815cef0 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d76d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8d7806ba90 0x7f8d780ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d76d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8d7806ba90 0x7f8d780ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38406/0 (socket says 172.21.15.153:38406) 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.742+0000 7f8d76d9d700 1 -- 172.21.15.153:0/1725336160 learned_addr learned my addr 172.21.15.153:0/1725336160 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:26.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d7659c700 1 --2- 172.21.15.153:0/1725336160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7815cef0 0x7f8d780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d76d9d700 1 -- 172.21.15.153:0/1725336160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7815cef0 msgr2=0x7f8d780af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d76d9d700 1 --2- 172.21.15.153:0/1725336160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7815cef0 0x7f8d780af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d76d9d700 1 -- 172.21.15.153:0/1725336160 >> v1:172.21.15.177:6789/0 conn(0x7f8d780cbdf0 legacy=0x7f8d7806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d76d9d700 1 -- 172.21.15.153:0/1725336160 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7815da30 con 0x7f8d7806ba90 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d76d9d700 1 --2- 172.21.15.153:0/1725336160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7806ba90 0x7f8d780ca650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d150 tx=0x7f8d7000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6aeb126b331ec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d7559a700 1 -- 172.21.15.153:0/1725336160 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70010070 con 0x7f8d7806ba90 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d7559a700 1 -- 172.21.15.153:0/1725336160 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7000ddc0 con 0x7f8d7806ba90 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d7559a700 1 -- 172.21.15.153:0/1725336160 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000b310 con 0x7f8d7806ba90 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d7c919700 1 -- 172.21.15.153:0/1725336160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7806ba90 msgr2=0x7f8d780ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.743+0000 7f8d7c919700 1 --2- 172.21.15.153:0/1725336160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7806ba90 0x7f8d780ca650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d150 tx=0x7f8d7000d460 comp rx=0 tx=0).stop 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.744+0000 7f8d7c919700 1 -- 172.21.15.153:0/1725336160 shutdown_connections 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.744+0000 7f8d7c919700 1 --2- 172.21.15.153:0/1725336160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7806ba90 0x7f8d780ca650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.744+0000 7f8d7c919700 1 --2- 172.21.15.153:0/1725336160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7815cef0 0x7f8d780af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.744+0000 7f8d7c919700 1 -- 172.21.15.153:0/1725336160 >> 172.21.15.153:0/1725336160 conn(0x7f8d780d0000 msgr2=0x7f8d780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.744+0000 7f8d7c919700 1 -- 172.21.15.153:0/1725336160 shutdown_connections 2023-11-28T13:00:26.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.744+0000 7f8d7c919700 1 -- 172.21.15.153:0/1725336160 wait complete. 2023-11-28T13:00:26.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.745+0000 7f8d7c919700 1 Processor -- start 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.745+0000 7f8d7c919700 1 -- start start 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d7806ba90 0x7f8d78152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d780cbdf0 0x7f8d780c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7815cef0 0x7f8d7806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b2e30 con 0x7f8d7815cef0 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b3080 con 0x7f8d7806ba90 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b3360 con 0x7f8d780cbdf0 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7815cef0 0x7f8d7806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7815cef0 0x7f8d7806d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38412/0 (socket says 172.21.15.153:38412) 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 -- 172.21.15.153:0/3302635445 learned_addr learned my addr 172.21.15.153:0/3302635445 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7659c700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d7806ba90 0x7f8d78152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d75d9b700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d780cbdf0 0x7f8d780c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 -- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d780cbdf0 msgr2=0x7f8d780c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d780cbdf0 0x7f8d780c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 -- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d7806ba90 msgr2=0x7f8d78152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d7806ba90 0x7f8d78152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 -- 172.21.15.153:0/3302635445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7000bd40 con 0x7f8d7815cef0 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d76d9d700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7815cef0 0x7f8d7806d290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8d7806c7b0 tx=0x7f8d70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d6effd700 1 -- 172.21.15.153:0/3302635445 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70009a10 con 0x7f8d7815cef0 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d6effd700 1 -- 172.21.15.153:0/3302635445 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d70009b90 con 0x7f8d7815cef0 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7815dff0 con 0x7f8d7815cef0 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d6effd700 1 -- 172.21.15.153:0/3302635445 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000b4c0 con 0x7f8d7815cef0 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.746+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7815dd10 con 0x7f8d7815cef0 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.747+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d780cab90 con 0x7f8d7815cef0 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.748+0000 7f8d6effd700 1 -- 172.21.15.153:0/3302635445 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d70004040 con 0x7f8d7815cef0 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.748+0000 7f8d6effd700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8d580682f0 0x7f8d5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.748+0000 7f8d6effd700 1 -- 172.21.15.153:0/3302635445 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d700877e0 con 0x7f8d7815cef0 2023-11-28T13:00:26.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.748+0000 7f8d7659c700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8d580682f0 0x7f8d5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.748+0000 7f8d7659c700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8d580682f0 0x7f8d5806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d60006b20 tx=0x7f8d60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.752+0000 7f8d6effd700 1 -- 172.21.15.153:0/3302635445 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d70054040 con 0x7f8d7815cef0 2023-11-28T13:00:26.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.987+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8d780cf7f0 con 0x7f8d7815cef0 2023-11-28T13:00:26.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.990+0000 7f8d6effd700 1 -- 172.21.15.153:0/3302635445 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166386 (secure 0 0 0) 0x7f8d70057660 con 0x7f8d7815cef0 2023-11-28T13:00:26.994 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8d580682f0 msgr2=0x7f8d5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8d580682f0 0x7f8d5806a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d60006b20 tx=0x7f8d60008040 comp rx=0 tx=0).stop 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7815cef0 msgr2=0x7f8d7806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7815cef0 0x7f8d7806d290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8d7806c7b0 tx=0x7f8d70004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 shutdown_connections 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8d580682f0 0x7f8d5806a7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d7815cef0 0x7f8d7806d290 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d7806ba90 0x7f8d78152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 --2- 172.21.15.153:0/3302635445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d780cbdf0 0x7f8d780c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 >> 172.21.15.153:0/3302635445 conn(0x7f8d780d0000 msgr2=0x7f8d780caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 shutdown_connections 2023-11-28T13:00:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:26.993+0000 7f8d7c919700 1 -- 172.21.15.153:0/3302635445 wait complete. 2023-11-28T13:00:27.007 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":103291},{"type":"v1","addr":"172.21.15.153:6833","nonce":103291}]},"active_addr":"172.21.15.153:6833/103291","active_change":"2023-11-28T12:59:58.127421+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2324981832}]},{"addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2777386525}]},{"addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2525346797}]},{"addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":1455515659}]}]} 2023-11-28T13:00:27.011 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T13:00:27.011 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T13:00:27.011 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:27.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.129+0000 7f0207fc8700 1 Processor -- start 2023-11-28T13:00:27.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.129+0000 7f0207fc8700 1 -- start start 2023-11-28T13:00:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0207fc8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200071040 0x7f0200071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0207fc8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f020006dee0 0x7f020006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0207fc8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0200071950 con 0x7f020006cdc0 2023-11-28T13:00:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0207fc8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0200161550 con 0x7f020006dee0 2023-11-28T13:00:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0207fc8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020015d060 con 0x7f0200071040 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0206565700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f020006cdc0 0x7f020006d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56584/0 (socket says 172.21.15.153:56584) 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0206565700 1 -- 172.21.15.153:0/1461138829 learned_addr learned my addr 172.21.15.153:0/1461138829 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0205563700 1 --2- 172.21.15.153:0/1461138829 >> v2:172.21.15.177:3300/0 conn(0x7f020006dee0 0x7f020006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0205d64700 1 --2- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200071040 0x7f0200071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0204d62700 1 -- 172.21.15.153:0/1461138829 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183922189 0 0) 0x7f0200071950 con 0x7f020006cdc0 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0204d62700 1 -- 172.21.15.153:0/1461138829 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec0035b0 con 0x7f020006cdc0 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0205563700 1 -- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200071040 msgr2=0x7f0200071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0205563700 1 --2- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200071040 0x7f0200071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0205563700 1 -- 172.21.15.153:0/1461138829 >> v1:172.21.15.153:6789/0 conn(0x7f020006cdc0 legacy=0x7f020006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0205563700 1 -- 172.21.15.153:0/1461138829 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020015d340 con 0x7f020006dee0 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.130+0000 7f0205d64700 1 --2- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200071040 0x7f0200071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0205563700 1 --2- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 0x7f020006e350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009840 tx=0x7f01f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2baa9426e959f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0204d62700 1 -- 172.21.15.153:0/1461138829 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0011830 con 0x7f020006dee0 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0204d62700 1 -- 172.21.15.153:0/1461138829 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f0011990 con 0x7f020006dee0 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0204d62700 1 -- 172.21.15.153:0/1461138829 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0011d20 con 0x7f020006dee0 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 -- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 msgr2=0x7f020006e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 0x7f020006e350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009840 tx=0x7f01f0009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 -- 172.21.15.153:0/1461138829 shutdown_connections 2023-11-28T13:00:27.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200071040 0x7f0200071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1461138829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 0x7f020006e350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 -- 172.21.15.153:0/1461138829 >> 172.21.15.153:0/1461138829 conn(0x7f02000d0000 msgr2=0x7f02000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 -- 172.21.15.153:0/1461138829 shutdown_connections 2023-11-28T13:00:27.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.131+0000 7f0207fc8700 1 -- 172.21.15.153:0/1461138829 wait complete. 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.132+0000 7f0207fc8700 1 Processor -- start 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.132+0000 7f0207fc8700 1 -- start start 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0207fc8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020006cdc0 0x7f02000b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0207fc8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 0x7f02000b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0207fc8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200071040 0x7f0200155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0207fc8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0200155ff0 con 0x7f0200071040 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0207fc8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c4b70 con 0x7f020006dee0 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0207fc8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c6b20 con 0x7f020006cdc0 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0206565700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200071040 0x7f0200155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0206565700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200071040 0x7f0200155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38426/0 (socket says 172.21.15.153:38426) 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.133+0000 7f0206565700 1 -- 172.21.15.153:0/1022219065 learned_addr learned my addr 172.21.15.153:0/1022219065 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0206565700 1 -- 172.21.15.153:0/1022219065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020006cdc0 msgr2=0x7f02000b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0206565700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020006cdc0 0x7f02000b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0206565700 1 -- 172.21.15.153:0/1022219065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 msgr2=0x7f02000b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0205563700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 0x7f02000b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0206565700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 0x7f02000b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0206565700 1 -- 172.21.15.153:0/1022219065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0094a0 con 0x7f0200071040 2023-11-28T13:00:27.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0206565700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200071040 0x7f0200155660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f01f800cad0 tx=0x7f01f800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f01f67fc700 1 -- 172.21.15.153:0/1022219065 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800aac0 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f01f67fc700 1 -- 172.21.15.153:0/1022219065 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f800ac20 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f00094a0 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f01f67fc700 1 -- 172.21.15.153:0/1022219065 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8012400 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.134+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020015d900 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.135+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020015d620 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.135+0000 7f01f67fc700 1 -- 172.21.15.153:0/1022219065 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f01f8019030 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.136+0000 7f01f67fc700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f01e40682f0 0x7f01e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.136+0000 7f01f67fc700 1 -- 172.21.15.153:0/1022219065 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f01f80858c0 con 0x7f0200071040 2023-11-28T13:00:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.136+0000 7f0205d64700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f01e40682f0 0x7f01e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.136+0000 7f0205d64700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f01e40682f0 0x7f01e406a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01fc006f10 tx=0x7f01fc006e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.140+0000 7f01f67fc700 1 -- 172.21.15.153:0/1022219065 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f01f8052120 con 0x7f0200071040 2023-11-28T13:00:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.322+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f020006e890 con 0x7f0200071040 2023-11-28T13:00:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.324+0000 7f01f67fc700 1 -- 172.21.15.153:0/1022219065 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f01f8055740 con 0x7f0200071040 2023-11-28T13:00:27.326 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:27.326 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:27.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.325+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f01e40682f0 msgr2=0x7f01e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f01e40682f0 0x7f01e406a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01fc006f10 tx=0x7f01fc006e60 comp rx=0 tx=0).stop 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200071040 msgr2=0x7f0200155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200071040 0x7f0200155660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f01f800cad0 tx=0x7f01f800ce90 comp rx=0 tx=0).stop 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 shutdown_connections 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f01e40682f0 0x7f01e406a7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0200071040 0x7f0200155660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020006cdc0 0x7f02000b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 --2- 172.21.15.153:0/1022219065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f020006dee0 0x7f02000b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 >> 172.21.15.153:0/1022219065 conn(0x7f02000d0000 msgr2=0x7f02000cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 shutdown_connections 2023-11-28T13:00:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.326+0000 7f0207fc8700 1 -- 172.21.15.153:0/1022219065 wait complete. 2023-11-28T13:00:27.340 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T13:00:27.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:27.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.458+0000 7f7eca6b7700 1 Processor -- start 2023-11-28T13:00:27.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.458+0000 7f7eca6b7700 1 -- start start 2023-11-28T13:00:27.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7eca6b7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ec4160740 0x7f7ec40cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7eca6b7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec406bbb0 con 0x7f7ec406b060 2023-11-28T13:00:27.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7eca6b7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec40cc030 con 0x7f7ec4160b10 2023-11-28T13:00:27.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7eca6b7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec40ca720 con 0x7f7ec4160740 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7ec3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7ec4160740 0x7f7ec40cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7ec8c54700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7ec406b060 0x7f7ec406b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56598/0 (socket says 172.21.15.153:56598) 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7ec8c54700 1 -- 172.21.15.153:0/145999704 learned_addr learned my addr 172.21.15.153:0/145999704 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7ec3fff700 1 -- 172.21.15.153:0/145999704 >> v1:172.21.15.177:6789/0 conn(0x7f7ec4160b10 legacy=0x7f7ec40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7ec3fff700 1 -- 172.21.15.153:0/145999704 >> v1:172.21.15.153:6789/0 conn(0x7f7ec406b060 legacy=0x7f7ec406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7ec3fff700 1 -- 172.21.15.153:0/145999704 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec40c9e10 con 0x7f7ec4160740 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.459+0000 7f7ec2ffd700 1 -- 172.21.15.153:0/145999704 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857438730 0 0) 0x7f7ec406bbb0 con 0x7f7ec406b060 2023-11-28T13:00:27.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7ec3fff700 1 --2- 172.21.15.153:0/145999704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec4160740 0x7f7ec40cbaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009840 tx=0x7f7eb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5009e795345f88c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7ec2ffd700 1 -- 172.21.15.153:0/145999704 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4004230 con 0x7f7ec4160740 2023-11-28T13:00:27.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7ec2ffd700 1 -- 172.21.15.153:0/145999704 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb4004390 con 0x7f7ec4160740 2023-11-28T13:00:27.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7ec2ffd700 1 -- 172.21.15.153:0/145999704 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40036a0 con 0x7f7ec4160740 2023-11-28T13:00:27.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7eca6b7700 1 -- 172.21.15.153:0/145999704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec4160740 msgr2=0x7f7ec40cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/145999704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec4160740 0x7f7ec40cbaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009840 tx=0x7f7eb4009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:27.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7eca6b7700 1 -- 172.21.15.153:0/145999704 shutdown_connections 2023-11-28T13:00:27.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/145999704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec4160740 0x7f7ec40cbaf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7eca6b7700 1 -- 172.21.15.153:0/145999704 >> 172.21.15.153:0/145999704 conn(0x7f7ec40b12b0 msgr2=0x7f7ec40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7eca6b7700 1 -- 172.21.15.153:0/145999704 shutdown_connections 2023-11-28T13:00:27.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.460+0000 7f7eca6b7700 1 -- 172.21.15.153:0/145999704 wait complete. 2023-11-28T13:00:27.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.462+0000 7f7eca6b7700 1 Processor -- start 2023-11-28T13:00:27.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.462+0000 7f7eca6b7700 1 -- start start 2023-11-28T13:00:27.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7eca6b7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec406b060 0x7f7ec4152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7eca6b7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4160740 0x7f7ec415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7eca6b7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec4160b10 0x7f7ec40c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7eca6b7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40c8560 con 0x7f7ec406b060 2023-11-28T13:00:27.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7eca6b7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec415d400 con 0x7f7ec4160740 2023-11-28T13:00:27.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7eca6b7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec40c4a20 con 0x7f7ec4160b10 2023-11-28T13:00:27.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec406b060 0x7f7ec4152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec406b060 0x7f7ec4152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38436/0 (socket says 172.21.15.153:38436) 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 -- 172.21.15.153:0/1343667989 learned_addr learned my addr 172.21.15.153:0/1343667989 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec37fe700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4160740 0x7f7ec415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec8c54700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec4160b10 0x7f7ec40c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 -- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec4160b10 msgr2=0x7f7ec40c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec4160b10 0x7f7ec40c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 -- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4160740 msgr2=0x7f7ec415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4160740 0x7f7ec415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec3fff700 1 -- 172.21.15.153:0/1343667989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb40094a0 con 0x7f7ec406b060 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.463+0000 7f7ec8c54700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec4160b10 0x7f7ec40c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.464+0000 7f7ec3fff700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec406b060 0x7f7ec4152240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40090b0 tx=0x7f7eb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.464+0000 7f7ec0ff9700 1 -- 172.21.15.153:0/1343667989 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4011630 con 0x7f7ec406b060 2023-11-28T13:00:27.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.464+0000 7f7ec0ff9700 1 -- 172.21.15.153:0/1343667989 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb401e070 con 0x7f7ec406b060 2023-11-28T13:00:27.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.464+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec40ca0d0 con 0x7f7ec406b060 2023-11-28T13:00:27.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.464+0000 7f7ec0ff9700 1 -- 172.21.15.153:0/1343667989 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4011990 con 0x7f7ec406b060 2023-11-28T13:00:27.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.464+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec406efc0 con 0x7f7ec406b060 2023-11-28T13:00:27.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.465+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec40ca3b0 con 0x7f7ec406b060 2023-11-28T13:00:27.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.465+0000 7f7ec0ff9700 1 -- 172.21.15.153:0/1343667989 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7eb4017020 con 0x7f7ec406b060 2023-11-28T13:00:27.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.465+0000 7f7ec0ff9700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea40682f0 0x7f7ea406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.465+0000 7f7ec0ff9700 1 -- 172.21.15.153:0/1343667989 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7eb4087000 con 0x7f7ec406b060 2023-11-28T13:00:27.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.466+0000 7f7ec37fe700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea40682f0 0x7f7ea406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.466+0000 7f7ec37fe700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea40682f0 0x7f7ea406a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ec404e880 tx=0x7f7eac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.470+0000 7f7ec0ff9700 1 -- 172.21.15.153:0/1343667989 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7eb4053860 con 0x7f7ec406b060 2023-11-28T13:00:27.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.654+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ec4066110 con 0x7f7ec406b060 2023-11-28T13:00:27.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.655+0000 7f7ec0ff9700 1 -- 172.21.15.153:0/1343667989 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f7eb4020090 con 0x7f7ec406b060 2023-11-28T13:00:27.657 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:27.657 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:27.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea40682f0 msgr2=0x7f7ea406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea40682f0 0x7f7ea406a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ec404e880 tx=0x7f7eac009040 comp rx=0 tx=0).stop 2023-11-28T13:00:27.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec406b060 msgr2=0x7f7ec4152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec406b060 0x7f7ec4152240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40090b0 tx=0x7f7eb4004750 comp rx=0 tx=0).stop 2023-11-28T13:00:27.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 shutdown_connections 2023-11-28T13:00:27.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea40682f0 0x7f7ea406a7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ec406b060 0x7f7ec4152240 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec4160b10 0x7f7ec40c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 --2- 172.21.15.153:0/1343667989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4160740 0x7f7ec415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 >> 172.21.15.153:0/1343667989 conn(0x7f7ec40b12b0 msgr2=0x7f7ec415bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 shutdown_connections 2023-11-28T13:00:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.657+0000 7f7eca6b7700 1 -- 172.21.15.153:0/1343667989 wait complete. 2023-11-28T13:00:27.672 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T13:00:27.672 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T13:00:27.672 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T13:00:27.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T13:00:27.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T13:00:27.673 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T13:00:27.674 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T13:00:27.674 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T13:00:27.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.874+0000 7f38af3e6700 1 Processor -- start 2023-11-28T13:00:27.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.874+0000 7f38af3e6700 1 -- start start 2023-11-28T13:00:27.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38af3e6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f38a8154e70 0x7f38a80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38af3e6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a80cf360 0x7f38a8071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38af3e6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a80719e0 con 0x7f38a8071580 2023-11-28T13:00:27.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38af3e6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a8155240 con 0x7f38a8154e70 2023-11-28T13:00:27.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38af3e6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a815b100 con 0x7f38a80cf360 2023-11-28T13:00:27.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ad182700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f38a8154e70 0x7f38a80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ad182700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f38a8154e70 0x7f38a80cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:47910/0 (socket says 172.21.15.153:47910) 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ad182700 1 -- 172.21.15.153:0/3688836442 learned_addr learned my addr 172.21.15.153:0/3688836442 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ac981700 1 --2- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a80cf360 0x7f38a8071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ad182700 1 -- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a80cf360 msgr2=0x7f38a8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ad182700 1 --2- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a80cf360 0x7f38a8071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ad182700 1 -- 172.21.15.153:0/3688836442 >> v1:172.21.15.153:6789/0 conn(0x7f38a8071580 legacy=0x7f38a815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.875+0000 7f38ad182700 1 -- 172.21.15.153:0/3688836442 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a815b3e0 con 0x7f38a8154e70 2023-11-28T13:00:27.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.876+0000 7f38ad182700 1 --2- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a8154e70 0x7f38a80cdbd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38980098f0 tx=0x7f3898009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5353da9bff4ea45 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.876+0000 7f389ffff700 1 -- 172.21.15.153:0/3688836442 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898004230 con 0x7f38a8154e70 2023-11-28T13:00:27.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.876+0000 7f389ffff700 1 -- 172.21.15.153:0/3688836442 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898004390 con 0x7f38a8154e70 2023-11-28T13:00:27.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.876+0000 7f389ffff700 1 -- 172.21.15.153:0/3688836442 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38980036a0 con 0x7f38a8154e70 2023-11-28T13:00:27.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.877+0000 7f38af3e6700 1 -- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a8154e70 msgr2=0x7f38a80cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.877+0000 7f38af3e6700 1 --2- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a8154e70 0x7f38a80cdbd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38980098f0 tx=0x7f3898009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:27.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.877+0000 7f38af3e6700 1 -- 172.21.15.153:0/3688836442 shutdown_connections 2023-11-28T13:00:27.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.877+0000 7f38af3e6700 1 --2- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a8154e70 0x7f38a80cdbd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.877+0000 7f38af3e6700 1 --2- 172.21.15.153:0/3688836442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a80cf360 0x7f38a8071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.877+0000 7f38af3e6700 1 -- 172.21.15.153:0/3688836442 >> 172.21.15.153:0/3688836442 conn(0x7f38a80d0000 msgr2=0x7f38a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.880+0000 7f38af3e6700 1 -- 172.21.15.153:0/3688836442 shutdown_connections 2023-11-28T13:00:27.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.880+0000 7f38af3e6700 1 -- 172.21.15.153:0/3688836442 wait complete. 2023-11-28T13:00:27.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.881+0000 7f38af3e6700 1 Processor -- start 2023-11-28T13:00:27.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.882+0000 7f38af3e6700 1 -- start start 2023-11-28T13:00:27.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38af3e6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a8071580 0x7f38a8150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38af3e6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a80cf360 0x7f38a80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38af3e6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a8154e70 0x7f38a80b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38af3e6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80b4f40 con 0x7f38a8154e70 2023-11-28T13:00:27.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38af3e6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a81f0a30 con 0x7f38a80cf360 2023-11-28T13:00:27.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38af3e6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80b46a0 con 0x7f38a8071580 2023-11-28T13:00:27.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38ad182700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a8071580 0x7f38a8150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38ad182700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a8071580 0x7f38a8150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:56060/0 (socket says 172.21.15.153:56060) 2023-11-28T13:00:27.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38ad182700 1 -- 172.21.15.153:0/461550368 learned_addr learned my addr 172.21.15.153:0/461550368 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38ac981700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a80cf360 0x7f38a80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.883+0000 7f38ad983700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a8154e70 0x7f38a80b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.884+0000 7f38ad983700 1 -- 172.21.15.153:0/461550368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a8071580 msgr2=0x7f38a8150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.884+0000 7f38ad983700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a8071580 0x7f38a8150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.884+0000 7f38ad983700 1 -- 172.21.15.153:0/461550368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a80cf360 msgr2=0x7f38a80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.884+0000 7f38ad983700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a80cf360 0x7f38a80b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.884+0000 7f38ad983700 1 -- 172.21.15.153:0/461550368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898009550 con 0x7f38a8154e70 2023-11-28T13:00:27.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.884+0000 7f38ad983700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a8154e70 0x7f38a80b4090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38a000c7b0 tx=0x7f38a000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.885+0000 7fb39aed5700 1 Processor -- start 2023-11-28T13:00:27.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.885+0000 7fb39aed5700 1 -- start start 2023-11-28T13:00:27.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.885+0000 7fb4d3ef1700 1 Processor -- start 2023-11-28T13:00:27.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb39aed5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb39415c9b0 0x7fb3940b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb39aed5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3940d1950 con 0x7fb394161340 2023-11-28T13:00:27.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb39aed5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3940d24f0 con 0x7fb3940cd7c0 2023-11-28T13:00:27.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb39aed5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3940b5800 con 0x7fb39415c9b0 2023-11-28T13:00:27.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb399472700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fb3940cd7c0 0x7fb3940b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33234/0 (socket says 172.21.15.153:33234) 2023-11-28T13:00:27.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb399472700 1 -- 172.21.15.153:0/3686859514 learned_addr learned my addr 172.21.15.153:0/3686859514 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:27.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0003e10 con 0x7f38a8154e70 2023-11-28T13:00:27.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a00050f0 con 0x7f38a8154e70 2023-11-28T13:00:27.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a00054a0 con 0x7f38a8154e70 2023-11-28T13:00:27.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898414617 0 0) 0x7fb3940d24f0 con 0x7fb3940cd7c0 2023-11-28T13:00:27.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3840035b0 con 0x7fb3940cd7c0 2023-11-28T13:00:27.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb398c71700 1 --2- 172.21.15.153:0/3686859514 >> v2:172.21.15.153:3300/0 conn(0x7fb39415c9b0 0x7fb3940b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096284086 0 0) 0x7fb3840035b0 con 0x7fb3940cd7c0 2023-11-28T13:00:27.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 >> v2:172.21.15.153:3300/0 conn(0x7fb39415c9b0 msgr2=0x7fb3940b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 --2- 172.21.15.153:0/3686859514 >> v2:172.21.15.153:3300/0 conn(0x7fb39415c9b0 0x7fb3940b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 >> v1:172.21.15.153:6789/0 conn(0x7fb394161340 legacy=0x7fb3940d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.885+0000 7fb4d3ef1700 1 -- start start 2023-11-28T13:00:27.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb4d3ef1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb4cc158710 0x7fb4cc159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb4d3ef1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc15c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb4d3ef1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc06e9e0 con 0x7fb4cc06e500 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb4d3ef1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc159cc0 con 0x7fb4cc158710 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.886+0000 7fb4d3ef1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15ccb0 con 0x7fb4cc15c2f0 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb4cc158710 0x7fb4cc159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d248e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb4cc158710 0x7fb4cc159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:47924/0 (socket says 172.21.15.153:47924) 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/3430288343 learned_addr learned my addr 172.21.15.153:0/3430288343 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 msgr2=0x7fb4cc15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 --2- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc15c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/3430288343 >> v1:172.21.15.153:6789/0 conn(0x7fb4cc06e500 legacy=0x7fb4cc0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/3430288343 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0ce090 con 0x7fb4cc158710 2023-11-28T13:00:27.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d248e700 1 --2- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc15c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:27.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb4d1c8d700 1 --2- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc158710 0x7fb4cc159780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098f0 tx=0x7fb4c8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a7150e4c1443a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3940b5520 con 0x7fb3940cd7c0 2023-11-28T13:00:27.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fb38c0019f0 con 0x7fb3940cd7c0 2023-11-28T13:00:27.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 >> v1:172.21.15.177:6789/0 conn(0x7fb3940cd7c0 legacy=0x7fb3940b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb384003a00 0x7fb384003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384004850 0x7fb384004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb384005200 0x7fb38400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940b5800 con 0x7fb384004850 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38400c110 con 0x7fb384005200 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.887+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38400c2d0 con 0x7fb384003a00 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb398c71700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb384003a00 0x7fb384003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb399472700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb384005200 0x7fb38400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb393fff700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384004850 0x7fb384004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb398c71700 1 -- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb384005200 msgr2=0x7fb38400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb398c71700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb384005200 0x7fb38400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb398c71700 1 -- 172.21.15.153:0/3686859514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384004850 msgr2=0x7fb384004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb398c71700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384004850 0x7fb384004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb398c71700 1 -- 172.21.15.153:0/3686859514 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3940b5520 con 0x7fb384003a00 2023-11-28T13:00:27.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb399472700 1 -- 172.21.15.153:0/3686859514 reap_dead start 2023-11-28T13:00:27.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb393fff700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384004850 0x7fb384004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:27.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb398c71700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb384003a00 0x7fb384003df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb388009b30 tx=0x7fb384005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a2c6a364b47019 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800e6c0 con 0x7fb384003a00 2023-11-28T13:00:27.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38800e820 con 0x7fb384003a00 2023-11-28T13:00:27.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.888+0000 7fb3937fe700 1 -- 172.21.15.153:0/3686859514 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800eb90 con 0x7fb384003a00 2023-11-28T13:00:27.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb4d0c8b700 1 -- 172.21.15.153:0/3430288343 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8011830 con 0x7fb4cc158710 2023-11-28T13:00:27.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb4d0c8b700 1 -- 172.21.15.153:0/3430288343 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c8011990 con 0x7fb4cc158710 2023-11-28T13:00:27.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb4d0c8b700 1 -- 172.21.15.153:0/3430288343 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8011d20 con 0x7fb4cc158710 2023-11-28T13:00:27.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb39aed5700 1 -- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb384003a00 msgr2=0x7fb384003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb39aed5700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb384003a00 0x7fb384003df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb388009b30 tx=0x7fb384005880 comp rx=0 tx=0).stop 2023-11-28T13:00:27.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb39aed5700 1 -- 172.21.15.153:0/3686859514 shutdown_connections 2023-11-28T13:00:27.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb39aed5700 1 --2- 172.21.15.153:0/3686859514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb384003a00 0x7fb384003df0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.889+0000 7fb39aed5700 1 -- 172.21.15.153:0/3686859514 >> 172.21.15.153:0/3686859514 conn(0x7fb394072600 msgr2=0x7fb3940cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.890+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc158710 msgr2=0x7fb4cc159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.890+0000 7fb4d3ef1700 1 --2- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc158710 0x7fb4cc159780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80098f0 tx=0x7fb4c8009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:27.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.890+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a815b9a0 con 0x7f38a8154e70 2023-11-28T13:00:27.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.891+0000 7fb39aed5700 1 -- 172.21.15.153:0/3686859514 shutdown_connections 2023-11-28T13:00:27.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.891+0000 7f2e9341c700 1 Processor -- start 2023-11-28T13:00:27.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.891+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a815b6c0 con 0x7f38a8154e70 2023-11-28T13:00:27.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.891+0000 7f4743cef700 1 Processor -- start 2023-11-28T13:00:27.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f38a80af520 con 0x7f38a8154e70 2023-11-28T13:00:27.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f4743cef700 1 -- start start 2023-11-28T13:00:27.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.891+0000 7f2e9341c700 1 -- start start 2023-11-28T13:00:27.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f2e9341c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2e8c0c2d40 0x7f2e8c161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f2e9341c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0c2780 0x7f2e8c161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f2e9341c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c0cbaf0 con 0x7f2e8c164d10 2023-11-28T13:00:27.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f2e9341c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c0ca6b0 con 0x7f2e8c0c2d40 2023-11-28T13:00:27.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f2e9341c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c165c00 con 0x7f2e8c0c2780 2023-11-28T13:00:27.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f4743cef700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f473c159490 0x7f473c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f4743cef700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f473c159860 0x7f473c0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f4743cef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f4743cef700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c0ccc00 con 0x7f473c159860 2023-11-28T13:00:27.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f4743cef700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c0ccec0 con 0x7f473c159490 2023-11-28T13:00:27.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f4743cef700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0cd1a0 con 0x7f473c0cdc80 2023-11-28T13:00:27.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f2e911b8700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f2e8c164d10 0x7f2e8c1650e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33252/0 (socket says 172.21.15.153:33252) 2023-11-28T13:00:27.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f2e911b8700 1 -- 172.21.15.153:0/911609029 learned_addr learned my addr 172.21.15.153:0/911609029 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:27.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/3430288343 shutdown_connections 2023-11-28T13:00:27.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7fb4d3ef1700 1 --2- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc15c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7fb4d3ef1700 1 --2- 172.21.15.153:0/3430288343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc158710 0x7fb4cc159780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/3430288343 >> 172.21.15.153:0/3430288343 conn(0x7fb4cc0b0c30 msgr2=0x7fb4cc0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f38a0019030 con 0x7f38a8154e70 2023-11-28T13:00:27.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f389dffb700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f388c0683c0 0x7f388c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38a0085120 con 0x7f38a8154e70 2023-11-28T13:00:27.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f389dffb700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7f388c070c70 0x7f388c073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 --> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f388c073730 con 0x7f388c070c70 2023-11-28T13:00:27.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a0087740 con 0x7f38a8154e70 2023-11-28T13:00:27.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f474228c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f474128a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f473c159860 0x7f473c0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f474128a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f473c159860 0x7f473c0b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38480/0 (socket says 172.21.15.153:38480) 2023-11-28T13:00:27.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f474128a700 1 -- 172.21.15.153:0/1435160809 learned_addr learned my addr 172.21.15.153:0/1435160809 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f474228c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:56092/0 (socket says 172.21.15.153:56092) 2023-11-28T13:00:27.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/3430288343 shutdown_connections 2023-11-28T13:00:27.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272142559 0 0) 0x7f2e8c0cbaf0 con 0x7f2e8c164d10 2023-11-28T13:00:27.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e780035b0 con 0x7f2e8c164d10 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f38ad182700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f388c0683c0 0x7f388c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f38ac981700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7f388c070c70 0x7f388c073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f474228c700 1 -- 172.21.15.153:0/1435160809 >> v2:172.21.15.177:3300/0 conn(0x7f473c159490 msgr2=0x7f473c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f474228c700 1 --2- 172.21.15.153:0/1435160809 >> v2:172.21.15.177:3300/0 conn(0x7f473c159490 0x7f473c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f474228c700 1 -- 172.21.15.153:0/1435160809 >> v2:172.21.15.153:3300/0 conn(0x7f473c159860 msgr2=0x7f473c0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f474228c700 1 --2- 172.21.15.153:0/1435160809 >> v2:172.21.15.153:3300/0 conn(0x7f473c159860 0x7f473c0b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f474228c700 1 -- 172.21.15.153:0/1435160809 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0cd480 con 0x7f473c0cdc80 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f474228c700 1 --2- 172.21.15.153:0/1435160809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c0c94c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f473c06d9b0 tx=0x7f473400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c09137d5fcb8cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e919b9700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0c2780 0x7f2e8c161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1329568390 0 0) 0x7f2e780035b0 con 0x7f2e8c164d10 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0c2780 msgr2=0x7f2e8c161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0c2780 0x7f2e8c161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 >> v2:172.21.15.177:3300/0 conn(0x7f2e8c0c2d40 msgr2=0x7f2e8c161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 --2- 172.21.15.153:0/911609029 >> v2:172.21.15.177:3300/0 conn(0x7f2e8c0c2d40 0x7f2e8c161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f7a84ae6700 1 Processor -- start 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0cbd20 con 0x7f2e8c164d10 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f2e7c001670 con 0x7f2e8c164d10 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 >> v1:172.21.15.177:6789/0 conn(0x7f2e8c164d10 legacy=0x7f2e8c1650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e78003a00 0x7f2e78003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e78004850 0x7f2e78004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78005200 0x7f2e7800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0ca6b0 con 0x7f2e78003a00 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c165c00 con 0x7f2e78005200 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7800c080 con 0x7f2e78004850 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f2e911b8700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e78003a00 0x7f2e78003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f38ad182700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f388c0683c0 0x7f388c06a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3898009c40 tx=0x7f3898008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7fb39aed5700 1 -- 172.21.15.153:0/3686859514 wait complete. 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7fb39aed5700 1 Processor -- start 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f38ac981700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7f388c070c70 0x7f388c073080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== osd.0 v2:172.21.15.153:6800/103561 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f388c073730 con 0x7f388c070c70 2023-11-28T13:00:27.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7faae7319700 1 Processor -- start 2023-11-28T13:00:27.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7faae7319700 1 -- start start 2023-11-28T13:00:27.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/3430288343 wait complete. 2023-11-28T13:00:27.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7fb4d3ef1700 1 Processor -- start 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb39aed5700 1 -- start start 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e919b9700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78005200 0x7f2e7800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e911b8700 1 -- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e78004850 msgr2=0x7f2e78004cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e911b8700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e78004850 0x7f2e78004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e911b8700 1 -- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78005200 msgr2=0x7f2e7800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e911b8700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78005200 0x7f2e7800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e911b8700 1 -- 172.21.15.153:0/911609029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e780035b0 con 0x7f2e78003a00 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e911b8700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e78003a00 0x7f2e78003e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c240 tx=0x7f2e78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afabcd7bb1d25995 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00cd80 con 0x7f2e78003a00 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c005960 con 0x7f2e78003a00 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f2e83fff700 1 -- 172.21.15.153:0/911609029 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c005d10 con 0x7f2e78003a00 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f7a84ae6700 1 -- start start 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f7a84ae6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2780 0x7f7a80161d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f7a84ae6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a800cbaf0 con 0x7f7a800c2d40 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f7a84ae6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a800ca6b0 con 0x7f7a80164d10 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f7a84ae6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80165c00 con 0x7f7a800c2780 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7ed9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2780 0x7f7a80161d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7ed9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2780 0x7f7a80161d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:56110/0 (socket says 172.21.15.153:56110) 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7659c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f7a80164d10 0x7f7a801650e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33280/0 (socket says 172.21.15.153:33280) 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7ed9d700 1 -- 172.21.15.153:0/3204847415 learned_addr learned my addr 172.21.15.153:0/3204847415 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7ed9d700 1 -- 172.21.15.153:0/3204847415 >> v1:172.21.15.177:6789/0 conn(0x7f7a80164d10 legacy=0x7f7a801650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7dd9b700 1 -- 172.21.15.153:0/3204847415 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922750926 0 0) 0x7f7a800ca6b0 con 0x7f7a80164d10 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7ed9d700 1 -- 172.21.15.153:0/3204847415 >> v1:172.21.15.153:6789/0 conn(0x7f7a800c2d40 legacy=0x7f7a80161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7ed9d700 1 -- 172.21.15.153:0/3204847415 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800cbd20 con 0x7f7a800c2780 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7faae7319700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faae006c990 0x7faae006ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7faae7319700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae006ae60 con 0x7faae00ca240 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7faae7319700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae006a270 con 0x7faae006e9f0 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7faae7319700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae006d350 con 0x7faae006c990 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7faae50b5700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7faae006e9f0 0x7faae006b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33282/0 (socket says 172.21.15.153:33282) 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7faae50b5700 1 -- 172.21.15.153:0/3321439848 learned_addr learned my addr 172.21.15.153:0/3321439848 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87714135 0 0) 0x7faae006a270 con 0x7faae006e9f0 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faacc0034d0 con 0x7faae006e9f0 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faae48b4700 1 --2- 172.21.15.153:0/3321439848 >> v2:172.21.15.153:3300/0 conn(0x7faae006c990 0x7faae006ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814331162 0 0) 0x7faae006ae60 con 0x7faae00ca240 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faae006a270 con 0x7faae00ca240 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3207509103 0 0) 0x7faacc0034d0 con 0x7faae006e9f0 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 >> v2:172.21.15.153:3300/0 conn(0x7faae006c990 msgr2=0x7faae006ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 --2- 172.21.15.153:0/3321439848 >> v2:172.21.15.153:3300/0 conn(0x7faae006c990 0x7faae006ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 >> v1:172.21.15.153:6789/0 conn(0x7faae00ca240 legacy=0x7faae00ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae006a530 con 0x7faae006e9f0 2023-11-28T13:00:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7faad0001670 con 0x7faae006e9f0 2023-11-28T13:00:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 >> v1:172.21.15.177:6789/0 conn(0x7faae006e9f0 legacy=0x7faae006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faacc003cd0 0x7faacc0088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faacc0041f0 0x7faacc00aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc00b5d0 0x7faacc00da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae006d350 con 0x7faacc00b5d0 2023-11-28T13:00:27.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc00e360 con 0x7faacc0041f0 2023-11-28T13:00:27.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faacc00e520 con 0x7faacc003cd0 2023-11-28T13:00:27.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7fb4d3ef1700 1 -- start start 2023-11-28T13:00:27.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d3ef1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc06e500 0x7fb4cc15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d3ef1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4cc158710 0x7fb4cc0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d3ef1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc165e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d3ef1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc14f850 con 0x7fb4cc158710 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d3ef1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0b07a0 con 0x7fb4cc06e500 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d3ef1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15b210 con 0x7fb4cc15c2f0 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d1c8d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc06e500 0x7fb4cc15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7fb4d1c8d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc06e500 0x7fb4cc15ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:47976/0 (socket says 172.21.15.153:47976) 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/1080188071 learned_addr learned my addr 172.21.15.153:0/1080188071 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d248e700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d148c700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4cc158710 0x7fb4cc0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/1080188071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 msgr2=0x7fb4cc165e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d1c8d700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc15c2f0 0x7fb4cc165e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4cc158710 msgr2=0x7fb4cc0b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d1c8d700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4cc158710 0x7fb4cc0b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7f2e9341c700 1 -- 172.21.15.153:0/911609029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e78003a00 msgr2=0x7f2e78003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7f2e9341c700 1 --2- 172.21.15.153:0/911609029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e78003a00 0x7f2e78003e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c240 tx=0x7f2e78005880 comp rx=0 tx=0).stop 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7f2e919b9700 1 -- 172.21.15.153:0/911609029 reap_dead start 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7ed9d700 1 --2- 172.21.15.153:0/3204847415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2780 0x7f7a80161d30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e80 tx=0x7f7a7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3cd58806da7ca35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7dd9b700 1 -- 172.21.15.153:0/3204847415 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800e070 con 0x7f7a800c2780 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7dd9b700 1 -- 172.21.15.153:0/3204847415 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78003a20 con 0x7f7a800c2780 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f7a7dd9b700 1 -- 172.21.15.153:0/3204847415 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78003dd0 con 0x7f7a800c2780 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faae48b4700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faacc0041f0 0x7faacc00aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7faae50b5700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faacc003cd0 0x7faacc0088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb4d1c8d700 1 -- 172.21.15.153:0/1080188071 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4009a70 con 0x7fb4cc06e500 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb39aed5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384003a00 0x7fb3940c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb39aed5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb39406fb70 0x7fb39406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb39aed5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3940b62b0 0x7fb3940b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb39aed5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb394070520 con 0x7fb384003a00 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb39aed5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940b6fe0 con 0x7fb39406fb70 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb39aed5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940b72c0 con 0x7fb3940b62b0 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb398c71700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384003a00 0x7fb3940c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.897+0000 7fb398c71700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384003a00 0x7fb3940c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38522/0 (socket says 172.21.15.153:38522) 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb398c71700 1 -- 172.21.15.153:0/1352300065 learned_addr learned my addr 172.21.15.153:0/1352300065 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f2e9341c700 1 -- 172.21.15.153:0/911609029 shutdown_connections 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f2e9341c700 1 -- 172.21.15.153:0/911609029 >> 172.21.15.153:0/911609029 conn(0x7f2e8c0b12b0 msgr2=0x7f2e8c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f151a48d700 1 Processor -- start 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.892+0000 7f151a48d700 1 -- start start 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.893+0000 7f151a48d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15140ce480 0x7f15140c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f151a48d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1514161220 0x7f151406b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f151a48d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1514162210 con 0x7f1514164e60 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f151a48d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15141649d0 con 0x7f1514161220 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f151a48d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151406c020 con 0x7f15140ce480 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f15137fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1514161220 0x7f151406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f15137fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1514161220 0x7f151406b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:47954/0 (socket says 172.21.15.153:47954) 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f15137fe700 1 -- 172.21.15.153:0/826907137 learned_addr learned my addr 172.21.15.153:0/826907137 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f15137fe700 1 -- 172.21.15.153:0/826907137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15140ce480 msgr2=0x7f15140c1fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f15137fe700 1 --2- 172.21.15.153:0/826907137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15140ce480 0x7f15140c1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f15137fe700 1 -- 172.21.15.153:0/826907137 >> v1:172.21.15.153:6789/0 conn(0x7f1514164e60 legacy=0x7f15140b0670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.894+0000 7f15137fe700 1 -- 172.21.15.153:0/826907137 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151406bd60 con 0x7f1514161220 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f15137fe700 1 --2- 172.21.15.153:0/826907137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514161220 0x7f151406b820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f15080098f0 tx=0x7f1508009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b5f1d6cf59f47a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f7a84ae6700 1 -- 172.21.15.153:0/3204847415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2780 msgr2=0x7f7a80161d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f7a84ae6700 1 --2- 172.21.15.153:0/3204847415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2780 0x7f7a80161d30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e80 tx=0x7f7a7800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f1512ffd700 1 -- 172.21.15.153:0/826907137 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508004230 con 0x7f1514161220 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f1512ffd700 1 -- 172.21.15.153:0/826907137 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1508004390 con 0x7f1514161220 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.895+0000 7f1512ffd700 1 -- 172.21.15.153:0/826907137 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15080036a0 con 0x7f1514161220 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f151a48d700 1 -- 172.21.15.153:0/826907137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514161220 msgr2=0x7f151406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.896+0000 7f151a48d700 1 --2- 172.21.15.153:0/826907137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514161220 0x7f151406b820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f15080098f0 tx=0x7f1508009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb4d1c8d700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc06e500 0x7fb4cc15ac30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009c40 tx=0x7fb4c8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb399472700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3940b62b0 0x7fb3940b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb393fff700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb39406fb70 0x7fb39406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f4740a89700 1 -- 172.21.15.153:0/1435160809 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400e070 con 0x7f473c0cdc80 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f4740a89700 1 -- 172.21.15.153:0/1435160809 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473400bbd0 con 0x7f473c0cdc80 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae58b6700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc00b5d0 0x7faacc00da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae50b5700 1 -- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faacc0041f0 msgr2=0x7faacc00aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae50b5700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faacc0041f0 0x7faacc00aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae50b5700 1 -- 172.21.15.153:0/3321439848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc00b5d0 msgr2=0x7faacc00da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae50b5700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc00b5d0 0x7faacc00da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae58b6700 1 -- 172.21.15.153:0/3321439848 reap_dead start 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae50b5700 1 -- 172.21.15.153:0/3321439848 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae006a530 con 0x7faacc003cd0 2023-11-28T13:00:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7faae50b5700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faacc003cd0 0x7faacc0088e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faad000c2f0 tx=0x7faacc003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=595d05ebeeecdc7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8004450 con 0x7fb4cc06e500 2023-11-28T13:00:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c80045b0 con 0x7fb4cc06e500 2023-11-28T13:00:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80038e0 con 0x7fb4cc06e500 2023-11-28T13:00:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c8009550 con 0x7fb4cc06e500 2023-11-28T13:00:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc0ce650 con 0x7fb4cc06e500 2023-11-28T13:00:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.899+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4cc0ce370 con 0x7fb4cc06e500 2023-11-28T13:00:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb398c71700 1 -- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3940b62b0 msgr2=0x7fb3940b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb398c71700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3940b62b0 0x7fb3940b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb398c71700 1 -- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb39406fb70 msgr2=0x7fb39406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb398c71700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb39406fb70 0x7fb39406ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb398c71700 1 -- 172.21.15.153:0/1352300065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3940d15e0 con 0x7fb384003a00 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7fb398c71700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384003a00 0x7fb3940c9910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c460 tx=0x7fb3880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f151a48d700 1 -- 172.21.15.153:0/826907137 shutdown_connections 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f151a48d700 1 --2- 172.21.15.153:0/826907137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15140ce480 0x7f15140c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f151a48d700 1 --2- 172.21.15.153:0/826907137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514161220 0x7f151406b820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f151a48d700 1 -- 172.21.15.153:0/826907137 >> 172.21.15.153:0/826907137 conn(0x7f15140cb670 msgr2=0x7f15140cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f151a48d700 1 -- 172.21.15.153:0/826907137 shutdown_connections 2023-11-28T13:00:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f151a48d700 1 -- 172.21.15.153:0/826907137 wait complete. 2023-11-28T13:00:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f7a84ae6700 1 -- 172.21.15.153:0/3204847415 shutdown_connections 2023-11-28T13:00:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f7a84ae6700 1 --2- 172.21.15.153:0/3204847415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2780 0x7f7a80161d30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f7a84ae6700 1 -- 172.21.15.153:0/3204847415 >> 172.21.15.153:0/3204847415 conn(0x7f7a800b12b0 msgr2=0x7f7a800b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f7a84ae6700 1 -- 172.21.15.153:0/3204847415 shutdown_connections 2023-11-28T13:00:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.898+0000 7f7a84ae6700 1 -- 172.21.15.153:0/3204847415 wait complete. 2023-11-28T13:00:27.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.899+0000 7f4740a89700 1 -- 172.21.15.153:0/1435160809 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734012430 con 0x7f473c0cdc80 2023-11-28T13:00:27.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.899+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38800e6c0 con 0x7fb384003a00 2023-11-28T13:00:27.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.899+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388003ad0 con 0x7fb384003a00 2023-11-28T13:00:27.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.899+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388003e80 con 0x7fb384003a00 2023-11-28T13:00:27.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.899+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3940d1300 con 0x7fb384003a00 2023-11-28T13:00:27.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.899+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39415cd80 con 0x7fb384003a00 2023-11-28T13:00:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb374000fc0 con 0x7fb384003a00 2023-11-28T13:00:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f151a48d700 1 Processor -- start 2023-11-28T13:00:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f4743cef700 1 -- 172.21.15.153:0/1435160809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 msgr2=0x7f473c0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f4743cef700 1 --2- 172.21.15.153:0/1435160809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c0c94c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f473c06d9b0 tx=0x7f473400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f4743cef700 1 -- 172.21.15.153:0/1435160809 shutdown_connections 2023-11-28T13:00:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f4743cef700 1 --2- 172.21.15.153:0/1435160809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c0c94c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f4743cef700 1 --2- 172.21.15.153:0/1435160809 >> v2:172.21.15.177:3300/0 conn(0x7f473c159490 0x7f473c0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f4743cef700 1 --2- 172.21.15.153:0/1435160809 >> v2:172.21.15.153:3300/0 conn(0x7f473c159860 0x7f473c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f4743cef700 1 -- 172.21.15.153:0/1435160809 >> 172.21.15.153:0/1435160809 conn(0x7f473c0cb670 msgr2=0x7f473c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f7a84ae6700 1 Processor -- start 2023-11-28T13:00:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f7a84ae6700 1 -- start start 2023-11-28T13:00:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f7a84ae6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a800c2780 0x7f7a800ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb4c8035070 con 0x7fb4cc06e500 2023-11-28T13:00:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7fb4c27fc700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4b8068450 0x7fb4b806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4c8087a20 con 0x7fb4cc06e500 2023-11-28T13:00:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7fb4c27fc700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7fb4b8070c00 0x7fb4b8073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 --> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4b80736c0 con 0x7fb4b8070c00 2023-11-28T13:00:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4c8087e80 con 0x7fb4cc06e500 2023-11-28T13:00:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f7a84ae6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2d40 0x7f7a80155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f7a84ae6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a80164d10 0x7f7a800c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f7a84ae6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80156300 con 0x7f7a800c2780 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f7a84ae6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800afc30 con 0x7f7a80164d10 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f7a84ae6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800aff10 con 0x7f7a800c2d40 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7fb4d148c700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4b8068450 0x7fb4b806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad000cdb0 con 0x7faacc003cd0 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad0005960 con 0x7faacc003cd0 2023-11-28T13:00:27.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f4743cef700 1 -- 172.21.15.153:0/1435160809 shutdown_connections 2023-11-28T13:00:27.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7faad7fff700 1 -- 172.21.15.153:0/3321439848 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0005d10 con 0x7faacc003cd0 2023-11-28T13:00:27.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7faae7319700 1 -- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faacc003cd0 msgr2=0x7faacc0088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7faae7319700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faacc003cd0 0x7faacc0088e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faad000c2f0 tx=0x7faacc003780 comp rx=0 tx=0).stop 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7fb4d248e700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7fb4b8070c00 0x7fb4b8073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2d40 0x7f7a80155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2d40 0x7f7a80155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:56148/0 (socket says 172.21.15.153:56148) 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 -- 172.21.15.153:0/2760237664 learned_addr learned my addr 172.21.15.153:0/2760237664 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.900+0000 7f151a48d700 1 -- start start 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f151a48d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15140ce480 0x7f15140b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f151a48d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1514161220 0x7f151415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f151a48d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514164e60 0x7f151415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f151a48d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151415b160 con 0x7f15140ce480 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f151a48d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151415e3a0 con 0x7f1514164e60 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.901+0000 7f151a48d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15141541c0 con 0x7f1514161220 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514164e60 0x7f151415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514164e60 0x7f151415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:47996/0 (socket says 172.21.15.153:47996) 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 -- 172.21.15.153:0/4032565373 learned_addr learned my addr 172.21.15.153:0/4032565373 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f15137fe700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1514161220 0x7f151415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7fb4d248e700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7fb4b8070c00 0x7fb4b8073010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 -- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a80164d10 msgr2=0x7f7a800c7180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a80164d10 0x7f7a800c7180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 -- 172.21.15.153:0/2760237664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a800c2780 msgr2=0x7f7a800ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a800c2780 0x7f7a800ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f7a7e59c700 1 -- 172.21.15.153:0/2760237664 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78009b30 con 0x7f7a800c2d40 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb388025070 con 0x7fb384003a00 2023-11-28T13:00:27.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 -- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1514161220 msgr2=0x7f151415ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1514161220 0x7f151415ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 -- 172.21.15.153:0/4032565373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15140ce480 msgr2=0x7f15140b4560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:00:27.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15140ce480 0x7f15140b4560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 -- 172.21.15.153:0/4032565373 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508009550 con 0x7f1514164e60 2023-11-28T13:00:27.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f1518a2a700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514164e60 0x7f151415de60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f150c00b970 tx=0x7f150c00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb4d148c700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4b8068450 0x7fb4b806a900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc14ae00 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7f7a7e59c700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2d40 0x7f7a80155dc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c4a0 tx=0x7f7a6800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7faae7319700 1 -- 172.21.15.153:0/3321439848 shutdown_connections 2023-11-28T13:00:27.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7faae7319700 1 --2- 172.21.15.153:0/3321439848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faacc003cd0 0x7faacc0088e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7faae7319700 1 -- 172.21.15.153:0/3321439848 >> 172.21.15.153:0/3321439848 conn(0x7faae00b12b0 msgr2=0x7faae00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7faae7319700 1 -- 172.21.15.153:0/3321439848 shutdown_connections 2023-11-28T13:00:27.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7f1510ff9700 1 -- 172.21.15.153:0/4032565373 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00c930 con 0x7f1514164e60 2023-11-28T13:00:27.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7f1510ff9700 1 -- 172.21.15.153:0/4032565373 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c00cab0 con 0x7f1514164e60 2023-11-28T13:00:27.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7f1510ff9700 1 -- 172.21.15.153:0/4032565373 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00cea0 con 0x7f1514164e60 2023-11-28T13:00:27.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1514161c50 con 0x7f1514164e60 2023-11-28T13:00:27.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1514161970 con 0x7f1514164e60 2023-11-28T13:00:27.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.902+0000 7f4743cef700 1 -- 172.21.15.153:0/1435160809 wait complete. 2023-11-28T13:00:27.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7faae7319700 1 -- 172.21.15.153:0/3321439848 wait complete. 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7faae7319700 1 Processor -- start 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7faae7319700 1 -- start start 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae7319700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc003cd0 0x7faae0159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae7319700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae01595d0 0x7faae0156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae7319700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae01e20b0 0x7faae01e2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae7319700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae01e2a90 con 0x7faacc003cd0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae7319700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae0159a40 con 0x7faae01595d0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae7319700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae01faf50 con 0x7faae01e20b0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae50b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc003cd0 0x7faae0159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae01595d0 0x7faae0156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae01595d0 0x7faae0156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:48016/0 (socket says 172.21.15.153:48016) 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae50b5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc003cd0 0x7faae0159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38554/0 (socket says 172.21.15.153:38554) 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 -- 172.21.15.153:0/3116757994 learned_addr learned my addr 172.21.15.153:0/3116757994 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 -- 172.21.15.153:0/3116757994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae01e20b0 msgr2=0x7faae01e2550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae01e20b0 0x7faae01e2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 -- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc003cd0 msgr2=0x7faae0159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faacc003cd0 0x7faae0159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7faae48b4700 1 -- 172.21.15.153:0/3116757994 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae006aaf0 con 0x7faae01595d0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7faae48b4700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae01595d0 0x7faae0156e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faad8004780 tx=0x7faad8003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8007b00 con 0x7faae01595d0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8007c60 con 0x7faae01595d0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae006a810 con 0x7faae01595d0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8015420 con 0x7faae01595d0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae006edc0 con 0x7faae01595d0 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== osd.3 v2:172.21.15.153:6822/103565 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb4b80736c0 con 0x7fb4b8070c00 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb3917fa700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb378068370 0x7fb37806a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb388087590 con 0x7fb384003a00 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb3917fa700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fb378070c20 0x7fb378073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 --> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3780736e0 con 0x7fb378070c20 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb388087a10 con 0x7fb384003a00 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb393fff700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb378068370 0x7fb37806a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7fb399472700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fb378070c20 0x7fb378073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7fb393fff700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb378068370 0x7fb37806a820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb39406f580 tx=0x7fb380009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7fb399472700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fb378070c20 0x7fb378073030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== osd.4 v2:172.21.15.177:6809/103772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb3780736e0 con 0x7fb378070c20 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.903+0000 7f2e9341c700 1 -- 172.21.15.153:0/911609029 shutdown_connections 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f2e9341c700 1 -- 172.21.15.153:0/911609029 wait complete. 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f1510ff9700 1 -- 172.21.15.153:0/4032565373 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f150c01b030 con 0x7f1514164e60 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f1510ff9700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f14f8068390 0x7f14f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f1510ff9700 1 -- 172.21.15.153:0/4032565373 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f150c0857e0 con 0x7f1514164e60 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7f1513fff700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f14f8068390 0x7f14f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7f1513fff700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f14f8068390 0x7f14f806a840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1504009670 tx=0x7f1504006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7f151a48d700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7f151415f950 0x7f1514150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 --> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15140745c0 con 0x7f151415f950 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7f15137fe700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7f151415f950 0x7f1514150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.907+0000 7f15137fe700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7f151415f950 0x7f1514150350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680054e0 con 0x7f7a800c2d40 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68003680 con 0x7f7a800c2d40 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68003a30 con 0x7f7a800c2d40 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a800c9eb0 con 0x7f7a800c2d40 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a800cc000 con 0x7f7a800c2d40 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.906+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a800ca190 con 0x7f7a800c2d40 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.908+0000 7f1510ff9700 1 -- 172.21.15.153:0/4032565373 <== osd.6 v2:172.21.15.177:6824/103787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f15140745c0 con 0x7f151415f950 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.908+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faae00b38b0 con 0x7faae01595d0 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.908+0000 7f2e9341c700 1 Processor -- start 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a68019030 con 0x7f7a800c2d40 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f7a777fe700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a60068340 0x7f7a6006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a68085750 con 0x7f7a800c2d40 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f7a7659c700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a60068340 0x7f7a6006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f7a777fe700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f7a60070bf0 0x7f7a60073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 --> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a600736b0 con 0x7f7a60070bf0 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a68085bd0 con 0x7f7a800c2d40 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f7a7ed9d700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f7a60070bf0 0x7f7a60073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f7a7ed9d700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f7a60070bf0 0x7f7a60073000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f7a7659c700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a60068340 0x7f7a6006a7f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a70000c00 tx=0x7f7a70007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f2e9341c700 1 -- start start 2023-11-28T13:00:27.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e9341c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78003a00 0x7f2e8c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e9341c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0b3570 0x7f2e8c1ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e9341c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c15e850 0x7f2e8c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e9341c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c15d170 con 0x7f2e8c15e850 2023-11-28T13:00:27.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e9341c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c15d3c0 con 0x7f2e78003a00 2023-11-28T13:00:27.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e9341c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c15d6a0 con 0x7f2e8c0b3570 2023-11-28T13:00:27.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e911b8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78003a00 0x7f2e8c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e909b7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0b3570 0x7f2e8c1ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e911b8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78003a00 0x7f2e8c15e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:48030/0 (socket says 172.21.15.153:48030) 2023-11-28T13:00:27.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e909b7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0b3570 0x7f2e8c1ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:56176/0 (socket says 172.21.15.153:56176) 2023-11-28T13:00:27.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f2e911b8700 1 -- 172.21.15.153:0/1111093366 learned_addr learned my addr 172.21.15.153:0/1111093366 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f2e909b7700 1 -- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78003a00 msgr2=0x7f2e8c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f2e909b7700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e78003a00 0x7f2e8c15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f2e909b7700 1 -- 172.21.15.153:0/1111093366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c15e850 msgr2=0x7f2e8c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f2e909b7700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2e8c15e850 0x7f2e8c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f2e909b7700 1 -- 172.21.15.153:0/1111093366 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0c9eb0 con 0x7f2e8c0b3570 2023-11-28T13:00:27.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== osd.7 v2:172.21.15.177:6808/103771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7a600736b0 con 0x7f7a60070bf0 2023-11-28T13:00:27.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f2e909b7700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0b3570 0x7f2e8c1ff410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800b450 tx=0x7f2e8800b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88011830 con 0x7f2e8c0b3570 2023-11-28T13:00:27.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.912+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88011990 con 0x7f2e8c0b3570 2023-11-28T13:00:27.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.912+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c0cc000 con 0x7f2e8c0b3570 2023-11-28T13:00:27.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.912+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c0ca190 con 0x7f2e8c0b3570 2023-11-28T13:00:27.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.912+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88011d40 con 0x7f2e8c0b3570 2023-11-28T13:00:27.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e6c000fc0 con 0x7f2e8c0b3570 2023-11-28T13:00:27.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faad8015580 con 0x7faae01595d0 2023-11-28T13:00:27.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7faad5ffb700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faac4068320 0x7faac406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7f4743cef700 1 Processor -- start 2023-11-28T13:00:27.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.904+0000 7f4743cef700 1 -- start start 2023-11-28T13:00:27.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f4743cef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f4743cef700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f473c159490 0x7f473c0c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f4743cef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f473c159c40 0x7f473c14da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f4743cef700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0c29d0 con 0x7f473c159490 2023-11-28T13:00:27.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f4743cef700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1566d0 con 0x7f473c159c40 2023-11-28T13:00:27.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.905+0000 7f4743cef700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c15a5d0 con 0x7f473c0cdc80 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f473c159c40 0x7f473c14da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f473c159c40 0x7f473c14da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:48012/0 (socket says 172.21.15.153:48012) 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 -- 172.21.15.153:0/3571824424 learned_addr learned my addr 172.21.15.153:0/3571824424 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f4741a8b700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 -- 172.21.15.153:0/3571824424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 msgr2=0x7f473c156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f473c0cdc80 0x7f473c156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 -- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f473c159490 msgr2=0x7f473c0c1d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:00:27.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f473c159490 0x7f473c0c1d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 -- 172.21.15.153:0/3571824424 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734009a70 con 0x7f473c159c40 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.909+0000 7f474228c700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f473c159c40 0x7f473c14da00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4734014040 tx=0x7f4734004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.910+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340048b0 con 0x7f473c159c40 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4734020070 con 0x7f473c159c40 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340276c0 con 0x7f473c159c40 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c0c9a00 con 0x7f473c159c40 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.911+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c0cd760 con 0x7f473c159c40 2023-11-28T13:00:27.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.912+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f473c0c9b90 con 0x7f473c159c40 2023-11-28T13:00:27.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.914+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2e88021070 con 0x7f2e8c0b3570 2023-11-28T13:00:27.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faad8086580 con 0x7faae01595d0 2023-11-28T13:00:27.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7faad5ffb700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7faac4070bd0 0x7faac4072fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 --> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faac4073690 con 0x7faac4070bd0 2023-11-28T13:00:27.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad8088ba0 con 0x7faae01595d0 2023-11-28T13:00:27.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.913+0000 7faae50b5700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faac4068320 0x7faac406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.914+0000 7faae50b5700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faac4068320 0x7faac406a7d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faad0000c00 tx=0x7faad0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.915+0000 7f2e81ffb700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2e700682f0 0x7f2e7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.915+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e880859e0 con 0x7f2e8c0b3570 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.915+0000 7f2e911b8700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2e700682f0 0x7f2e7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.915+0000 7f2e81ffb700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f2e70070b20 0x7f2e70072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.915+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 --> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e700735e0 con 0x7f2e70070b20 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.915+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e8808a2a0 con 0x7f2e8c0b3570 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.915+0000 7f2e919b9700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f2e70070b20 0x7f2e70072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.916+0000 7f2e911b8700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2e700682f0 0x7f2e7006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c830 tx=0x7f2e7c00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.916+0000 7f2e919b9700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f2e70070b20 0x7f2e70072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.917+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== osd.5 v2:172.21.15.177:6800/103770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2e700735e0 con 0x7f2e70070b20 2023-11-28T13:00:27.971 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.921+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 --> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f38a81f0b70 con 0x7f388c070c70 2023-11-28T13:00:27.971 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.922+0000 7f389dffb700 1 -- 172.21.15.153:0/461550368 <== osd.0 v2:172.21.15.153:6800/103561 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f38a81f0b70 con 0x7f388c070c70 2023-11-28T13:00:27.972 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.921+0000 7faae58b6700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7faac4070bd0 0x7faac4072fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.972 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.922+0000 7faae58b6700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7faac4070bd0 0x7faac4072fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.972 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.923+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4734027820 con 0x7f473c159c40 2023-11-28T13:00:27.972 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.923+0000 7f47327fc700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47200683c0 0x7f472006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.972 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.923+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== osd.2 v2:172.21.15.153:6808/103562 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faac4073690 con 0x7faac4070bd0 2023-11-28T13:00:27.973 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.923+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4734022070 con 0x7f473c159c40 2023-11-28T13:00:27.973 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.923+0000 7f47327fc700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f4720070c70 0x7f4720073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.973 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.923+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 --> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4720073730 con 0x7f4720070c70 2023-11-28T13:00:27.973 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.923+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4734088600 con 0x7f473c159c40 2023-11-28T13:00:27.973 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.924+0000 7f4741a8b700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47200683c0 0x7f472006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.974 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.924+0000 7f4741a8b700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47200683c0 0x7f472006a870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4738009a10 tx=0x7f4738009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.974 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.924+0000 7f474128a700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f4720070c70 0x7f4720073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.974 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.924+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7f388c070c70 msgr2=0x7f388c073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.974 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.924+0000 7f38af3e6700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7f388c070c70 0x7f388c073080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.974 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f474128a700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f4720070c70 0x7f4720073080 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f388c0683c0 msgr2=0x7f388c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f38af3e6700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f388c0683c0 0x7f388c06a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3898009c40 tx=0x7f3898008030 comp rx=0 tx=0).stop 2023-11-28T13:00:27.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a8154e70 msgr2=0x7f38a80b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f38af3e6700 1 --2- 172.21.15.153:0/461550368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38a8154e70 0x7f38a80b4090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38a000c7b0 tx=0x7f38a000cac0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f38ad983700 1 -- 172.21.15.153:0/461550368 reap_dead start 2023-11-28T13:00:27.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 shutdown_connections 2023-11-28T13:00:27.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.925+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 >> 172.21.15.153:0/461550368 conn(0x7f38a80d0000 msgr2=0x7f38a806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.976 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.926+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== osd.1 v2:172.21.15.153:6816/103564 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4720073730 con 0x7f4720070c70 2023-11-28T13:00:27.976 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.926+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 shutdown_connections 2023-11-28T13:00:27.976 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.926+0000 7f38af3e6700 1 -- 172.21.15.153:0/461550368 wait complete. 2023-11-28T13:00:27.976 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.929+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 --> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb374002d40 con 0x7fb378070c20 2023-11-28T13:00:27.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.933+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 --> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f151406c970 con 0x7f151415f950 2023-11-28T13:00:27.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.933+0000 7f1510ff9700 1 -- 172.21.15.153:0/4032565373 <== osd.6 v2:172.21.15.177:6824/103787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f151406c970 con 0x7f151415f950 2023-11-28T13:00:27.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.934+0000 7fb3917fa700 1 -- 172.21.15.153:0/1352300065 <== osd.4 v2:172.21.15.177:6809/103772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb374002d40 con 0x7fb378070c20 2023-11-28T13:00:27.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 --> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4cc0ce930 con 0x7fb4b8070c00 2023-11-28T13:00:27.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fb378070c20 msgr2=0x7fb378073030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb39aed5700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fb378070c20 0x7fb378073030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb4c27fc700 1 -- 172.21.15.153:0/1080188071 <== osd.3 v2:172.21.15.153:6822/103565 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4cc0ce930 con 0x7fb4b8070c00 2023-11-28T13:00:27.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb378068370 msgr2=0x7fb37806a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb39aed5700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb378068370 0x7fb37806a820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb39406f580 tx=0x7fb380009040 comp rx=0 tx=0).stop 2023-11-28T13:00:27.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384003a00 msgr2=0x7fb3940c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb39aed5700 1 --2- 172.21.15.153:0/1352300065 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb384003a00 0x7fb3940c9910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c460 tx=0x7fb3880047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.936+0000 7fb399472700 1 -- 172.21.15.153:0/1352300065 reap_dead start 2023-11-28T13:00:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.937+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7fb4b8070c00 msgr2=0x7fb4b8073010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.937+0000 7fb4d3ef1700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7fb4b8070c00 0x7fb4b8073010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.937+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4b8068450 msgr2=0x7fb4b806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.937+0000 7fb4d3ef1700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4b8068450 0x7fb4b806a900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc14ae00 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2023-11-28T13:00:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.937+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc06e500 msgr2=0x7fb4cc15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.937+0000 7fb4d3ef1700 1 --2- 172.21.15.153:0/1080188071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc06e500 0x7fb4cc15ac30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009c40 tx=0x7fb4c8004750 comp rx=0 tx=0).stop 2023-11-28T13:00:27.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.937+0000 7fb4d248e700 1 -- 172.21.15.153:0/1080188071 reap_dead start 2023-11-28T13:00:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.938+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 shutdown_connections 2023-11-28T13:00:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.938+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 >> 172.21.15.153:0/1352300065 conn(0x7fb394072600 msgr2=0x7fb3940cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.939+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 shutdown_connections 2023-11-28T13:00:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.939+0000 7fb39aed5700 1 -- 172.21.15.153:0/1352300065 wait complete. 2023-11-28T13:00:27.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.939+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 shutdown_connections 2023-11-28T13:00:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.939+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 >> 172.21.15.153:0/1080188071 conn(0x7fb4cc0b0c30 msgr2=0x7fb4cc0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.940+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 shutdown_connections 2023-11-28T13:00:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.940+0000 7fb4d3ef1700 1 -- 172.21.15.153:0/1080188071 wait complete. 2023-11-28T13:00:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.940+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 --> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a80066110 con 0x7f7a60070bf0 2023-11-28T13:00:27.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a777fe700 1 -- 172.21.15.153:0/2760237664 <== osd.7 v2:172.21.15.177:6808/103771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7a80066110 con 0x7f7a60070bf0 2023-11-28T13:00:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f7a60070bf0 msgr2=0x7f7a60073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f7a60070bf0 0x7f7a60073000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a60068340 msgr2=0x7f7a6006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a60068340 0x7f7a6006a7f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7a70000c00 tx=0x7f7a70007990 comp rx=0 tx=0).stop 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2d40 msgr2=0x7f7a80155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 --2- 172.21.15.153:0/2760237664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a800c2d40 0x7f7a80155dc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c4a0 tx=0x7f7a6800c860 comp rx=0 tx=0).stop 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a7ed9d700 1 -- 172.21.15.153:0/2760237664 reap_dead start 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 shutdown_connections 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.941+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 >> 172.21.15.153:0/2760237664 conn(0x7f7a800b12b0 msgr2=0x7f7a8006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.942+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 shutdown_connections 2023-11-28T13:00:27.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.942+0000 7f7a84ae6700 1 -- 172.21.15.153:0/2760237664 wait complete. 2023-11-28T13:00:27.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.949+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 --> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2e6c002d40 con 0x7f2e70070b20 2023-11-28T13:00:27.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.949+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7f151415f950 msgr2=0x7f1514150350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.949+0000 7f151a48d700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7f151415f950 0x7f1514150350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.950+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f14f8068390 msgr2=0x7f14f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.950+0000 7f151a48d700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f14f8068390 0x7f14f806a840 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1504009670 tx=0x7f1504006bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.950+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514164e60 msgr2=0x7f151415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.950+0000 7f151a48d700 1 --2- 172.21.15.153:0/4032565373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1514164e60 0x7f151415de60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f150c00b970 tx=0x7f150c00bc80 comp rx=0 tx=0).stop 2023-11-28T13:00:27.986 INFO:teuthology.orchestra.run.smithi153.stdout:55834574857 2023-11-28T13:00:27.987 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T13:00:27.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.951+0000 7f1518a2a700 1 -- 172.21.15.153:0/4032565373 reap_dead start 2023-11-28T13:00:27.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.950+0000 7f2e81ffb700 1 -- 172.21.15.153:0/1111093366 <== osd.5 v2:172.21.15.177:6800/103770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2e6c002d40 con 0x7f2e70070b20 2023-11-28T13:00:27.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.951+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 shutdown_connections 2023-11-28T13:00:27.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.951+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 >> 172.21.15.153:0/4032565373 conn(0x7f15140cb670 msgr2=0x7f15141623c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.951+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f2e70070b20 msgr2=0x7f2e70072f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.951+0000 7f2e9341c700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f2e70070b20 0x7f2e70072f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.951+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 shutdown_connections 2023-11-28T13:00:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f151a48d700 1 -- 172.21.15.153:0/4032565373 wait complete. 2023-11-28T13:00:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2e700682f0 msgr2=0x7f2e7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2e700682f0 0x7f2e7006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00c830 tx=0x7f2e7c00a000 comp rx=0 tx=0).stop 2023-11-28T13:00:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0b3570 msgr2=0x7f2e8c1ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 --2- 172.21.15.153:0/1111093366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e8c0b3570 0x7f2e8c1ff410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800b450 tx=0x7f2e8800b760 comp rx=0 tx=0).stop 2023-11-28T13:00:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e919b9700 1 -- 172.21.15.153:0/1111093366 reap_dead start 2023-11-28T13:00:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 shutdown_connections 2023-11-28T13:00:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 >> 172.21.15.153:0/1111093366 conn(0x7f2e8c0b12b0 msgr2=0x7f2e8c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 shutdown_connections 2023-11-28T13:00:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.952+0000 7f2e9341c700 1 -- 172.21.15.153:0/1111093366 wait complete. 2023-11-28T13:00:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.954+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 --> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faae00cd5b0 con 0x7faac4070bd0 2023-11-28T13:00:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.954+0000 7faad5ffb700 1 -- 172.21.15.153:0/3116757994 <== osd.2 v2:172.21.15.153:6808/103562 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faae00cd5b0 con 0x7faac4070bd0 2023-11-28T13:00:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.955+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7faac4070bd0 msgr2=0x7faac4072fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.955+0000 7faae7319700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7faac4070bd0 0x7faac4072fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.956+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faac4068320 msgr2=0x7faac406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.956+0000 7faae7319700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faac4068320 0x7faac406a7d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faad0000c00 tx=0x7faad0007000 comp rx=0 tx=0).stop 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.956+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae01595d0 msgr2=0x7faae0156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.956+0000 7faae7319700 1 --2- 172.21.15.153:0/3116757994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae01595d0 0x7faae0156e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faad8004780 tx=0x7faad8003f80 comp rx=0 tx=0).stop 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.956+0000 7faae58b6700 1 -- 172.21.15.153:0/3116757994 reap_dead start 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.956+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 shutdown_connections 2023-11-28T13:00:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.956+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 >> 172.21.15.153:0/3116757994 conn(0x7faae00b12b0 msgr2=0x7faae0158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.993 INFO:teuthology.orchestra.run.smithi153.stdout:60129542153 2023-11-28T13:00:27.993 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T13:00:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.958+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 --> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f473c0c2b80 con 0x7f4720070c70 2023-11-28T13:00:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.958+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 shutdown_connections 2023-11-28T13:00:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.958+0000 7faae7319700 1 -- 172.21.15.153:0/3116757994 wait complete. 2023-11-28T13:00:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.958+0000 7f47327fc700 1 -- 172.21.15.153:0/3571824424 <== osd.1 v2:172.21.15.153:6816/103564 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f473c0c2b80 con 0x7f4720070c70 2023-11-28T13:00:27.995 INFO:teuthology.orchestra.run.smithi153.stdout:64424509449 2023-11-28T13:00:27.995 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T13:00:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.961+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f4720070c70 msgr2=0x7f4720073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.961+0000 7f4743cef700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f4720070c70 0x7f4720073080 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.961+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47200683c0 msgr2=0x7f472006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.961+0000 7f4743cef700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47200683c0 0x7f472006a870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4738009a10 tx=0x7f4738009190 comp rx=0 tx=0).stop 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.961+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f473c159c40 msgr2=0x7f473c14da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.961+0000 7f4743cef700 1 --2- 172.21.15.153:0/3571824424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f473c159c40 0x7f473c14da00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4734014040 tx=0x7f4734004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.962+0000 7f474228c700 1 -- 172.21.15.153:0/3571824424 reap_dead start 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.962+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 shutdown_connections 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.962+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 >> 172.21.15.153:0/3571824424 conn(0x7f473c0cb670 msgr2=0x7f473c157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.962+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 shutdown_connections 2023-11-28T13:00:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:27.962+0000 7f4743cef700 1 -- 172.21.15.153:0/3571824424 wait complete. 2023-11-28T13:00:27.998 INFO:teuthology.orchestra.run.smithi153.stdout:60129542153 2023-11-28T13:00:27.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T13:00:27.999 INFO:teuthology.orchestra.run.smithi153.stdout:60129542153 2023-11-28T13:00:27.999 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T13:00:28.000 INFO:teuthology.orchestra.run.smithi153.stdout:60129542152 2023-11-28T13:00:28.000 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T13:00:28.001 INFO:teuthology.orchestra.run.smithi153.stdout:64424509448 2023-11-28T13:00:28.001 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T13:00:28.002 INFO:teuthology.orchestra.run.smithi153.stdout:64424509449 2023-11-28T13:00:28.002 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T13:00:28.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.229+0000 7f29aff76700 1 Processor -- start 2023-11-28T13:00:28.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.229+0000 7f29aff76700 1 -- start start 2023-11-28T13:00:28.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.230+0000 7f29aff76700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f29a80cacb0 0x7f29a80cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.230+0000 7f29aff76700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a806c2f0 con 0x7f29a80b0d80 2023-11-28T13:00:28.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.230+0000 7f29aff76700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a806b9c0 con 0x7f29a80cde10 2023-11-28T13:00:28.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.230+0000 7f29aff76700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a806bca0 con 0x7f29a80cacb0 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.230+0000 7f29ad511700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f29a80cde10 0x7f29a80b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33364/0 (socket says 172.21.15.153:33364) 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.230+0000 7f29ad511700 1 -- 172.21.15.153:0/2833180567 learned_addr learned my addr 172.21.15.153:0/2833180567 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.230+0000 7f29add12700 1 --2- 172.21.15.153:0/2833180567 >> v2:172.21.15.153:3300/0 conn(0x7f29a80cacb0 0x7f29a80cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29acd10700 1 -- 172.21.15.153:0/2833180567 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621911989 0 0) 0x7f29a806b9c0 con 0x7f29a80cde10 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29acd10700 1 -- 172.21.15.153:0/2833180567 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29940035b0 con 0x7f29a80cde10 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29add12700 1 -- 172.21.15.153:0/2833180567 >> v1:172.21.15.177:6789/0 conn(0x7f29a80cde10 legacy=0x7f29a80b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29add12700 1 -- 172.21.15.153:0/2833180567 >> v1:172.21.15.153:6789/0 conn(0x7f29a80b0d80 legacy=0x7f29a80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29add12700 1 -- 172.21.15.153:0/2833180567 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a806bf80 con 0x7f29a80cacb0 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29add12700 1 --2- 172.21.15.153:0/2833180567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80cacb0 0x7f29a80cb080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2998009840 tx=0x7f2998009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b6934b0827b7b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29acd10700 1 -- 172.21.15.153:0/2833180567 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998004230 con 0x7f29a80cacb0 2023-11-28T13:00:28.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29acd10700 1 -- 172.21.15.153:0/2833180567 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998004390 con 0x7f29a80cacb0 2023-11-28T13:00:28.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.231+0000 7f29acd10700 1 -- 172.21.15.153:0/2833180567 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980036c0 con 0x7f29a80cacb0 2023-11-28T13:00:28.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.232+0000 7f29aff76700 1 -- 172.21.15.153:0/2833180567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80cacb0 msgr2=0x7f29a80cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.232+0000 7f29aff76700 1 --2- 172.21.15.153:0/2833180567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80cacb0 0x7f29a80cb080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2998009840 tx=0x7f2998009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:28.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.234+0000 7f2b804f3700 1 Processor -- start 2023-11-28T13:00:28.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.234+0000 7f2b804f3700 1 -- start start 2023-11-28T13:00:28.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b804f3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b78163720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b804f3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7815c490 con 0x7f2b7806d4f0 2023-11-28T13:00:28.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b804f3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7815c730 con 0x7f2b78158890 2023-11-28T13:00:28.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b804f3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7815ca10 con 0x7f2b780b0d80 2023-11-28T13:00:28.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b7e28f700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f2b78158890 0x7f2b780ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33366/0 (socket says 172.21.15.153:33366) 2023-11-28T13:00:28.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b7e28f700 1 -- 172.21.15.153:0/3034615636 learned_addr learned my addr 172.21.15.153:0/3034615636 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b7ea90700 1 --2- 172.21.15.153:0/3034615636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b78163720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b7ea90700 1 -- 172.21.15.153:0/3034615636 >> v1:172.21.15.177:6789/0 conn(0x7f2b78158890 legacy=0x7f2b780ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b7ea90700 1 -- 172.21.15.153:0/3034615636 >> v1:172.21.15.153:6789/0 conn(0x7f2b7806d4f0 legacy=0x7f2b780b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.236+0000 7f2b7ea90700 1 -- 172.21.15.153:0/3034615636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7815ccf0 con 0x7f2b780b0d80 2023-11-28T13:00:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.238+0000 7f29aff76700 1 -- 172.21.15.153:0/2833180567 shutdown_connections 2023-11-28T13:00:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.238+0000 7f29aff76700 1 --2- 172.21.15.153:0/2833180567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80cacb0 0x7f29a80cb080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.238+0000 7f29aff76700 1 -- 172.21.15.153:0/2833180567 >> 172.21.15.153:0/2833180567 conn(0x7f29a80cb670 msgr2=0x7f29a80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.238+0000 7f29aff76700 1 -- 172.21.15.153:0/2833180567 shutdown_connections 2023-11-28T13:00:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.238+0000 7f29aff76700 1 -- 172.21.15.153:0/2833180567 wait complete. 2023-11-28T13:00:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.238+0000 7f2b7ea90700 1 --2- 172.21.15.153:0/3034615636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b78163720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009dc0 tx=0x7f2b7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=937d6980cf3fe51f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.238+0000 7f2b7d28d700 1 -- 172.21.15.153:0/3034615636 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000e070 con 0x7f2b780b0d80 2023-11-28T13:00:28.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.239+0000 7f29aff76700 1 Processor -- start 2023-11-28T13:00:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.239+0000 7f29aff76700 1 -- start start 2023-11-28T13:00:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.240+0000 7f29aff76700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80b0d80 0x7f29a8206c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.240+0000 7f29aff76700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a80cacb0 0x7f29a80c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.240+0000 7f29aff76700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29a80cde10 0x7f29a816bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.240+0000 7f29aff76700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81f32a0 con 0x7f29a80b0d80 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.240+0000 7f29aff76700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a820adb0 con 0x7f29a80cacb0 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.240+0000 7f29aff76700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a820b090 con 0x7f29a80cde10 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29add12700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80b0d80 0x7f29a8206c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29add12700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80b0d80 0x7f29a8206c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38594/0 (socket says 172.21.15.153:38594) 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ad511700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a80cacb0 0x7f29a80c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29add12700 1 -- 172.21.15.153:0/1783833262 learned_addr learned my addr 172.21.15.153:0/1783833262 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ae513700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29a80cde10 0x7f29a816bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ad511700 1 -- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29a80cde10 msgr2=0x7f29a816bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ad511700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29a80cde10 0x7f29a816bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ad511700 1 -- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80b0d80 msgr2=0x7f29a8206c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ad511700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80b0d80 0x7f29a8206c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ad511700 1 -- 172.21.15.153:0/1783833262 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980094a0 con 0x7f29a80cacb0 2023-11-28T13:00:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.241+0000 7f29ad511700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a80cacb0 0x7f29a80c65e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c7b0 tx=0x7f29a400cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.242+0000 7f2b7d28d700 1 -- 172.21.15.153:0/3034615636 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7000bad0 con 0x7f2b780b0d80 2023-11-28T13:00:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.242+0000 7f2b7d28d700 1 -- 172.21.15.153:0/3034615636 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000be80 con 0x7f2b780b0d80 2023-11-28T13:00:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.243+0000 7f2b804f3700 1 -- 172.21.15.153:0/3034615636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 msgr2=0x7f2b78163720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.243+0000 7f2b804f3700 1 --2- 172.21.15.153:0/3034615636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b78163720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009dc0 tx=0x7f2b7000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:28.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.243+0000 7f299e7fc700 1 -- 172.21.15.153:0/1783833262 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a40074e0 con 0x7f29a80cacb0 2023-11-28T13:00:28.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.243+0000 7f299e7fc700 1 -- 172.21.15.153:0/1783833262 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4007640 con 0x7f29a80cacb0 2023-11-28T13:00:28.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.243+0000 7f299e7fc700 1 -- 172.21.15.153:0/1783833262 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a40079d0 con 0x7f29a80cacb0 2023-11-28T13:00:28.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.243+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a80ce260 con 0x7f29a80cacb0 2023-11-28T13:00:28.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.243+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a806c780 con 0x7f29a80cacb0 2023-11-28T13:00:28.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7fcb627b0700 1 Processor -- start 2023-11-28T13:00:28.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7f2b804f3700 1 -- 172.21.15.153:0/3034615636 shutdown_connections 2023-11-28T13:00:28.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7f2b804f3700 1 --2- 172.21.15.153:0/3034615636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b78163720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7f2b804f3700 1 -- 172.21.15.153:0/3034615636 >> 172.21.15.153:0/3034615636 conn(0x7f2b780cb670 msgr2=0x7f2b780cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29900052f0 con 0x7f29a80cacb0 2023-11-28T13:00:28.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7fcb627b0700 1 -- start start 2023-11-28T13:00:28.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7f2b804f3700 1 -- 172.21.15.153:0/3034615636 shutdown_connections 2023-11-28T13:00:28.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.244+0000 7f2b804f3700 1 -- 172.21.15.153:0/3034615636 wait complete. 2023-11-28T13:00:28.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.245+0000 7fcb627b0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcb5c06e960 0x7fcb5c065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.245+0000 7fcb627b0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb5c06ed30 0x7fcb5c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.245+0000 7fcb627b0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c06ae60 con 0x7fcb5c06b830 2023-11-28T13:00:28.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.245+0000 7fcb627b0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c06a270 con 0x7fcb5c06e960 2023-11-28T13:00:28.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.245+0000 7fcb627b0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c06a530 con 0x7fcb5c06ed30 2023-11-28T13:00:28.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7fcb60d4d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcb5c06b830 0x7fcb5c06c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56764/0 (socket says 172.21.15.153:56764) 2023-11-28T13:00:28.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7fcb60d4d700 1 -- 172.21.15.153:0/4174914851 learned_addr learned my addr 172.21.15.153:0/4174914851 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256239426 0 0) 0x7fcb5c06ae60 con 0x7fcb5c06b830 2023-11-28T13:00:28.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb440035b0 con 0x7fcb5c06b830 2023-11-28T13:00:28.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7f2b804f3700 1 Processor -- start 2023-11-28T13:00:28.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7f2b804f3700 1 -- start start 2023-11-28T13:00:28.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7fcb5b7fe700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb5c06ed30 0x7fcb5c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1219368385 0 0) 0x7fcb440035b0 con 0x7fcb5c06b830 2023-11-28T13:00:28.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb5c06ed30 msgr2=0x7fcb5c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb5c06ed30 0x7fcb5c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 >> v2:172.21.15.177:3300/0 conn(0x7fcb5c06e960 msgr2=0x7fcb5c065ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:00:28.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7f299e7fc700 1 -- 172.21.15.153:0/1783833262 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f29a4019030 con 0x7f29a80cacb0 2023-11-28T13:00:28.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7f299e7fc700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2994068840 0x7f299406acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.246+0000 7f299e7fc700 1 -- 172.21.15.153:0/1783833262 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29a40853d0 con 0x7f29a80cacb0 2023-11-28T13:00:28.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 --2- 172.21.15.153:0/4174914851 >> v2:172.21.15.177:3300/0 conn(0x7fcb5c06e960 0x7fcb5c065ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c06a810 con 0x7fcb5c06b830 2023-11-28T13:00:28.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fcb540019f0 con 0x7fcb5c06b830 2023-11-28T13:00:28.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 >> v1:172.21.15.153:6789/0 conn(0x7fcb5c06b830 legacy=0x7fcb5c06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb44003a00 0x7fcb44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb44004850 0x7fcb44004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb44005200 0x7fcb4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c06a270 con 0x7fcb44004850 2023-11-28T13:00:28.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c06a530 con 0x7fcb44003a00 2023-11-28T13:00:28.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4400c170 con 0x7fcb44005200 2023-11-28T13:00:28.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f29add12700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2994068840 0x7f299406acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5b7fe700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb5c06ed30 0x7fcb5c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:28.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5bfff700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb44003a00 0x7fcb44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b804f3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7806d4f0 0x7f2b780c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b804f3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b780cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b804f3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78158890 0x7f2b7815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b804f3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78159b50 con 0x7f2b78158890 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b804f3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7814cf30 con 0x7f2b7806d4f0 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b804f3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7814c3b0 con 0x7f2b780b0d80 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b7ea90700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78158890 0x7f2b7815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b7ea90700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78158890 0x7f2b7815b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38616/0 (socket says 172.21.15.153:38616) 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b7ea90700 1 -- 172.21.15.153:0/1716679459 learned_addr learned my addr 172.21.15.153:0/1716679459 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f2b7da8e700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b780cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5b7fe700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb44004850 0x7fcb44004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7f29add12700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2994068840 0x7f299406acf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2998009b90 tx=0x7f2998010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5bfff700 1 -- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb44005200 msgr2=0x7fcb4400ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:00:28.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5bfff700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb44005200 0x7fcb4400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5bfff700 1 -- 172.21.15.153:0/4174914851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb44004850 msgr2=0x7fcb44004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5bfff700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb44004850 0x7fcb44004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.247+0000 7fcb5bfff700 1 -- 172.21.15.153:0/4174914851 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440035b0 con 0x7fcb44003a00 2023-11-28T13:00:28.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b7ea90700 1 -- 172.21.15.153:0/1716679459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 msgr2=0x7f2b780cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b7ea90700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b780cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b7ea90700 1 -- 172.21.15.153:0/1716679459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7806d4f0 msgr2=0x7f2b780c2ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:00:28.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b7ea90700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7806d4f0 0x7f2b780c2ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b7ea90700 1 -- 172.21.15.153:0/1716679459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70009a70 con 0x7f2b78158890 2023-11-28T13:00:28.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7fcb5bfff700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb44003a00 0x7fcb44003e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009860 tx=0x7fcb44005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d1e857b8928979 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f5a45997700 1 Processor -- start 2023-11-28T13:00:28.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f5a45997700 1 -- start start 2023-11-28T13:00:28.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b7ea90700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78158890 0x7f2b7815b460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000b020 tx=0x7f2b70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b6effd700 1 -- 172.21.15.153:0/1716679459 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70012630 con 0x7f2b78158890 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b6effd700 1 -- 172.21.15.153:0/1716679459 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70012790 con 0x7f2b78158890 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.248+0000 7f2b6effd700 1 -- 172.21.15.153:0/1716679459 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70012b40 con 0x7f2b78158890 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.249+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b78158c60 con 0x7f2b78158890 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.249+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b780c9370 con 0x7f2b78158890 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.249+0000 7f5a45997700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40160b30 0x7f5a400c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.249+0000 7f5a45997700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5a40164840 0x7f5a400b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.249+0000 7f5a45997700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4006ba50 con 0x7f5a40160f00 2023-11-28T13:00:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.249+0000 7f5a45997700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a40164ce0 con 0x7f5a40164840 2023-11-28T13:00:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.249+0000 7f5a45997700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a40164f90 con 0x7f5a40160b30 2023-11-28T13:00:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5a40164840 0x7f5a400b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3e7fc700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5a40160f00 0x7f5a400cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33400/0 (socket says 172.21.15.153:33400) 2023-11-28T13:00:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5a40164840 0x7f5a400b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38620/0 (socket says 172.21.15.153:38620) 2023-11-28T13:00:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3e7fc700 1 -- 172.21.15.153:0/3983627670 learned_addr learned my addr 172.21.15.153:0/3983627670 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f3484658700 1 Processor -- start 2023-11-28T13:00:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f3484658700 1 -- start start 2023-11-28T13:00:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3effd700 1 --2- 172.21.15.153:0/3983627670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40160b30 0x7f5a400c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3f7fe700 1 -- 172.21.15.153:0/3983627670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40160b30 msgr2=0x7f5a400c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3f7fe700 1 --2- 172.21.15.153:0/3983627670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40160b30 0x7f5a400c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3f7fe700 1 -- 172.21.15.153:0/3983627670 >> v1:172.21.15.177:6789/0 conn(0x7f5a40160f00 legacy=0x7f5a400cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.250+0000 7f5a3f7fe700 1 -- 172.21.15.153:0/3983627670 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40165270 con 0x7f5a40164840 2023-11-28T13:00:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f5a3f7fe700 1 --2- 172.21.15.153:0/3983627670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40164840 0x7f5a400b0670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009d90 tx=0x7f5a3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3eccbf82ed6aa07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f5a3dffb700 1 -- 172.21.15.153:0/3983627670 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800e070 con 0x7f5a40164840 2023-11-28T13:00:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f5a3dffb700 1 -- 172.21.15.153:0/3983627670 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800bb20 con 0x7f5a40164840 2023-11-28T13:00:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f5a3dffb700 1 -- 172.21.15.153:0/3983627670 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38012430 con 0x7f5a40164840 2023-11-28T13:00:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c011620 con 0x7fcb44003a00 2023-11-28T13:00:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c011780 con 0x7fcb44003a00 2023-11-28T13:00:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7fcb5affd700 1 -- 172.21.15.153:0/4174914851 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c011b10 con 0x7fcb44003a00 2023-11-28T13:00:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f3484658700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f347c0d1330 0x7f347c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f3484658700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c0b20b0 0x7f347c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f3484658700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c0b0230 con 0x7f347c0d01b0 2023-11-28T13:00:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f3484658700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347c15cec0 con 0x7f347c0d1330 2023-11-28T13:00:28.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f3484658700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0b4570 con 0x7f347c0b20b0 2023-11-28T13:00:28.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f34823f4700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f347c0d01b0 0x7f347c0d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56788/0 (socket says 172.21.15.153:56788) 2023-11-28T13:00:28.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f34823f4700 1 -- 172.21.15.153:0/1297701666 learned_addr learned my addr 172.21.15.153:0/1297701666 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.251+0000 7f2b6effd700 1 -- 172.21.15.153:0/1716679459 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b70012ca0 con 0x7f2b78158890 2023-11-28T13:00:28.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f2b6effd700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b5c0683c0 0x7f2b5c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f2b6effd700 1 -- 172.21.15.153:0/1716679459 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b700880a0 con 0x7f2b78158890 2023-11-28T13:00:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7fcb627b0700 1 -- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb44003a00 msgr2=0x7fcb44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7fcb627b0700 1 --2- 172.21.15.153:0/4174914851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb44003a00 0x7fcb44003e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009860 tx=0x7fcb44005880 comp rx=0 tx=0).stop 2023-11-28T13:00:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f2b7e28f700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b5c0683c0 0x7f2b5c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934240607 0 0) 0x7f347c0b0230 con 0x7f347c0d01b0 2023-11-28T13:00:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34680035b0 con 0x7f347c0d01b0 2023-11-28T13:00:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243393543 0 0) 0x7f34680035b0 con 0x7f347c0d01b0 2023-11-28T13:00:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c0b20b0 msgr2=0x7f347c0b0c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c0b20b0 0x7f347c0b0c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 >> v2:172.21.15.153:3300/0 conn(0x7f347c0d1330 msgr2=0x7f347c0b06c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 --2- 172.21.15.153:0/1297701666 >> v2:172.21.15.153:3300/0 conn(0x7f347c0d1330 0x7f347c0b06c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0b4850 con 0x7f347c0d01b0 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f3478001670 con 0x7f347c0d01b0 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 >> v1:172.21.15.153:6789/0 conn(0x7f347c0d01b0 legacy=0x7f347c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3468003a00 0x7f3468003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3468004850 0x7f3468004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3468005250 0x7f346800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c15cec0 con 0x7f3468003a00 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0b4570 con 0x7f3468005250 2023-11-28T13:00:28.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.252+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346800c1c0 con 0x7f3468004850 2023-11-28T13:00:28.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f5a45997700 1 -- 172.21.15.153:0/3983627670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40164840 msgr2=0x7f5a400b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f5a45997700 1 --2- 172.21.15.153:0/3983627670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40164840 0x7f5a400b0670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009d90 tx=0x7f5a3800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:28.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f5a45997700 1 -- 172.21.15.153:0/3983627670 shutdown_connections 2023-11-28T13:00:28.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f5a45997700 1 --2- 172.21.15.153:0/3983627670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40164840 0x7f5a400b0670 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f5a45997700 1 --2- 172.21.15.153:0/3983627670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40160b30 0x7f5a400c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f5a45997700 1 -- 172.21.15.153:0/3983627670 >> 172.21.15.153:0/3983627670 conn(0x7f5a400cb670 msgr2=0x7f5a400cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f2b7e28f700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b5c0683c0 0x7f2b5c06a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b68001660 tx=0x7f2b68007a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f34823f4700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3468003a00 0x7f3468003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7fcb60d4d700 1 -- 172.21.15.153:0/4174914851 reap_dead start 2023-11-28T13:00:28.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7fcb627b0700 1 -- 172.21.15.153:0/4174914851 shutdown_connections 2023-11-28T13:00:28.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7fcb627b0700 1 -- 172.21.15.153:0/4174914851 >> 172.21.15.153:0/4174914851 conn(0x7fcb5c0b12b0 msgr2=0x7fcb5c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b780ce8f0 con 0x7f2b78158890 2023-11-28T13:00:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f3481bf3700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3468004850 0x7f3468004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f3482bf5700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3468005250 0x7f346800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f34823f4700 1 -- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3468004850 msgr2=0x7f3468004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f34823f4700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3468004850 0x7f3468004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f34823f4700 1 -- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3468005250 msgr2=0x7f346800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f34823f4700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3468005250 0x7f346800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f34823f4700 1 -- 172.21.15.153:0/1297701666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0b4850 con 0x7f3468003a00 2023-11-28T13:00:28.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7fcb627b0700 1 -- 172.21.15.153:0/4174914851 shutdown_connections 2023-11-28T13:00:28.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f3482bf5700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3468005250 0x7f346800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:28.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.253+0000 7f34823f4700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3468003a00 0x7f3468003e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f347800c3c0 tx=0x7f3468005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc2a7b6821a3fc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.254+0000 7f4e27009700 1 Processor -- start 2023-11-28T13:00:28.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.254+0000 7fcb627b0700 1 -- 172.21.15.153:0/4174914851 wait complete. 2023-11-28T13:00:28.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.254+0000 7f5a45997700 1 -- 172.21.15.153:0/3983627670 shutdown_connections 2023-11-28T13:00:28.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.254+0000 7f4e27009700 1 -- start start 2023-11-28T13:00:28.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e27009700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e20160b30 0x7f4e200c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.254+0000 7f5a45997700 1 -- 172.21.15.153:0/3983627670 wait complete. 2023-11-28T13:00:28.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f5a45997700 1 Processor -- start 2023-11-28T13:00:28.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e27009700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 0x7f4e20164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e27009700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2006ba50 con 0x7f4e200cc730 2023-11-28T13:00:28.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e27009700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e20164d80 con 0x7f4e20160b30 2023-11-28T13:00:28.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e27009700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20164f90 con 0x7f4e20160f00 2023-11-28T13:00:28.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e24da5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e20160b30 0x7f4e200c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e24da5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e20160b30 0x7f4e200c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38634/0 (socket says 172.21.15.153:38634) 2023-11-28T13:00:28.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.255+0000 7f4e24da5700 1 -- 172.21.15.153:0/4087081183 learned_addr learned my addr 172.21.15.153:0/4087081183 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7fcb627b0700 1 Processor -- start 2023-11-28T13:00:28.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f4e24da5700 1 -- 172.21.15.153:0/4087081183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 msgr2=0x7f4e20164840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:00:28.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f4e24da5700 1 --2- 172.21.15.153:0/4087081183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 0x7f4e20164840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f4e24da5700 1 -- 172.21.15.153:0/4087081183 >> v1:172.21.15.153:6789/0 conn(0x7f4e200cc730 legacy=0x7f4e200b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f4e24da5700 1 -- 172.21.15.153:0/4087081183 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20165270 con 0x7f4e20160b30 2023-11-28T13:00:28.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f4e24da5700 1 --2- 172.21.15.153:0/4087081183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e20160b30 0x7f4e200c1e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e140098f0 tx=0x7f4e14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b7f0b9fd5e1e0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7fcb627b0700 1 -- start start 2023-11-28T13:00:28.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f5a45997700 1 -- start start 2023-11-28T13:00:28.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f5a45997700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40160b30 0x7f5a4015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f5a45997700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40160f00 0x7f5a40163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.256+0000 7f5a45997700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40164840 0x7f5a40162210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a45997700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4015df40 con 0x7f5a40160f00 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a45997700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400b4af0 con 0x7f5a40160b30 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a45997700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a401643c0 con 0x7f5a40164840 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40160b30 0x7f5a4015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40164840 0x7f5a40162210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40160b30 0x7f5a4015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:48106/0 (socket says 172.21.15.153:48106) 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40164840 0x7f5a40162210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:56254/0 (socket says 172.21.15.153:56254) 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 -- 172.21.15.153:0/1683070896 learned_addr learned my addr 172.21.15.153:0/1683070896 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3e7fc700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40160f00 0x7f5a40163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f299e7fc700 1 -- 172.21.15.153:0/1783833262 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f29a4051bc0 con 0x7f29a80cacb0 2023-11-28T13:00:28.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 -- 172.21.15.153:0/1683070896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40164840 msgr2=0x7f5a40162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40164840 0x7f5a40162210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 -- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40160f00 msgr2=0x7f5a40163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40160f00 0x7f5a40163940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 -- 172.21.15.153:0/1683070896 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009a70 con 0x7f5a40160b30 2023-11-28T13:00:28.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3e7fc700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40160f00 0x7f5a40163940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:28.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7fcb627b0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb44003a00 0x7fcb5c0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7fcb627b0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c0b49a0 0x7fcb5c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7fcb627b0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c0c8490 0x7fcb5c14e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7fcb627b0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c14ff20 con 0x7fcb5c0c8490 2023-11-28T13:00:28.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7fcb627b0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c159690 con 0x7fcb5c0b49a0 2023-11-28T13:00:28.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7fcb627b0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0ae2d0 con 0x7fcb44003a00 2023-11-28T13:00:28.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7f5a3effd700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40160b30 0x7f5a4015da00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000ea10 tx=0x7f5a3000ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.257+0000 7fcb5b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c0b49a0 0x7fcb5c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c0c8490 0x7fcb5c14e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c0c8490 0x7fcb5c14e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38650/0 (socket says 172.21.15.153:38650) 2023-11-28T13:00:28.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 -- 172.21.15.153:0/79454722 learned_addr learned my addr 172.21.15.153:0/79454722 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7f5a27fff700 1 -- 172.21.15.153:0/1683070896 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a30009710 con 0x7f5a40160b30 2023-11-28T13:00:28.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 -- 172.21.15.153:0/79454722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb44003a00 msgr2=0x7fcb5c0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb44003a00 0x7fcb5c0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 -- 172.21.15.153:0/79454722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c0b49a0 msgr2=0x7fcb5c0c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c0b49a0 0x7fcb5c0c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 -- 172.21.15.153:0/79454722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c066430 con 0x7fcb5c0c8490 2023-11-28T13:00:28.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7fcb60d4d700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c0c8490 0x7fcb5c14e3d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c06be40 tx=0x7fcb5400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7f5a27fff700 1 -- 172.21.15.153:0/1683070896 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3000c850 con 0x7f5a40160b30 2023-11-28T13:00:28.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7f5a27fff700 1 -- 172.21.15.153:0/1683070896 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000cbc0 con 0x7f5a40160b30 2023-11-28T13:00:28.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a400cce40 con 0x7f5a40160b30 2023-11-28T13:00:28.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.258+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a40165550 con 0x7f5a40160b30 2023-11-28T13:00:28.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.259+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a200052f0 con 0x7f5a40160b30 2023-11-28T13:00:28.289 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.260+0000 7f2b6effd700 1 -- 172.21.15.153:0/1716679459 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2b70054890 con 0x7f2b78158890 2023-11-28T13:00:28.289 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.260+0000 7f4e1f7fe700 1 -- 172.21.15.153:0/4087081183 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14004230 con 0x7f4e20160b30 2023-11-28T13:00:28.289 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.260+0000 7f4e1f7fe700 1 -- 172.21.15.153:0/4087081183 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e14004390 con 0x7f4e20160b30 2023-11-28T13:00:28.289 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.260+0000 7f4e1f7fe700 1 -- 172.21.15.153:0/4087081183 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e140036a0 con 0x7f4e20160b30 2023-11-28T13:00:28.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.261+0000 7f5a27fff700 1 -- 172.21.15.153:0/1683070896 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5a30019030 con 0x7f5a40160b30 2023-11-28T13:00:28.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.261+0000 7f5a27fff700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5a2c068340 0x7f5a2c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.261+0000 7f5a27fff700 1 -- 172.21.15.153:0/1683070896 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a300862d0 con 0x7f5a40160b30 2023-11-28T13:00:28.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.261+0000 7f4e27009700 1 -- 172.21.15.153:0/4087081183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e20160b30 msgr2=0x7f4e200c1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.261+0000 7f4e27009700 1 --2- 172.21.15.153:0/4087081183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e20160b30 0x7f4e200c1e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e140098f0 tx=0x7f4e14009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:28.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.261+0000 7f5a3e7fc700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5a2c068340 0x7f5a2c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.262+0000 7f5a3e7fc700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5a2c068340 0x7f5a2c06a7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a400c57c0 tx=0x7f5a34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.264+0000 7f5a27fff700 1 -- 172.21.15.153:0/1683070896 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5a30052bd0 con 0x7f5a40160b30 2023-11-28T13:00:28.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.265+0000 7f1a45395700 1 Processor -- start 2023-11-28T13:00:28.292 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.265+0000 7f1a45395700 1 -- start start 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.266+0000 7f1a45395700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1a4006e9f0 0x7f1a4006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.266+0000 7f1a45395700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006b830 0x7f1a4006c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.266+0000 7f1a45395700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4006ae60 con 0x7f1a4006ced0 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.266+0000 7f1a45395700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4006d330 con 0x7f1a4006e9f0 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.266+0000 7f1a45395700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4006a270 con 0x7f1a4006b830 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.266+0000 7f1a3effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1a4006e9f0 0x7f1a4006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1a4006ced0 0x7f1a400ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56822/0 (socket says 172.21.15.153:56822) 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3f7fe700 1 -- 172.21.15.153:0/1477794693 learned_addr learned my addr 172.21.15.153:0/1477794693 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3e7fc700 1 --2- 172.21.15.153:0/1477794693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006b830 0x7f1a4006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7fcb58ff9700 1 -- 172.21.15.153:0/79454722 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400ad30 con 0x7fcb5c0c8490 2023-11-28T13:00:28.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478007ca0 con 0x7f3468003a00 2023-11-28T13:00:28.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478005960 con 0x7f3468003a00 2023-11-28T13:00:28.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f34813f2700 1 -- 172.21.15.153:0/1297701666 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478005cf0 con 0x7f3468003a00 2023-11-28T13:00:28.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7fcb58ff9700 1 -- 172.21.15.153:0/79454722 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5400ae90 con 0x7fcb5c0c8490 2023-11-28T13:00:28.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7fcb58ff9700 1 -- 172.21.15.153:0/79454722 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540072f0 con 0x7fcb5c0c8490 2023-11-28T13:00:28.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3dffb700 1 -- 172.21.15.153:0/1477794693 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821598179 0 0) 0x7f1a4006ae60 con 0x7f1a4006ced0 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3e7fc700 1 -- 172.21.15.153:0/1477794693 >> v2:172.21.15.153:3300/0 conn(0x7f1a4006e9f0 msgr2=0x7f1a4006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3e7fc700 1 --2- 172.21.15.153:0/1477794693 >> v2:172.21.15.153:3300/0 conn(0x7f1a4006e9f0 0x7f1a4006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3e7fc700 1 -- 172.21.15.153:0/1477794693 >> v1:172.21.15.153:6789/0 conn(0x7f1a4006ced0 legacy=0x7f1a400ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3e7fc700 1 -- 172.21.15.153:0/1477794693 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4006a530 con 0x7f1a4006b830 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.267+0000 7f1a3e7fc700 1 --2- 172.21.15.153:0/1477794693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006b830 0x7f1a4006c990 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009840 tx=0x7f1a34009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4bde3564a3b6f08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f4e27009700 1 -- 172.21.15.153:0/4087081183 shutdown_connections 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f4e27009700 1 --2- 172.21.15.153:0/4087081183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e20160b30 0x7f4e200c1e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f4e27009700 1 --2- 172.21.15.153:0/4087081183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 0x7f4e20164840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f4e27009700 1 -- 172.21.15.153:0/4087081183 >> 172.21.15.153:0/4087081183 conn(0x7f4e200cb670 msgr2=0x7f4e200cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f1a3dffb700 1 -- 172.21.15.153:0/1477794693 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400f8a0 con 0x7f1a4006b830 2023-11-28T13:00:28.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f1a3dffb700 1 -- 172.21.15.153:0/1477794693 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3400fa00 con 0x7f1a4006b830 2023-11-28T13:00:28.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f1a3dffb700 1 -- 172.21.15.153:0/1477794693 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400fd90 con 0x7f1a4006b830 2023-11-28T13:00:28.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f3484658700 1 -- 172.21.15.153:0/1297701666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3468003a00 msgr2=0x7f3468003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f3484658700 1 --2- 172.21.15.153:0/1297701666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3468003a00 0x7f3468003e10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f347800c3c0 tx=0x7f3468005880 comp rx=0 tx=0).stop 2023-11-28T13:00:28.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c06aaf0 con 0x7fcb5c0c8490 2023-11-28T13:00:28.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c0665c0 con 0x7fcb5c0c8490 2023-11-28T13:00:28.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f1a45395700 1 -- 172.21.15.153:0/1477794693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006b830 msgr2=0x7f1a4006c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f1a45395700 1 --2- 172.21.15.153:0/1477794693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006b830 0x7f1a4006c990 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009840 tx=0x7f1a34009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:28.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.268+0000 7f3482bf5700 1 -- 172.21.15.153:0/1297701666 reap_dead start 2023-11-28T13:00:28.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.269+0000 7f3484658700 1 -- 172.21.15.153:0/1297701666 shutdown_connections 2023-11-28T13:00:28.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.269+0000 7f3484658700 1 -- 172.21.15.153:0/1297701666 >> 172.21.15.153:0/1297701666 conn(0x7f347c09c3f0 msgr2=0x7f347c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.269+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c0cd5b0 con 0x7fcb5c0c8490 2023-11-28T13:00:28.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.270+0000 7fcb58ff9700 1 -- 172.21.15.153:0/79454722 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb5401a030 con 0x7fcb5c0c8490 2023-11-28T13:00:28.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.270+0000 7f1a45395700 1 -- 172.21.15.153:0/1477794693 shutdown_connections 2023-11-28T13:00:28.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.270+0000 7f1a45395700 1 --2- 172.21.15.153:0/1477794693 >> v2:172.21.15.153:3300/0 conn(0x7f1a4006e9f0 0x7f1a4006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.270+0000 7f1a45395700 1 --2- 172.21.15.153:0/1477794693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006b830 0x7f1a4006c990 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.270+0000 7f1a45395700 1 -- 172.21.15.153:0/1477794693 >> 172.21.15.153:0/1477794693 conn(0x7f1a400b12b0 msgr2=0x7f1a400b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.270+0000 7fcb58ff9700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb48068340 0x7fcb4806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.270+0000 7fcb58ff9700 1 -- 172.21.15.153:0/79454722 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb54085a40 con 0x7fcb5c0c8490 2023-11-28T13:00:28.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.271+0000 7fcb5bfff700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb48068340 0x7fcb4806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.271+0000 7f90f965f700 1 Processor -- start 2023-11-28T13:00:28.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.271+0000 7f90f965f700 1 -- start start 2023-11-28T13:00:28.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f4e27009700 1 -- 172.21.15.153:0/4087081183 shutdown_connections 2023-11-28T13:00:28.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f4e27009700 1 -- 172.21.15.153:0/4087081183 wait complete. 2023-11-28T13:00:28.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.271+0000 7fcb5bfff700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb48068340 0x7fcb4806a7f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c014040 tx=0x7fcb4c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f90f965f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f90f40d1330 0x7f90f40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f90f965f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f40b0230 con 0x7f90f40d01b0 2023-11-28T13:00:28.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f90f965f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f40b4570 con 0x7f90f40b20b0 2023-11-28T13:00:28.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f90f965f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f415cec0 con 0x7f90f40d1330 2023-11-28T13:00:28.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f90f2ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f90f40d01b0 0x7f90f40d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56826/0 (socket says 172.21.15.153:56826) 2023-11-28T13:00:28.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.272+0000 7f90f2ffd700 1 -- 172.21.15.153:0/1886893062 learned_addr learned my addr 172.21.15.153:0/1886893062 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.274+0000 7f90f27fc700 1 --2- 172.21.15.153:0/1886893062 >> v2:172.21.15.177:3300/0 conn(0x7f90f40d1330 0x7f90f40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.274+0000 7f4e27009700 1 Processor -- start 2023-11-28T13:00:28.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.274+0000 7f3484658700 1 -- 172.21.15.153:0/1297701666 shutdown_connections 2023-11-28T13:00:28.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612613937 0 0) 0x7f90f40b4570 con 0x7f90f40b20b0 2023-11-28T13:00:28.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90dc0035b0 con 0x7f90f40b20b0 2023-11-28T13:00:28.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1742037129 0 0) 0x7f90dc0035b0 con 0x7f90f40b20b0 2023-11-28T13:00:28.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 >> v2:172.21.15.177:3300/0 conn(0x7f90f40d1330 msgr2=0x7f90f40b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 --2- 172.21.15.153:0/1886893062 >> v2:172.21.15.177:3300/0 conn(0x7f90f40d1330 0x7f90f40b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 >> v1:172.21.15.153:6789/0 conn(0x7f90f40d01b0 legacy=0x7f90f40d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40b4850 con 0x7f90f40b20b0 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f90ec0019f0 con 0x7f90f40b20b0 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 >> v1:172.21.15.177:6789/0 conn(0x7f90f40b20b0 legacy=0x7f90f40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90dc003a00 0x7f90dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc004850 0x7f90dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90dc005200 0x7f90dc014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f415cec0 con 0x7f90dc005200 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc014890 con 0x7f90dc003a00 2023-11-28T13:00:28.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc014a50 con 0x7f90dc004850 2023-11-28T13:00:28.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.274+0000 7f4e27009700 1 -- start start 2023-11-28T13:00:28.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f4e27009700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e200cc730 0x7f4e200c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f4e27009700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e20160b30 0x7f4e201575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f4e27009700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 0x7f4e2015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f4e27009700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200c8ba0 con 0x7f4e200cc730 2023-11-28T13:00:28.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f4e27009700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200c83a0 con 0x7f4e20160b30 2023-11-28T13:00:28.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.275+0000 7f4e27009700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200b1a20 con 0x7f4e20160f00 2023-11-28T13:00:28.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7fcb58ff9700 1 -- 172.21.15.153:0/79454722 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcb54052340 con 0x7fcb5c0c8490 2023-11-28T13:00:28.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f37fe700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90dc005200 0x7f90dc014220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f2ffd700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90dc003a00 0x7f90dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.274+0000 7f3484658700 1 -- 172.21.15.153:0/1297701666 wait complete. 2023-11-28T13:00:28.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f3484658700 1 Processor -- start 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f2ffd700 1 -- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc004850 msgr2=0x7f90dc004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f2ffd700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc004850 0x7f90dc004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f2ffd700 1 -- 172.21.15.153:0/1886893062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90dc005200 msgr2=0x7f90dc014220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f2ffd700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90dc005200 0x7f90dc014220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f2ffd700 1 -- 172.21.15.153:0/1886893062 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40b4850 con 0x7f90dc003a00 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f37fe700 1 -- 172.21.15.153:0/1886893062 reap_dead start 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f2ffd700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90dc003a00 0x7f90dc003e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f90e40018d0 tx=0x7f90dc005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c47ae879b09d30a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4004b30 con 0x7f90dc003a00 2023-11-28T13:00:28.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4004c90 con 0x7f90dc003a00 2023-11-28T13:00:28.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f90f1ffb700 1 -- 172.21.15.153:0/1886893062 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e40070d0 con 0x7f90dc003a00 2023-11-28T13:00:28.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e200cc730 0x7f4e200c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e200cc730 0x7f4e200c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38676/0 (socket says 172.21.15.153:38676) 2023-11-28T13:00:28.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 -- 172.21.15.153:0/60746747 learned_addr learned my addr 172.21.15.153:0/60746747 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e255a6700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 0x7f4e2015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 -- 172.21.15.153:0/60746747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 msgr2=0x7f4e2015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 0x7f4e2015efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.310 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 -- 172.21.15.153:0/60746747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e20160b30 msgr2=0x7f4e201575f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:00:28.310 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e20160b30 0x7f4e201575f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.310 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f4e24da5700 1 -- 172.21.15.153:0/60746747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e14009550 con 0x7f4e200cc730 2023-11-28T13:00:28.310 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f4e24da5700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e200cc730 0x7f4e200c7e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e14000c00 tx=0x7f4e14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.310 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f1a45395700 1 -- 172.21.15.153:0/1477794693 shutdown_connections 2023-11-28T13:00:28.310 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f90f965f700 1 -- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90dc003a00 msgr2=0x7f90dc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f90f965f700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90dc003a00 0x7f90dc003e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f90e40018d0 tx=0x7f90dc005880 comp rx=0 tx=0).stop 2023-11-28T13:00:28.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.276+0000 7f3484658700 1 -- start start 2023-11-28T13:00:28.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3484658700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c1fe910 0x7f347c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3484658700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f347c1fece0 0x7f347c0b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3484658700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f347c15d540 0x7f347c0b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3484658700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0b6120 con 0x7f347c15d540 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3484658700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c15e2f0 con 0x7f347c1fece0 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3484658700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c15da50 con 0x7f347c1fe910 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f34823f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c1fe910 0x7f347c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3482bf5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f347c15d540 0x7f347c0b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3482bf5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f347c15d540 0x7f347c0b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38680/0 (socket says 172.21.15.153:38680) 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f3482bf5700 1 -- 172.21.15.153:0/347299948 learned_addr learned my addr 172.21.15.153:0/347299948 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3481bf3700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f347c1fece0 0x7f347c0b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f90f965f700 1 -- 172.21.15.153:0/1886893062 shutdown_connections 2023-11-28T13:00:28.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f90f965f700 1 --2- 172.21.15.153:0/1886893062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90dc003a00 0x7f90dc003e10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f90f965f700 1 -- 172.21.15.153:0/1886893062 >> 172.21.15.153:0/1886893062 conn(0x7f90f409c3f0 msgr2=0x7f90f40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3481bf3700 1 -- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c1fe910 msgr2=0x7f347c15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3481bf3700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c1fe910 0x7f347c15d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3481bf3700 1 -- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f347c15d540 msgr2=0x7f347c0b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3481bf3700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f347c15d540 0x7f347c0b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3481bf3700 1 -- 172.21.15.153:0/347299948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0afec0 con 0x7f347c1fece0 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3481bf3700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f347c1fece0 0x7f347c0b8370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f347400de30 tx=0x7f347400de60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3472ffd700 1 -- 172.21.15.153:0/347299948 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400bc90 con 0x7f347c1fece0 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3472ffd700 1 -- 172.21.15.153:0/347299948 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3474012070 con 0x7f347c1fece0 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.278+0000 7f3472ffd700 1 -- 172.21.15.153:0/347299948 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400e660 con 0x7f347c1fece0 2023-11-28T13:00:28.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.279+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c0afbe0 con 0x7f347c1fece0 2023-11-28T13:00:28.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.279+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c0b2600 con 0x7f347c1fece0 2023-11-28T13:00:28.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.277+0000 7f1a45395700 1 -- 172.21.15.153:0/1477794693 wait complete. 2023-11-28T13:00:28.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.279+0000 7f1a45395700 1 Processor -- start 2023-11-28T13:00:28.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.280+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c0b5e80 con 0x7f347c1fece0 2023-11-28T13:00:28.316 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.280+0000 7f1a45395700 1 -- start start 2023-11-28T13:00:28.316 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a45395700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4006b830 0x7f1a400b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.316 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a45395700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006ced0 0x7f1a400b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.316 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a45395700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a4006e9f0 0x7f1a401e6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.316 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a45395700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40158750 con 0x7f1a4006e9f0 2023-11-28T13:00:28.316 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a45395700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400b2bd0 con 0x7f1a4006b830 2023-11-28T13:00:28.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a45395700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400b2eb0 con 0x7f1a4006ced0 2023-11-28T13:00:28.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a3f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a4006e9f0 0x7f1a401e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a3f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a4006e9f0 0x7f1a401e6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38688/0 (socket says 172.21.15.153:38688) 2023-11-28T13:00:28.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a3f7fe700 1 -- 172.21.15.153:0/1210967544 learned_addr learned my addr 172.21.15.153:0/1210967544 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.280+0000 7f3472ffd700 1 -- 172.21.15.153:0/347299948 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f347400e7c0 con 0x7f347c1fece0 2023-11-28T13:00:28.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f3472ffd700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34600682f0 0x7f346006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f3472ffd700 1 -- 172.21.15.153:0/347299948 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3474085c00 con 0x7f347c1fece0 2023-11-28T13:00:28.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f34823f4700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34600682f0 0x7f346006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a3effd700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4006b830 0x7f1a400b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.281+0000 7f1a3e7fc700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006ced0 0x7f1a400b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3effd700 1 -- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006ced0 msgr2=0x7f1a400b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3effd700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006ced0 0x7f1a400b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3effd700 1 -- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a4006e9f0 msgr2=0x7f1a401e6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3effd700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a4006e9f0 0x7f1a401e6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3effd700 1 -- 172.21.15.153:0/1210967544 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340094a0 con 0x7f1a4006b830 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3e7fc700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006ced0 0x7f1a400b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3f7fe700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a4006e9f0 0x7f1a401e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f1a3effd700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4006b830 0x7f1a400b25f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e870 tx=0x7f1a3000eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.282+0000 7f34823f4700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34600682f0 0x7f346006a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3478015040 tx=0x7f347800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.283+0000 7f90f965f700 1 -- 172.21.15.153:0/1886893062 shutdown_connections 2023-11-28T13:00:28.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.283+0000 7f4e1d7fa700 1 -- 172.21.15.153:0/60746747 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14003d00 con 0x7f4e200cc730 2023-11-28T13:00:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.283+0000 7f4e1d7fa700 1 -- 172.21.15.153:0/60746747 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1401f070 con 0x7f4e200cc730 2023-11-28T13:00:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.283+0000 7f4e1d7fa700 1 -- 172.21.15.153:0/60746747 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1400b880 con 0x7f4e200cc730 2023-11-28T13:00:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.283+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e200b0d80 con 0x7f4e200cc730 2023-11-28T13:00:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.283+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e20165550 con 0x7f4e200cc730 2023-11-28T13:00:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.283+0000 7f90f965f700 1 -- 172.21.15.153:0/1886893062 wait complete. 2023-11-28T13:00:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.284+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e200b0f10 con 0x7f4e200cc730 2023-11-28T13:00:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.285+0000 7f90f965f700 1 Processor -- start 2023-11-28T13:00:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.285+0000 7f3472ffd700 1 -- 172.21.15.153:0/347299948 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34740505e0 con 0x7f347c1fece0 2023-11-28T13:00:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.285+0000 7f90f965f700 1 -- start start 2023-11-28T13:00:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.286+0000 7f90f965f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc003a00 0x7f90f4156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.286+0000 7f90f965f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40ca750 0x7f90f40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.286+0000 7f90f965f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f40cb0e0 0x7f90f40b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.286+0000 7f90f965f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f4156fb0 con 0x7f90f40ca750 2023-11-28T13:00:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.286+0000 7f90f965f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40b71a0 con 0x7f90f40cb0e0 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.286+0000 7f90f965f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40b7480 con 0x7f90dc003a00 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.286+0000 7f90f2ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc003a00 0x7f90f4156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.287+0000 7f90f2ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc003a00 0x7f90f4156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:56304/0 (socket says 172.21.15.153:56304) 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.287+0000 7f90f2ffd700 1 -- 172.21.15.153:0/1178694000 learned_addr learned my addr 172.21.15.153:0/1178694000 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.287+0000 7f90f37fe700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f40cb0e0 0x7f90f40b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f90f37fe700 1 -- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc003a00 msgr2=0x7f90f4156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f90f37fe700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc003a00 0x7f90f4156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f90f37fe700 1 -- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40ca750 msgr2=0x7f90f40caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f90f37fe700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40ca750 0x7f90f40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f90f37fe700 1 -- 172.21.15.153:0/1178694000 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40afec0 con 0x7f90f40cb0e0 2023-11-28T13:00:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f90f37fe700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f40cb0e0 0x7f90f40b8240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90f406aa00 tx=0x7f90ec00a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f4e1d7fa700 1 -- 172.21.15.153:0/60746747 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e14017020 con 0x7f4e200cc730 2023-11-28T13:00:28.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f4e1d7fa700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e04068340 0x7f4e0406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.289+0000 7f4e1d7fa700 1 -- 172.21.15.153:0/60746747 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e14087820 con 0x7f4e200cc730 2023-11-28T13:00:28.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.290+0000 7f4e1ffff700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e04068340 0x7f4e0406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.290+0000 7f4e1ffff700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e04068340 0x7f4e0406a7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c006fd0 tx=0x7f4e0c006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.292+0000 7f1a277fe700 1 -- 172.21.15.153:0/1210967544 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30013680 con 0x7f1a4006b830 2023-11-28T13:00:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.292+0000 7f1a277fe700 1 -- 172.21.15.153:0/1210967544 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a300137e0 con 0x7f1a4006b830 2023-11-28T13:00:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.292+0000 7f1a277fe700 1 -- 172.21.15.153:0/1210967544 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30013b70 con 0x7f1a4006b830 2023-11-28T13:00:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.292+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4006aaf0 con 0x7f1a4006b830 2023-11-28T13:00:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.292+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4006a810 con 0x7f1a4006b830 2023-11-28T13:00:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.293+0000 7f90db7fe700 1 -- 172.21.15.153:0/1178694000 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00aba0 con 0x7f90f40cb0e0 2023-11-28T13:00:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.293+0000 7f90db7fe700 1 -- 172.21.15.153:0/1178694000 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec00ad20 con 0x7f90f40cb0e0 2023-11-28T13:00:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.293+0000 7f90db7fe700 1 -- 172.21.15.153:0/1178694000 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec013630 con 0x7f90f40cb0e0 2023-11-28T13:00:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.293+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f40afbe0 con 0x7f90f40cb0e0 2023-11-28T13:00:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.293+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f40b2600 con 0x7f90f40cb0e0 2023-11-28T13:00:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.295+0000 7f4e1d7fa700 1 -- 172.21.15.153:0/60746747 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e14054010 con 0x7f4e200cc730 2023-11-28T13:00:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f1a277fe700 1 -- 172.21.15.153:0/1210967544 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a30019030 con 0x7f1a4006b830 2023-11-28T13:00:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a200052f0 con 0x7f1a4006b830 2023-11-28T13:00:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f1a277fe700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a2c0683c0 0x7f1a2c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f1a277fe700 1 -- 172.21.15.153:0/1210967544 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a300859b0 con 0x7f1a4006b830 2023-11-28T13:00:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f90db7fe700 1 -- 172.21.15.153:0/1178694000 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90ec013790 con 0x7f90f40cb0e0 2023-11-28T13:00:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f90db7fe700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00683c0 0x7f90e006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f90db7fe700 1 -- 172.21.15.153:0/1178694000 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f90ec086470 con 0x7f90f40cb0e0 2023-11-28T13:00:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.296+0000 7f90f2ffd700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00683c0 0x7f90e006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.297+0000 7f90f2ffd700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00683c0 0x7f90e006a870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f90e4015040 tx=0x7f90e400a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.301+0000 7f1a277fe700 1 -- 172.21.15.153:0/1210967544 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a300521a0 con 0x7f1a4006b830 2023-11-28T13:00:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.306+0000 7f1a3e7fc700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a2c0683c0 0x7f1a2c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.307+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d40052f0 con 0x7f90f40cb0e0 2023-11-28T13:00:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.307+0000 7f1a3e7fc700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a2c0683c0 0x7f1a2c06a870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a34008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.319+0000 7f90db7fe700 1 -- 172.21.15.153:0/1178694000 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90ec052c40 con 0x7f90f40cb0e0 2023-11-28T13:00:28.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.638+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f29900055a0 con 0x7f29a80cacb0 2023-11-28T13:00:28.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.639+0000 7f299e7fc700 1 -- 172.21.15.153:0/1783833262 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29a40551e0 con 0x7f29a80cacb0 2023-11-28T13:00:28.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.640+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcb5c06dcb0 con 0x7fcb5c0c8490 2023-11-28T13:00:28.645 INFO:teuthology.orchestra.run.smithi153.stdout:55834574857 2023-11-28T13:00:28.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.641+0000 7fcb58ff9700 1 -- 172.21.15.153:0/79454722 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb54018020 con 0x7fcb5c0c8490 2023-11-28T13:00:28.646 INFO:teuthology.orchestra.run.smithi153.stdout:64424509448 2023-11-28T13:00:28.646 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.643+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2994068840 msgr2=0x7f299406acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.646 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.643+0000 7f29aff76700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2994068840 0x7f299406acf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2998009b90 tx=0x7f2998010000 comp rx=0 tx=0).stop 2023-11-28T13:00:28.646 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.643+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a80cacb0 msgr2=0x7f29a80c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.646 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.643+0000 7f29aff76700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a80cacb0 0x7f29a80c65e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c7b0 tx=0x7f29a400cac0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb48068340 msgr2=0x7fcb4806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7fcb627b0700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb48068340 0x7fcb4806a7f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c014040 tx=0x7fcb4c010000 comp rx=0 tx=0).stop 2023-11-28T13:00:28.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c0c8490 msgr2=0x7fcb5c14e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7fcb627b0700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c0c8490 0x7fcb5c14e3d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c06be40 tx=0x7fcb5400a340 comp rx=0 tx=0).stop 2023-11-28T13:00:28.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 shutdown_connections 2023-11-28T13:00:28.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2994068840 0x7f299406acf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29a80b0d80 0x7f29a8206c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29a80cde10 0x7f29a816bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 --2- 172.21.15.153:0/1783833262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a80cacb0 0x7f29a80c65e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 >> 172.21.15.153:0/1783833262 conn(0x7f29a80cb670 msgr2=0x7f29a8070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 shutdown_connections 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.644+0000 7f29aff76700 1 -- 172.21.15.153:0/1783833262 wait complete. 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.648+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 shutdown_connections 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.648+0000 7fcb627b0700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb48068340 0x7fcb4806a7f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.648+0000 7fcb627b0700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb5c0c8490 0x7fcb5c14e3d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.648+0000 7fcb627b0700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb44003a00 0x7fcb5c0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.648+0000 7fcb627b0700 1 --2- 172.21.15.153:0/79454722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5c0b49a0 0x7fcb5c0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.648+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 >> 172.21.15.153:0/79454722 conn(0x7fcb5c0b12b0 msgr2=0x7fcb5c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.649+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 shutdown_connections 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.649+0000 7fcb627b0700 1 -- 172.21.15.153:0/79454722 wait complete. 2023-11-28T13:00:28.657 INFO:teuthology.orchestra.run.smithi153.stdout:64424509449 2023-11-28T13:00:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.658+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5a20005870 con 0x7f5a40160b30 2023-11-28T13:00:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.652+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2b780745c0 con 0x7f2b78158890 2023-11-28T13:00:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.652+0000 7f2b6effd700 1 -- 172.21.15.153:0/1716679459 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b70057eb0 con 0x7f2b78158890 2023-11-28T13:00:28.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.659+0000 7f5a27fff700 1 -- 172.21.15.153:0/1683070896 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a300561f0 con 0x7f5a40160b30 2023-11-28T13:00:28.661 INFO:teuthology.orchestra.run.smithi153.stdout:60129542153 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.659+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b5c0683c0 msgr2=0x7f2b5c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.659+0000 7f2b804f3700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b5c0683c0 0x7f2b5c06a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b68001660 tx=0x7f2b68007a30 comp rx=0 tx=0).stop 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.659+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78158890 msgr2=0x7f2b7815b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.659+0000 7f2b804f3700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78158890 0x7f2b7815b460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000b020 tx=0x7f2b70004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 shutdown_connections 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b5c0683c0 0x7f2b5c06a870 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b68001660 tx=0x7f2b68007a30 comp rx=0 tx=0).stop 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b78158890 0x7f2b7815b460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7806d4f0 0x7f2b780c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 --2- 172.21.15.153:0/1716679459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b780b0d80 0x7f2b780cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 >> 172.21.15.153:0/1716679459 conn(0x7f2b780cb670 msgr2=0x7f2b78156ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 shutdown_connections 2023-11-28T13:00:28.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.660+0000 7f2b804f3700 1 -- 172.21.15.153:0/1716679459 wait complete. 2023-11-28T13:00:28.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2023-11-28T13:00:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.664+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5a2c068340 msgr2=0x7f5a2c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.664+0000 7f5a45997700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5a2c068340 0x7f5a2c06a7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a400c57c0 tx=0x7f5a34009040 comp rx=0 tx=0).stop 2023-11-28T13:00:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.664+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40160b30 msgr2=0x7f5a4015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.664+0000 7f5a45997700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40160b30 0x7f5a4015da00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000ea10 tx=0x7f5a3000ed20 comp rx=0 tx=0).stop 2023-11-28T13:00:28.666 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.665+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 shutdown_connections 2023-11-28T13:00:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.665+0000 7f5a45997700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5a40164840 0x7f5a40162210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.665+0000 7f5a45997700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40160b30 0x7f5a4015da00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.665+0000 7f5a45997700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5a2c068340 0x7f5a2c06a7f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.665+0000 7f5a45997700 1 --2- 172.21.15.153:0/1683070896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a40160f00 0x7f5a40163940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.665+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 >> 172.21.15.153:0/1683070896 conn(0x7f5a400cb670 msgr2=0x7f5a400702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.669+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 shutdown_connections 2023-11-28T13:00:28.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.669+0000 7f5a45997700 1 -- 172.21.15.153:0/1683070896 wait complete. 2023-11-28T13:00:28.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-11-28T13:00:28.672 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.672+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f347c0b6260 con 0x7f347c1fece0 2023-11-28T13:00:28.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.672+0000 7f3472ffd700 1 -- 172.21.15.153:0/347299948 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f347c0b6260 con 0x7f347c1fece0 2023-11-28T13:00:28.674 INFO:teuthology.orchestra.run.smithi153.stdout:64424509449 2023-11-28T13:00:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.676+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34600682f0 msgr2=0x7f346006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.676+0000 7f3484658700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34600682f0 0x7f346006a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3478015040 tx=0x7f347800a000 comp rx=0 tx=0).stop 2023-11-28T13:00:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.676+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f347c1fece0 msgr2=0x7f347c0b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.676+0000 7f3484658700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f347c1fece0 0x7f347c0b8370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f347400de30 tx=0x7f347400de60 comp rx=0 tx=0).stop 2023-11-28T13:00:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 shutdown_connections 2023-11-28T13:00:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34600682f0 0x7f346006a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f347c15d540 0x7f347c0b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f347c1fe910 0x7f347c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 --2- 172.21.15.153:0/347299948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f347c1fece0 0x7f347c0b8370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 >> 172.21.15.153:0/347299948 conn(0x7f347c09c3f0 msgr2=0x7f347c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 shutdown_connections 2023-11-28T13:00:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.677+0000 7f3484658700 1 -- 172.21.15.153:0/347299948 wait complete. 2023-11-28T13:00:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.678+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4e200b4ba0 con 0x7f4e200cc730 2023-11-28T13:00:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.678+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1a200055a0 con 0x7f1a4006b830 2023-11-28T13:00:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.678+0000 7f4e1d7fa700 1 -- 172.21.15.153:0/60746747 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e14003e60 con 0x7f4e200cc730 2023-11-28T13:00:28.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.678+0000 7f1a277fe700 1 -- 172.21.15.153:0/1210967544 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a300557c0 con 0x7f1a4006b830 2023-11-28T13:00:28.681 INFO:teuthology.orchestra.run.smithi153.stdout:60129542153 2023-11-28T13:00:28.681 INFO:teuthology.orchestra.run.smithi153.stdout:60129542152 2023-11-28T13:00:28.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.681+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e04068340 msgr2=0x7f4e0406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.681+0000 7f4e27009700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e04068340 0x7f4e0406a7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c006fd0 tx=0x7f4e0c006bc0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.681+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e200cc730 msgr2=0x7f4e200c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.681+0000 7f4e27009700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e200cc730 0x7f4e200c7e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e14000c00 tx=0x7f4e14004750 comp rx=0 tx=0).stop 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a2c0683c0 msgr2=0x7f1a2c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a2c0683c0 0x7f1a2c06a870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a34008030 comp rx=0 tx=0).stop 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4006b830 msgr2=0x7f1a400b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4006b830 0x7f1a400b25f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e870 tx=0x7f1a3000eb80 comp rx=0 tx=0).stop 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 shutdown_connections 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a2c0683c0 0x7f1a2c06a870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a4006e9f0 0x7f1a401e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4006b830 0x7f1a400b25f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 --2- 172.21.15.153:0/1210967544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4006ced0 0x7f1a400b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.682+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 >> 172.21.15.153:0/1210967544 conn(0x7f1a400b12b0 msgr2=0x7f1a400b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2023-11-28T13:00:28.685 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 shutdown_connections 2023-11-28T13:00:28.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f1a45395700 1 -- 172.21.15.153:0/1210967544 wait complete. 2023-11-28T13:00:28.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 shutdown_connections 2023-11-28T13:00:28.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e04068340 0x7f4e0406a7f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e200cc730 0x7f4e200c7e60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e20160f00 0x7f4e2015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 --2- 172.21.15.153:0/60746747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e20160b30 0x7f4e201575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 >> 172.21.15.153:0/60746747 conn(0x7f4e200cb670 msgr2=0x7f4e20070540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 shutdown_connections 2023-11-28T13:00:28.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.687+0000 7f4e27009700 1 -- 172.21.15.153:0/60746747 wait complete. 2023-11-28T13:00:28.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.4 2023-11-28T13:00:28.695 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.1 2023-11-28T13:00:28.696 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.704+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f90d40055a0 con 0x7f90f40cb0e0 2023-11-28T13:00:28.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.704+0000 7f90db7fe700 1 -- 172.21.15.153:0/1178694000 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90ec017020 con 0x7f90f40cb0e0 2023-11-28T13:00:28.706 INFO:teuthology.orchestra.run.smithi153.stdout:60129542153 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00683c0 msgr2=0x7f90e006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00683c0 0x7f90e006a870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f90e4015040 tx=0x7f90e400a000 comp rx=0 tx=0).stop 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f40cb0e0 msgr2=0x7f90f40b8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f40cb0e0 0x7f90f40b8240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f90f406aa00 tx=0x7f90ec00a390 comp rx=0 tx=0).stop 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 shutdown_connections 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90f40cb0e0 0x7f90f40b8240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90dc003a00 0x7f90f4156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00683c0 0x7f90e006a870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 --2- 172.21.15.153:0/1178694000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f40ca750 0x7f90f40caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 >> 172.21.15.153:0/1178694000 conn(0x7f90f409c3f0 msgr2=0x7f90f40d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 shutdown_connections 2023-11-28T13:00:28.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.707+0000 7f90f965f700 1 -- 172.21.15.153:0/1178694000 wait complete. 2023-11-28T13:00:28.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.5 2023-11-28T13:00:28.711 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2023-11-28T13:00:28.712 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.7 2023-11-28T13:00:28.721 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:28.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T13:00:28.722 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:28.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.879+0000 7f0008cc3700 1 Processor -- start 2023-11-28T13:00:28.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.880+0000 7f0008cc3700 1 -- start start 2023-11-28T13:00:28.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.880+0000 7f0008cc3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f000406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.880+0000 7f0008cc3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0004160540 0x7f00041609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.880+0000 7f0008cc3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000415d060 con 0x7f0004071040 2023-11-28T13:00:28.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.880+0000 7f0008cc3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000415d340 con 0x7f0004160540 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.880+0000 7f0008cc3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000415d620 con 0x7f000406dee0 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000259c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0004071040 0x7f0004071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56868/0 (socket says 172.21.15.153:56868) 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000259c700 1 -- 172.21.15.153:0/2931893781 learned_addr learned my addr 172.21.15.153:0/2931893781 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f0002d9d700 1 --2- 172.21.15.153:0/2931893781 >> v2:172.21.15.177:3300/0 conn(0x7f0004160540 0x7f00041609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f0001d9b700 1 --2- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f000406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000159a700 1 -- 172.21.15.153:0/2931893781 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230904091 0 0) 0x7f000415d060 con 0x7f0004071040 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000159a700 1 -- 172.21.15.153:0/2931893781 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effec0035b0 con 0x7f0004071040 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f0002d9d700 1 -- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 msgr2=0x7f000406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f0002d9d700 1 --2- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f000406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000159a700 1 -- 172.21.15.153:0/2931893781 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 806600888 0 0) 0x7effec0035b0 con 0x7f0004071040 2023-11-28T13:00:28.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f0002d9d700 1 -- 172.21.15.153:0/2931893781 >> v1:172.21.15.153:6789/0 conn(0x7f0004071040 legacy=0x7f0004071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f0002d9d700 1 -- 172.21.15.153:0/2931893781 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000415d900 con 0x7f0004160540 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f0002d9d700 1 --2- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 0x7f00041609c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efffc009d90 tx=0x7efffc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b45f0198c14a876f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000159a700 1 -- 172.21.15.153:0/2931893781 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc00e070 con 0x7f0004160540 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000159a700 1 -- 172.21.15.153:0/2931893781 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc003a20 con 0x7f0004160540 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.881+0000 7f000159a700 1 -- 172.21.15.153:0/2931893781 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8001670 con 0x7f0004160540 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 -- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 msgr2=0x7f00041609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 0x7f00041609c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efffc009d90 tx=0x7efffc00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 -- 172.21.15.153:0/2931893781 shutdown_connections 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 0x7f00041609c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2931893781 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f000406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 -- 172.21.15.153:0/2931893781 >> 172.21.15.153:0/2931893781 conn(0x7f00040d0000 msgr2=0x7f00040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 -- 172.21.15.153:0/2931893781 shutdown_connections 2023-11-28T13:00:28.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.882+0000 7f0008cc3700 1 -- 172.21.15.153:0/2931893781 wait complete. 2023-11-28T13:00:28.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.883+0000 7f0008cc3700 1 Processor -- start 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.883+0000 7f0008cc3700 1 -- start start 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0008cc3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f0004156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0008cc3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004071040 0x7f00040afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0008cc3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 0x7f00041f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0008cc3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00041f2200 con 0x7f0004071040 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0008cc3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00041f1f00 con 0x7f0004160540 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0008cc3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00040b4170 con 0x7f000406dee0 2023-11-28T13:00:28.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004071040 0x7f00040afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004071040 0x7f00040afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38712/0 (socket says 172.21.15.153:38712) 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 -- 172.21.15.153:0/2124455357 learned_addr learned my addr 172.21.15.153:0/2124455357 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0002d9d700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 0x7f00041f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f000259c700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f0004156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 -- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 msgr2=0x7f0004156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f0004156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 -- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 msgr2=0x7f00041f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 0x7f00041f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.884+0000 7f0001d9b700 1 -- 172.21.15.153:0/2124455357 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc009a70 con 0x7f0004071040 2023-11-28T13:00:28.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.885+0000 7f0001d9b700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004071040 0x7f00040afc70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efff400e8a0 tx=0x7efff400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.885+0000 7efff2ffd700 1 -- 172.21.15.153:0/2124455357 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4009590 con 0x7f0004071040 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.885+0000 7efff2ffd700 1 -- 172.21.15.153:0/2124455357 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400eea0 con 0x7f0004071040 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.885+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000415ccd0 con 0x7f0004071040 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.885+0000 7efff2ffd700 1 -- 172.21.15.153:0/2124455357 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400ca50 con 0x7f0004071040 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.885+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000406e870 con 0x7f0004071040 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.886+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000406a270 con 0x7f0004071040 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.886+0000 7efff2ffd700 1 -- 172.21.15.153:0/2124455357 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efff4019030 con 0x7f0004071040 2023-11-28T13:00:28.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.886+0000 7efff2ffd700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7effe40682f0 0x7effe406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.887+0000 7efff2ffd700 1 -- 172.21.15.153:0/2124455357 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efff4085ad0 con 0x7f0004071040 2023-11-28T13:00:28.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.887+0000 7f000259c700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7effe40682f0 0x7effe406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.887+0000 7f000259c700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7effe40682f0 0x7effe406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efff8001eb0 tx=0x7efff8001c40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:28.891+0000 7efff2ffd700 1 -- 172.21.15.153:0/2124455357 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efff4052330 con 0x7f0004071040 2023-11-28T13:00:29.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.069+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00041f2340 con 0x7effe40682f0 2023-11-28T13:00:29.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.077+0000 7efff2ffd700 1 -- 172.21.15.153:0/2124455357 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286618 (secure 0 0 0) 0x7f00041f2340 con 0x7effe40682f0 2023-11-28T13:00:29.082 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:29.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7effe40682f0 msgr2=0x7effe406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7effe40682f0 0x7effe406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efff8001eb0 tx=0x7efff8001c40 comp rx=0 tx=0).stop 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004071040 msgr2=0x7f00040afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004071040 0x7f00040afc70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efff400e8a0 tx=0x7efff400ec60 comp rx=0 tx=0).stop 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 shutdown_connections 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0004160540 0x7f00041f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f000406dee0 0x7f0004156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7effe40682f0 0x7effe406a7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 --2- 172.21.15.153:0/2124455357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004071040 0x7f00040afc70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 >> 172.21.15.153:0/2124455357 conn(0x7f00040d0000 msgr2=0x7f000415f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 shutdown_connections 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.080+0000 7f0008cc3700 1 -- 172.21.15.153:0/2124455357 wait complete. 2023-11-28T13:00:29.085 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:00:29.094 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-28T13:00:28.144657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66404,"kb_used_data":3236,"kb_used_omap":0,"kb_used_meta":63104,"kb_avail":754908316,"statfs":{"total":773094113280,"available":773026115584,"internally_reserved":0,"allocated":3313664,"data_stored":1959483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64618496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009156"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217825+0000","last_change":"2023-11-28T13:00:17.635428+0000","last_active":"2023-11-28T13:00:19.217825+0000","last_peered":"2023-11-28T13:00:19.217825+0000","last_clean":"2023-11-28T13:00:19.217825+0000","last_became_active":"2023-11-28T13:00:17.635148+0000","last_became_peered":"2023-11-28T13:00:17.635148+0000","last_unstale":"2023-11-28T13:00:19.217825+0000","last_undegraded":"2023-11-28T13:00:19.217825+0000","last_fullsized":"2023-11-28T13:00:19.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338757+0000","last_change":"2023-11-28T13:00:17.645017+0000","last_active":"2023-11-28T13:00:18.338757+0000","last_peered":"2023-11-28T13:00:18.338757+0000","last_clean":"2023-11-28T13:00:18.338757+0000","last_became_active":"2023-11-28T13:00:17.644524+0000","last_became_peered":"2023-11-28T13:00:17.644524+0000","last_unstale":"2023-11-28T13:00:18.338757+0000","last_undegraded":"2023-11-28T13:00:18.338757+0000","last_fullsized":"2023-11-28T13:00:18.338757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497428+0000","last_change":"2023-11-28T13:00:17.648662+0000","last_active":"2023-11-28T13:00:18.497428+0000","last_peered":"2023-11-28T13:00:18.497428+0000","last_clean":"2023-11-28T13:00:18.497428+0000","last_became_active":"2023-11-28T13:00:17.648271+0000","last_became_peered":"2023-11-28T13:00:17.648271+0000","last_unstale":"2023-11-28T13:00:18.497428+0000","last_undegraded":"2023-11-28T13:00:18.497428+0000","last_fullsized":"2023-11-28T13:00:18.497428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:37.103258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330841+0000","last_change":"2023-11-28T13:00:17.643733+0000","last_active":"2023-11-28T13:00:18.330841+0000","last_peered":"2023-11-28T13:00:18.330841+0000","last_clean":"2023-11-28T13:00:18.330841+0000","last_became_active":"2023-11-28T13:00:17.643428+0000","last_became_peered":"2023-11-28T13:00:17.643428+0000","last_unstale":"2023-11-28T13:00:18.330841+0000","last_undegraded":"2023-11-28T13:00:18.330841+0000","last_fullsized":"2023-11-28T13:00:18.330841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652247+0000","last_change":"2023-11-28T13:00:17.650740+0000","last_active":"2023-11-28T13:00:17.652247+0000","last_peered":"2023-11-28T13:00:17.652247+0000","last_clean":"2023-11-28T13:00:17.652247+0000","last_became_active":"2023-11-28T13:00:17.650427+0000","last_became_peered":"2023-11-28T13:00:17.650427+0000","last_unstale":"2023-11-28T13:00:17.652247+0000","last_undegraded":"2023-11-28T13:00:17.652247+0000","last_fullsized":"2023-11-28T13:00:17.652247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497199+0000","last_change":"2023-11-28T13:00:17.652679+0000","last_active":"2023-11-28T13:00:18.497199+0000","last_peered":"2023-11-28T13:00:18.497199+0000","last_clean":"2023-11-28T13:00:18.497199+0000","last_became_active":"2023-11-28T13:00:17.648169+0000","last_became_peered":"2023-11-28T13:00:17.648169+0000","last_unstale":"2023-11-28T13:00:18.497199+0000","last_undegraded":"2023-11-28T13:00:18.497199+0000","last_fullsized":"2023-11-28T13:00:18.497199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:58.762522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331071+0000","last_change":"2023-11-28T13:00:17.647769+0000","last_active":"2023-11-28T13:00:18.331071+0000","last_peered":"2023-11-28T13:00:18.331071+0000","last_clean":"2023-11-28T13:00:18.331071+0000","last_became_active":"2023-11-28T13:00:17.647476+0000","last_became_peered":"2023-11-28T13:00:17.647476+0000","last_unstale":"2023-11-28T13:00:18.331071+0000","last_undegraded":"2023-11-28T13:00:18.331071+0000","last_fullsized":"2023-11-28T13:00:18.331071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338605+0000","last_change":"2023-11-28T13:00:17.653233+0000","last_active":"2023-11-28T13:00:18.338605+0000","last_peered":"2023-11-28T13:00:18.338605+0000","last_clean":"2023-11-28T13:00:18.338605+0000","last_became_active":"2023-11-28T13:00:17.652957+0000","last_became_peered":"2023-11-28T13:00:17.652957+0000","last_unstale":"2023-11-28T13:00:18.338605+0000","last_undegraded":"2023-11-28T13:00:18.338605+0000","last_fullsized":"2023-11-28T13:00:18.338605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652321+0000","last_change":"2023-11-28T13:00:17.650826+0000","last_active":"2023-11-28T13:00:17.652321+0000","last_peered":"2023-11-28T13:00:17.652321+0000","last_clean":"2023-11-28T13:00:17.652321+0000","last_became_active":"2023-11-28T13:00:17.650572+0000","last_became_peered":"2023-11-28T13:00:17.650572+0000","last_unstale":"2023-11-28T13:00:17.652321+0000","last_undegraded":"2023-11-28T13:00:17.652321+0000","last_fullsized":"2023-11-28T13:00:17.652321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338780+0000","last_change":"2023-11-28T13:00:17.642144+0000","last_active":"2023-11-28T13:00:18.338780+0000","last_peered":"2023-11-28T13:00:18.338780+0000","last_clean":"2023-11-28T13:00:18.338780+0000","last_became_active":"2023-11-28T13:00:17.641814+0000","last_became_peered":"2023-11-28T13:00:17.641814+0000","last_unstale":"2023-11-28T13:00:18.338780+0000","last_undegraded":"2023-11-28T13:00:18.338780+0000","last_fullsized":"2023-11-28T13:00:18.338780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497444+0000","last_change":"2023-11-28T13:00:17.643190+0000","last_active":"2023-11-28T13:00:18.497444+0000","last_peered":"2023-11-28T13:00:18.497444+0000","last_clean":"2023-11-28T13:00:18.497444+0000","last_became_active":"2023-11-28T13:00:17.642741+0000","last_became_peered":"2023-11-28T13:00:17.642741+0000","last_unstale":"2023-11-28T13:00:18.497444+0000","last_undegraded":"2023-11-28T13:00:18.497444+0000","last_fullsized":"2023-11-28T13:00:18.497444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:33.241895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217739+0000","last_change":"2023-11-28T13:00:17.654503+0000","last_active":"2023-11-28T13:00:19.217739+0000","last_peered":"2023-11-28T13:00:19.217739+0000","last_clean":"2023-11-28T13:00:19.217739+0000","last_became_active":"2023-11-28T13:00:17.654204+0000","last_became_peered":"2023-11-28T13:00:17.654204+0000","last_unstale":"2023-11-28T13:00:19.217739+0000","last_undegraded":"2023-11-28T13:00:19.217739+0000","last_fullsized":"2023-11-28T13:00:19.217739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:27.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217678+0000","last_change":"2023-11-28T13:00:17.634718+0000","last_active":"2023-11-28T13:00:19.217678+0000","last_peered":"2023-11-28T13:00:19.217678+0000","last_clean":"2023-11-28T13:00:19.217678+0000","last_became_active":"2023-11-28T13:00:17.634420+0000","last_became_peered":"2023-11-28T13:00:17.634420+0000","last_unstale":"2023-11-28T13:00:19.217678+0000","last_undegraded":"2023-11-28T13:00:19.217678+0000","last_fullsized":"2023-11-28T13:00:19.217678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.567102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338634+0000","last_change":"2023-11-28T13:00:17.641361+0000","last_active":"2023-11-28T13:00:18.338634+0000","last_peered":"2023-11-28T13:00:18.338634+0000","last_clean":"2023-11-28T13:00:18.338634+0000","last_became_active":"2023-11-28T13:00:17.641078+0000","last_became_peered":"2023-11-28T13:00:17.641078+0000","last_unstale":"2023-11-28T13:00:18.338634+0000","last_undegraded":"2023-11-28T13:00:18.338634+0000","last_fullsized":"2023-11-28T13:00:18.338634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:38.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331110+0000","last_change":"2023-11-28T13:00:17.647800+0000","last_active":"2023-11-28T13:00:18.331110+0000","last_peered":"2023-11-28T13:00:18.331110+0000","last_clean":"2023-11-28T13:00:18.331110+0000","last_became_active":"2023-11-28T13:00:17.647519+0000","last_became_peered":"2023-11-28T13:00:17.647519+0000","last_unstale":"2023-11-28T13:00:18.331110+0000","last_undegraded":"2023-11-28T13:00:18.331110+0000","last_fullsized":"2023-11-28T13:00:18.331110+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497110+0000","last_change":"2023-11-28T13:00:17.651090+0000","last_active":"2023-11-28T13:00:18.497110+0000","last_peered":"2023-11-28T13:00:18.497110+0000","last_clean":"2023-11-28T13:00:18.497110+0000","last_became_active":"2023-11-28T13:00:17.649174+0000","last_became_peered":"2023-11-28T13:00:17.649174+0000","last_unstale":"2023-11-28T13:00:18.497110+0000","last_undegraded":"2023-11-28T13:00:18.497110+0000","last_fullsized":"2023-11-28T13:00:18.497110+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:32.070272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652338+0000","last_change":"2023-11-28T13:00:17.650764+0000","last_active":"2023-11-28T13:00:17.652338+0000","last_peered":"2023-11-28T13:00:17.652338+0000","last_clean":"2023-11-28T13:00:17.652338+0000","last_became_active":"2023-11-28T13:00:17.650252+0000","last_became_peered":"2023-11-28T13:00:17.650252+0000","last_unstale":"2023-11-28T13:00:17.652338+0000","last_undegraded":"2023-11-28T13:00:17.652338+0000","last_fullsized":"2023-11-28T13:00:17.652338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497486+0000","last_change":"2023-11-28T13:00:17.648771+0000","last_active":"2023-11-28T13:00:18.497486+0000","last_peered":"2023-11-28T13:00:18.497486+0000","last_clean":"2023-11-28T13:00:18.497486+0000","last_became_active":"2023-11-28T13:00:17.646367+0000","last_became_peered":"2023-11-28T13:00:17.646367+0000","last_unstale":"2023-11-28T13:00:18.497486+0000","last_undegraded":"2023-11-28T13:00:18.497486+0000","last_fullsized":"2023-11-28T13:00:18.497486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:33.549492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217793+0000","last_change":"2023-11-28T13:00:17.654191+0000","last_active":"2023-11-28T13:00:19.217793+0000","last_peered":"2023-11-28T13:00:19.217793+0000","last_clean":"2023-11-28T13:00:19.217793+0000","last_became_active":"2023-11-28T13:00:17.653922+0000","last_became_peered":"2023-11-28T13:00:17.653922+0000","last_unstale":"2023-11-28T13:00:19.217793+0000","last_undegraded":"2023-11-28T13:00:19.217793+0000","last_fullsized":"2023-11-28T13:00:19.217793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.066202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497368+0000","last_change":"2023-11-28T13:00:17.653882+0000","last_active":"2023-11-28T13:00:18.497368+0000","last_peered":"2023-11-28T13:00:18.497368+0000","last_clean":"2023-11-28T13:00:18.497368+0000","last_became_active":"2023-11-28T13:00:17.653497+0000","last_became_peered":"2023-11-28T13:00:17.653497+0000","last_unstale":"2023-11-28T13:00:18.497368+0000","last_undegraded":"2023-11-28T13:00:18.497368+0000","last_fullsized":"2023-11-28T13:00:18.497368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:59.591407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331153+0000","last_change":"2023-11-28T13:00:17.652105+0000","last_active":"2023-11-28T13:00:18.331153+0000","last_peered":"2023-11-28T13:00:18.331153+0000","last_clean":"2023-11-28T13:00:18.331153+0000","last_became_active":"2023-11-28T13:00:17.651804+0000","last_became_peered":"2023-11-28T13:00:17.651804+0000","last_unstale":"2023-11-28T13:00:18.331153+0000","last_undegraded":"2023-11-28T13:00:18.331153+0000","last_fullsized":"2023-11-28T13:00:18.331153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650622+0000","last_change":"2023-11-28T13:00:17.647496+0000","last_active":"2023-11-28T13:00:17.650622+0000","last_peered":"2023-11-28T13:00:17.650622+0000","last_clean":"2023-11-28T13:00:17.650622+0000","last_became_active":"2023-11-28T13:00:17.647234+0000","last_became_peered":"2023-11-28T13:00:17.647234+0000","last_unstale":"2023-11-28T13:00:17.650622+0000","last_undegraded":"2023-11-28T13:00:17.650622+0000","last_fullsized":"2023-11-28T13:00:17.650622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215336+0000","last_change":"2023-11-28T13:00:17.645773+0000","last_active":"2023-11-28T13:00:19.215336+0000","last_peered":"2023-11-28T13:00:19.215336+0000","last_clean":"2023-11-28T13:00:19.215336+0000","last_became_active":"2023-11-28T13:00:17.643545+0000","last_became_peered":"2023-11-28T13:00:17.643545+0000","last_unstale":"2023-11-28T13:00:19.215336+0000","last_undegraded":"2023-11-28T13:00:19.215336+0000","last_fullsized":"2023-11-28T13:00:19.215336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:36.504863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331033+0000","last_change":"2023-11-28T13:00:17.651625+0000","last_active":"2023-11-28T13:00:18.331033+0000","last_peered":"2023-11-28T13:00:18.331033+0000","last_clean":"2023-11-28T13:00:18.331033+0000","last_became_active":"2023-11-28T13:00:17.651210+0000","last_became_peered":"2023-11-28T13:00:17.651210+0000","last_unstale":"2023-11-28T13:00:18.331033+0000","last_undegraded":"2023-11-28T13:00:18.331033+0000","last_fullsized":"2023-11-28T13:00:18.331033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635880+0000","last_change":"2023-11-28T13:00:17.646690+0000","last_active":"2023-11-28T13:00:18.635880+0000","last_peered":"2023-11-28T13:00:18.635880+0000","last_clean":"2023-11-28T13:00:18.635880+0000","last_became_active":"2023-11-28T13:00:17.646438+0000","last_became_peered":"2023-11-28T13:00:17.646438+0000","last_unstale":"2023-11-28T13:00:18.635880+0000","last_undegraded":"2023-11-28T13:00:18.635880+0000","last_fullsized":"2023-11-28T13:00:18.635880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:52.870009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636047+0000","last_change":"2023-11-28T13:00:17.641404+0000","last_active":"2023-11-28T13:00:18.636047+0000","last_peered":"2023-11-28T13:00:18.636047+0000","last_clean":"2023-11-28T13:00:18.636047+0000","last_became_active":"2023-11-28T13:00:17.641140+0000","last_became_peered":"2023-11-28T13:00:17.641140+0000","last_unstale":"2023-11-28T13:00:18.636047+0000","last_undegraded":"2023-11-28T13:00:18.636047+0000","last_fullsized":"2023-11-28T13:00:18.636047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338691+0000","last_change":"2023-11-28T13:00:17.645292+0000","last_active":"2023-11-28T13:00:18.338691+0000","last_peered":"2023-11-28T13:00:18.338691+0000","last_clean":"2023-11-28T13:00:18.338691+0000","last_became_active":"2023-11-28T13:00:17.645001+0000","last_became_peered":"2023-11-28T13:00:17.645001+0000","last_unstale":"2023-11-28T13:00:18.338691+0000","last_undegraded":"2023-11-28T13:00:18.338691+0000","last_fullsized":"2023-11-28T13:00:18.338691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497240+0000","last_change":"2023-11-28T13:00:17.245858+0000","last_active":"2023-11-28T13:00:18.497240+0000","last_peered":"2023-11-28T13:00:18.497240+0000","last_clean":"2023-11-28T13:00:18.497240+0000","last_became_active":"2023-11-28T13:00:17.245568+0000","last_became_peered":"2023-11-28T13:00:17.245568+0000","last_unstale":"2023-11-28T13:00:18.497240+0000","last_undegraded":"2023-11-28T13:00:18.497240+0000","last_fullsized":"2023-11-28T13:00:18.497240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215475+0000","last_change":"2023-11-28T13:00:17.644504+0000","last_active":"2023-11-28T13:00:19.215475+0000","last_peered":"2023-11-28T13:00:19.215475+0000","last_clean":"2023-11-28T13:00:19.215475+0000","last_became_active":"2023-11-28T13:00:17.644099+0000","last_became_peered":"2023-11-28T13:00:17.644099+0000","last_unstale":"2023-11-28T13:00:19.215475+0000","last_undegraded":"2023-11-28T13:00:19.215475+0000","last_fullsized":"2023-11-28T13:00:19.215475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635846+0000","last_change":"2023-11-28T13:00:17.236124+0000","last_active":"2023-11-28T13:00:18.635846+0000","last_peered":"2023-11-28T13:00:18.635846+0000","last_clean":"2023-11-28T13:00:18.635846+0000","last_became_active":"2023-11-28T13:00:17.235500+0000","last_became_peered":"2023-11-28T13:00:17.235500+0000","last_unstale":"2023-11-28T13:00:18.635846+0000","last_undegraded":"2023-11-28T13:00:18.635846+0000","last_fullsized":"2023-11-28T13:00:18.635846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:19.252999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648987+0000","last_change":"2023-11-28T13:00:17.642697+0000","last_active":"2023-11-28T13:00:17.648987+0000","last_peered":"2023-11-28T13:00:17.648987+0000","last_clean":"2023-11-28T13:00:17.648987+0000","last_became_active":"2023-11-28T13:00:17.642400+0000","last_became_peered":"2023-11-28T13:00:17.642400+0000","last_unstale":"2023-11-28T13:00:17.648987+0000","last_undegraded":"2023-11-28T13:00:17.648987+0000","last_fullsized":"2023-11-28T13:00:17.648987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215390+0000","last_change":"2023-11-28T13:00:17.240445+0000","last_active":"2023-11-28T13:00:19.215390+0000","last_peered":"2023-11-28T13:00:19.215390+0000","last_clean":"2023-11-28T13:00:19.215390+0000","last_became_active":"2023-11-28T13:00:17.240095+0000","last_became_peered":"2023-11-28T13:00:17.240095+0000","last_unstale":"2023-11-28T13:00:19.215390+0000","last_undegraded":"2023-11-28T13:00:19.215390+0000","last_fullsized":"2023-11-28T13:00:19.215390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330869+0000","last_change":"2023-11-28T13:00:17.644329+0000","last_active":"2023-11-28T13:00:18.330869+0000","last_peered":"2023-11-28T13:00:18.330869+0000","last_clean":"2023-11-28T13:00:18.330869+0000","last_became_active":"2023-11-28T13:00:17.644046+0000","last_became_peered":"2023-11-28T13:00:17.644046+0000","last_unstale":"2023-11-28T13:00:18.330869+0000","last_undegraded":"2023-11-28T13:00:18.330869+0000","last_fullsized":"2023-11-28T13:00:18.330869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.849042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631440+0000","last_change":"2023-11-28T13:00:17.242514+0000","last_active":"2023-11-28T13:00:17.631440+0000","last_peered":"2023-11-28T13:00:17.631440+0000","last_clean":"2023-11-28T13:00:17.631440+0000","last_became_active":"2023-11-28T13:00:17.242258+0000","last_became_peered":"2023-11-28T13:00:17.242258+0000","last_unstale":"2023-11-28T13:00:17.631440+0000","last_undegraded":"2023-11-28T13:00:17.631440+0000","last_fullsized":"2023-11-28T13:00:17.631440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215295+0000","last_change":"2023-11-28T13:00:17.642442+0000","last_active":"2023-11-28T13:00:19.215295+0000","last_peered":"2023-11-28T13:00:19.215295+0000","last_clean":"2023-11-28T13:00:19.215295+0000","last_became_active":"2023-11-28T13:00:17.642162+0000","last_became_peered":"2023-11-28T13:00:17.642162+0000","last_unstale":"2023-11-28T13:00:19.215295+0000","last_undegraded":"2023-11-28T13:00:19.215295+0000","last_fullsized":"2023-11-28T13:00:19.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338768+0000","last_change":"2023-11-28T13:00:17.238867+0000","last_active":"2023-11-28T13:00:18.338768+0000","last_peered":"2023-11-28T13:00:18.338768+0000","last_clean":"2023-11-28T13:00:18.338768+0000","last_became_active":"2023-11-28T13:00:17.238545+0000","last_became_peered":"2023-11-28T13:00:17.238545+0000","last_unstale":"2023-11-28T13:00:18.338768+0000","last_undegraded":"2023-11-28T13:00:18.338768+0000","last_fullsized":"2023-11-28T13:00:18.338768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215335+0000","last_change":"2023-11-28T13:00:17.645527+0000","last_active":"2023-11-28T13:00:19.215335+0000","last_peered":"2023-11-28T13:00:19.215335+0000","last_clean":"2023-11-28T13:00:19.215335+0000","last_became_active":"2023-11-28T13:00:17.644057+0000","last_became_peered":"2023-11-28T13:00:17.644057+0000","last_unstale":"2023-11-28T13:00:19.215335+0000","last_undegraded":"2023-11-28T13:00:19.215335+0000","last_fullsized":"2023-11-28T13:00:19.215335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:55.013485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331175+0000","last_change":"2023-11-28T13:00:17.242375+0000","last_active":"2023-11-28T13:00:18.331175+0000","last_peered":"2023-11-28T13:00:18.331175+0000","last_clean":"2023-11-28T13:00:18.331175+0000","last_became_active":"2023-11-28T13:00:17.242010+0000","last_became_peered":"2023-11-28T13:00:17.242010+0000","last_unstale":"2023-11-28T13:00:18.331175+0000","last_undegraded":"2023-11-28T13:00:18.331175+0000","last_fullsized":"2023-11-28T13:00:18.331175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.522601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635882+0000","last_change":"2023-11-28T13:00:17.647427+0000","last_active":"2023-11-28T13:00:18.635882+0000","last_peered":"2023-11-28T13:00:18.635882+0000","last_clean":"2023-11-28T13:00:18.635882+0000","last_became_active":"2023-11-28T13:00:17.646853+0000","last_became_peered":"2023-11-28T13:00:17.646853+0000","last_unstale":"2023-11-28T13:00:18.635882+0000","last_undegraded":"2023-11-28T13:00:18.635882+0000","last_fullsized":"2023-11-28T13:00:18.635882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497407+0000","last_change":"2023-11-28T13:00:17.243823+0000","last_active":"2023-11-28T13:00:18.497407+0000","last_peered":"2023-11-28T13:00:18.497407+0000","last_clean":"2023-11-28T13:00:18.497407+0000","last_became_active":"2023-11-28T13:00:17.241238+0000","last_became_peered":"2023-11-28T13:00:17.241238+0000","last_unstale":"2023-11-28T13:00:18.497407+0000","last_undegraded":"2023-11-28T13:00:18.497407+0000","last_fullsized":"2023-11-28T13:00:18.497407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497280+0000","last_change":"2023-11-28T13:00:17.649986+0000","last_active":"2023-11-28T13:00:18.497280+0000","last_peered":"2023-11-28T13:00:18.497280+0000","last_clean":"2023-11-28T13:00:18.497280+0000","last_became_active":"2023-11-28T13:00:17.649402+0000","last_became_peered":"2023-11-28T13:00:17.649402+0000","last_unstale":"2023-11-28T13:00:18.497280+0000","last_undegraded":"2023-11-28T13:00:18.497280+0000","last_fullsized":"2023-11-28T13:00:18.497280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:15.471790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331132+0000","last_change":"2023-11-28T13:00:17.247047+0000","last_active":"2023-11-28T13:00:18.331132+0000","last_peered":"2023-11-28T13:00:18.331132+0000","last_clean":"2023-11-28T13:00:18.331132+0000","last_became_active":"2023-11-28T13:00:17.246742+0000","last_became_peered":"2023-11-28T13:00:17.246742+0000","last_unstale":"2023-11-28T13:00:18.331132+0000","last_undegraded":"2023-11-28T13:00:18.331132+0000","last_fullsized":"2023-11-28T13:00:18.331132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:50.559956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215527+0000","last_change":"2023-11-28T13:00:17.641605+0000","last_active":"2023-11-28T13:00:19.215527+0000","last_peered":"2023-11-28T13:00:19.215527+0000","last_clean":"2023-11-28T13:00:19.215527+0000","last_became_active":"2023-11-28T13:00:17.641350+0000","last_became_peered":"2023-11-28T13:00:17.641350+0000","last_unstale":"2023-11-28T13:00:19.215527+0000","last_undegraded":"2023-11-28T13:00:19.215527+0000","last_fullsized":"2023-11-28T13:00:19.215527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.630517+0000","last_change":"2023-11-28T13:00:17.240432+0000","last_active":"2023-11-28T13:00:17.630517+0000","last_peered":"2023-11-28T13:00:17.630517+0000","last_clean":"2023-11-28T13:00:17.630517+0000","last_became_active":"2023-11-28T13:00:17.239192+0000","last_became_peered":"2023-11-28T13:00:17.239192+0000","last_unstale":"2023-11-28T13:00:17.630517+0000","last_undegraded":"2023-11-28T13:00:17.630517+0000","last_fullsized":"2023-11-28T13:00:17.630517+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338702+0000","last_change":"2023-11-28T13:00:17.648774+0000","last_active":"2023-11-28T13:00:18.338702+0000","last_peered":"2023-11-28T13:00:18.338702+0000","last_clean":"2023-11-28T13:00:18.338702+0000","last_became_active":"2023-11-28T13:00:17.647403+0000","last_became_peered":"2023-11-28T13:00:17.647403+0000","last_unstale":"2023-11-28T13:00:18.338702+0000","last_undegraded":"2023-11-28T13:00:18.338702+0000","last_fullsized":"2023-11-28T13:00:18.338702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:17.592744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331091+0000","last_change":"2023-11-28T13:00:17.245977+0000","last_active":"2023-11-28T13:00:18.331091+0000","last_peered":"2023-11-28T13:00:18.331091+0000","last_clean":"2023-11-28T13:00:18.331091+0000","last_became_active":"2023-11-28T13:00:17.245651+0000","last_became_peered":"2023-11-28T13:00:17.245651+0000","last_unstale":"2023-11-28T13:00:18.331091+0000","last_undegraded":"2023-11-28T13:00:18.331091+0000","last_fullsized":"2023-11-28T13:00:18.331091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:30.760573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649006+0000","last_change":"2023-11-28T13:00:17.644029+0000","last_active":"2023-11-28T13:00:17.649006+0000","last_peered":"2023-11-28T13:00:17.649006+0000","last_clean":"2023-11-28T13:00:17.649006+0000","last_became_active":"2023-11-28T13:00:17.643789+0000","last_became_peered":"2023-11-28T13:00:17.643789+0000","last_unstale":"2023-11-28T13:00:17.649006+0000","last_undegraded":"2023-11-28T13:00:17.649006+0000","last_fullsized":"2023-11-28T13:00:17.649006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330786+0000","last_change":"2023-11-28T13:00:17.240716+0000","last_active":"2023-11-28T13:00:18.330786+0000","last_peered":"2023-11-28T13:00:18.330786+0000","last_clean":"2023-11-28T13:00:18.330786+0000","last_became_active":"2023-11-28T13:00:17.239928+0000","last_became_peered":"2023-11-28T13:00:17.239928+0000","last_unstale":"2023-11-28T13:00:18.330786+0000","last_undegraded":"2023-11-28T13:00:18.330786+0000","last_fullsized":"2023-11-28T13:00:18.330786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:32.086174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215680+0000","last_change":"2023-11-28T13:00:17.643923+0000","last_active":"2023-11-28T13:00:19.215680+0000","last_peered":"2023-11-28T13:00:19.215680+0000","last_clean":"2023-11-28T13:00:19.215680+0000","last_became_active":"2023-11-28T13:00:17.643579+0000","last_became_peered":"2023-11-28T13:00:17.643579+0000","last_unstale":"2023-11-28T13:00:19.215680+0000","last_undegraded":"2023-11-28T13:00:19.215680+0000","last_fullsized":"2023-11-28T13:00:19.215680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331252+0000","last_change":"2023-11-28T13:00:17.247259+0000","last_active":"2023-11-28T13:00:18.331252+0000","last_peered":"2023-11-28T13:00:18.331252+0000","last_clean":"2023-11-28T13:00:18.331252+0000","last_became_active":"2023-11-28T13:00:17.246943+0000","last_became_peered":"2023-11-28T13:00:17.246943+0000","last_unstale":"2023-11-28T13:00:18.331252+0000","last_undegraded":"2023-11-28T13:00:18.331252+0000","last_fullsized":"2023-11-28T13:00:18.331252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:02.643388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338621+0000","last_change":"2023-11-28T13:00:17.644322+0000","last_active":"2023-11-28T13:00:18.338621+0000","last_peered":"2023-11-28T13:00:18.338621+0000","last_clean":"2023-11-28T13:00:18.338621+0000","last_became_active":"2023-11-28T13:00:17.644054+0000","last_became_peered":"2023-11-28T13:00:17.644054+0000","last_unstale":"2023-11-28T13:00:18.338621+0000","last_undegraded":"2023-11-28T13:00:18.338621+0000","last_fullsized":"2023-11-28T13:00:18.338621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:46.098419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331230+0000","last_change":"2023-11-28T13:00:17.247274+0000","last_active":"2023-11-28T13:00:18.331230+0000","last_peered":"2023-11-28T13:00:18.331230+0000","last_clean":"2023-11-28T13:00:18.331230+0000","last_became_active":"2023-11-28T13:00:17.246851+0000","last_became_peered":"2023-11-28T13:00:17.246851+0000","last_unstale":"2023-11-28T13:00:18.331230+0000","last_undegraded":"2023-11-28T13:00:18.331230+0000","last_fullsized":"2023-11-28T13:00:18.331230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:17.621082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497137+0000","last_change":"2023-11-28T13:00:17.640395+0000","last_active":"2023-11-28T13:00:18.497137+0000","last_peered":"2023-11-28T13:00:18.497137+0000","last_clean":"2023-11-28T13:00:18.497137+0000","last_became_active":"2023-11-28T13:00:17.640195+0000","last_became_peered":"2023-11-28T13:00:17.640195+0000","last_unstale":"2023-11-28T13:00:18.497137+0000","last_undegraded":"2023-11-28T13:00:18.497137+0000","last_fullsized":"2023-11-28T13:00:18.497137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:14.878366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331208+0000","last_change":"2023-11-28T13:00:17.240698+0000","last_active":"2023-11-28T13:00:18.331208+0000","last_peered":"2023-11-28T13:00:18.331208+0000","last_clean":"2023-11-28T13:00:18.331208+0000","last_became_active":"2023-11-28T13:00:17.240387+0000","last_became_peered":"2023-11-28T13:00:17.240387+0000","last_unstale":"2023-11-28T13:00:18.331208+0000","last_undegraded":"2023-11-28T13:00:18.331208+0000","last_fullsized":"2023-11-28T13:00:18.331208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:05.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330832+0000","last_change":"2023-11-28T13:00:17.647318+0000","last_active":"2023-11-28T13:00:18.330832+0000","last_peered":"2023-11-28T13:00:18.330832+0000","last_clean":"2023-11-28T13:00:18.330832+0000","last_became_active":"2023-11-28T13:00:17.647043+0000","last_became_peered":"2023-11-28T13:00:17.647043+0000","last_unstale":"2023-11-28T13:00:18.330832+0000","last_undegraded":"2023-11-28T13:00:18.330832+0000","last_fullsized":"2023-11-28T13:00:18.330832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:38.321116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338680+0000","last_change":"2023-11-28T13:00:17.235843+0000","last_active":"2023-11-28T13:00:18.338680+0000","last_peered":"2023-11-28T13:00:18.338680+0000","last_clean":"2023-11-28T13:00:18.338680+0000","last_became_active":"2023-11-28T13:00:17.235495+0000","last_became_peered":"2023-11-28T13:00:17.235495+0000","last_unstale":"2023-11-28T13:00:18.338680+0000","last_undegraded":"2023-11-28T13:00:18.338680+0000","last_fullsized":"2023-11-28T13:00:18.338680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648938+0000","last_change":"2023-11-28T13:00:17.640727+0000","last_active":"2023-11-28T13:00:17.648938+0000","last_peered":"2023-11-28T13:00:17.648938+0000","last_clean":"2023-11-28T13:00:17.648938+0000","last_became_active":"2023-11-28T13:00:17.640438+0000","last_became_peered":"2023-11-28T13:00:17.640438+0000","last_unstale":"2023-11-28T13:00:17.648938+0000","last_undegraded":"2023-11-28T13:00:17.648938+0000","last_fullsized":"2023-11-28T13:00:17.648938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215456+0000","last_change":"2023-11-28T13:00:17.240709+0000","last_active":"2023-11-28T13:00:19.215456+0000","last_peered":"2023-11-28T13:00:19.215456+0000","last_clean":"2023-11-28T13:00:19.215456+0000","last_became_active":"2023-11-28T13:00:17.239893+0000","last_became_peered":"2023-11-28T13:00:17.239893+0000","last_unstale":"2023-11-28T13:00:19.215456+0000","last_undegraded":"2023-11-28T13:00:19.215456+0000","last_fullsized":"2023-11-28T13:00:19.215456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497386+0000","last_change":"2023-11-28T13:00:17.653235+0000","last_active":"2023-11-28T13:00:18.497386+0000","last_peered":"2023-11-28T13:00:18.497386+0000","last_clean":"2023-11-28T13:00:18.497386+0000","last_became_active":"2023-11-28T13:00:17.650249+0000","last_became_peered":"2023-11-28T13:00:17.650249+0000","last_unstale":"2023-11-28T13:00:18.497386+0000","last_undegraded":"2023-11-28T13:00:18.497386+0000","last_fullsized":"2023-11-28T13:00:18.497386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330820+0000","last_change":"2023-11-28T13:00:17.245224+0000","last_active":"2023-11-28T13:00:18.330820+0000","last_peered":"2023-11-28T13:00:18.330820+0000","last_clean":"2023-11-28T13:00:18.330820+0000","last_became_active":"2023-11-28T13:00:17.244913+0000","last_became_peered":"2023-11-28T13:00:17.244913+0000","last_unstale":"2023-11-28T13:00:18.330820+0000","last_undegraded":"2023-11-28T13:00:18.330820+0000","last_fullsized":"2023-11-28T13:00:18.330820+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497463+0000","last_change":"2023-11-28T13:00:17.643015+0000","last_active":"2023-11-28T13:00:18.497463+0000","last_peered":"2023-11-28T13:00:18.497463+0000","last_clean":"2023-11-28T13:00:18.497463+0000","last_became_active":"2023-11-28T13:00:17.642470+0000","last_became_peered":"2023-11-28T13:00:17.642470+0000","last_unstale":"2023-11-28T13:00:18.497463+0000","last_undegraded":"2023-11-28T13:00:18.497463+0000","last_fullsized":"2023-11-28T13:00:18.497463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:41.679867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497180+0000","last_change":"2023-11-28T13:00:17.244207+0000","last_active":"2023-11-28T13:00:18.497180+0000","last_peered":"2023-11-28T13:00:18.497180+0000","last_clean":"2023-11-28T13:00:18.497180+0000","last_became_active":"2023-11-28T13:00:17.243861+0000","last_became_peered":"2023-11-28T13:00:17.243861+0000","last_unstale":"2023-11-28T13:00:18.497180+0000","last_undegraded":"2023-11-28T13:00:18.497180+0000","last_fullsized":"2023-11-28T13:00:18.497180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497504+0000","last_change":"2023-11-28T13:00:17.651269+0000","last_active":"2023-11-28T13:00:18.497504+0000","last_peered":"2023-11-28T13:00:18.497504+0000","last_clean":"2023-11-28T13:00:18.497504+0000","last_became_active":"2023-11-28T13:00:17.650883+0000","last_became_peered":"2023-11-28T13:00:17.650883+0000","last_unstale":"2023-11-28T13:00:18.497504+0000","last_undegraded":"2023-11-28T13:00:18.497504+0000","last_fullsized":"2023-11-28T13:00:18.497504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215191+0000","last_change":"2023-11-28T13:00:17.238617+0000","last_active":"2023-11-28T13:00:19.215191+0000","last_peered":"2023-11-28T13:00:19.215191+0000","last_clean":"2023-11-28T13:00:19.215191+0000","last_became_active":"2023-11-28T13:00:17.238344+0000","last_became_peered":"2023-11-28T13:00:17.238344+0000","last_unstale":"2023-11-28T13:00:19.215191+0000","last_undegraded":"2023-11-28T13:00:19.215191+0000","last_fullsized":"2023-11-28T13:00:19.215191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:46.225565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636017+0000","last_change":"2023-11-28T13:00:17.649564+0000","last_active":"2023-11-28T13:00:18.636017+0000","last_peered":"2023-11-28T13:00:18.636017+0000","last_clean":"2023-11-28T13:00:18.636017+0000","last_became_active":"2023-11-28T13:00:17.649312+0000","last_became_peered":"2023-11-28T13:00:17.649312+0000","last_unstale":"2023-11-28T13:00:18.636017+0000","last_undegraded":"2023-11-28T13:00:18.636017+0000","last_fullsized":"2023-11-28T13:00:18.636017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634764+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:00:17.634764+0000","last_peered":"2023-11-28T13:00:17.634764+0000","last_clean":"2023-11-28T13:00:17.634764+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:00:17.634764+0000","last_undegraded":"2023-11-28T13:00:17.634764+0000","last_fullsized":"2023-11-28T13:00:17.634764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338730+0000","last_change":"2023-11-28T13:00:17.240503+0000","last_active":"2023-11-28T13:00:18.338730+0000","last_peered":"2023-11-28T13:00:18.338730+0000","last_clean":"2023-11-28T13:00:18.338730+0000","last_became_active":"2023-11-28T13:00:17.240256+0000","last_became_peered":"2023-11-28T13:00:17.240256+0000","last_unstale":"2023-11-28T13:00:18.338730+0000","last_undegraded":"2023-11-28T13:00:18.338730+0000","last_fullsized":"2023-11-28T13:00:18.338730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635946+0000","last_change":"2023-11-28T13:00:17.645964+0000","last_active":"2023-11-28T13:00:18.635946+0000","last_peered":"2023-11-28T13:00:18.635946+0000","last_clean":"2023-11-28T13:00:18.635946+0000","last_became_active":"2023-11-28T13:00:17.645384+0000","last_became_peered":"2023-11-28T13:00:17.645384+0000","last_unstale":"2023-11-28T13:00:18.635946+0000","last_undegraded":"2023-11-28T13:00:18.635946+0000","last_fullsized":"2023-11-28T13:00:18.635946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634535+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:00:17.634535+0000","last_peered":"2023-11-28T13:00:17.634535+0000","last_clean":"2023-11-28T13:00:17.634535+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:00:17.634535+0000","last_undegraded":"2023-11-28T13:00:17.634535+0000","last_fullsized":"2023-11-28T13:00:17.634535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497313+0000","last_change":"2023-11-28T13:00:17.245773+0000","last_active":"2023-11-28T13:00:18.497313+0000","last_peered":"2023-11-28T13:00:18.497313+0000","last_clean":"2023-11-28T13:00:18.497313+0000","last_became_active":"2023-11-28T13:00:17.245409+0000","last_became_peered":"2023-11-28T13:00:17.245409+0000","last_unstale":"2023-11-28T13:00:18.497313+0000","last_undegraded":"2023-11-28T13:00:18.497313+0000","last_fullsized":"2023-11-28T13:00:18.497313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:00.519142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636216+0000","last_change":"2023-11-28T13:00:17.641395+0000","last_active":"2023-11-28T13:00:18.636216+0000","last_peered":"2023-11-28T13:00:18.636216+0000","last_clean":"2023-11-28T13:00:18.636216+0000","last_became_active":"2023-11-28T13:00:17.641149+0000","last_became_peered":"2023-11-28T13:00:17.641149+0000","last_unstale":"2023-11-28T13:00:18.636216+0000","last_undegraded":"2023-11-28T13:00:18.636216+0000","last_fullsized":"2023-11-28T13:00:18.636216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.628022+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:00:17.628022+0000","last_peered":"2023-11-28T13:00:17.628022+0000","last_clean":"2023-11-28T13:00:17.628022+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:00:17.628022+0000","last_undegraded":"2023-11-28T13:00:17.628022+0000","last_fullsized":"2023-11-28T13:00:17.628022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635610+0000","last_change":"2023-11-28T13:00:17.237802+0000","last_active":"2023-11-28T13:00:18.635610+0000","last_peered":"2023-11-28T13:00:18.635610+0000","last_clean":"2023-11-28T13:00:18.635610+0000","last_became_active":"2023-11-28T13:00:17.237487+0000","last_became_peered":"2023-11-28T13:00:17.237487+0000","last_unstale":"2023-11-28T13:00:18.635610+0000","last_undegraded":"2023-11-28T13:00:18.635610+0000","last_fullsized":"2023-11-28T13:00:18.635610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635877+0000","last_change":"2023-11-28T13:00:17.644511+0000","last_active":"2023-11-28T13:00:18.635877+0000","last_peered":"2023-11-28T13:00:18.635877+0000","last_clean":"2023-11-28T13:00:18.635877+0000","last_became_active":"2023-11-28T13:00:17.644095+0000","last_became_peered":"2023-11-28T13:00:17.644095+0000","last_unstale":"2023-11-28T13:00:18.635877+0000","last_undegraded":"2023-11-28T13:00:18.635877+0000","last_fullsized":"2023-11-28T13:00:18.635877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.636163+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:00:17.636163+0000","last_peered":"2023-11-28T13:00:17.636163+0000","last_clean":"2023-11-28T13:00:17.636163+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:00:17.636163+0000","last_undegraded":"2023-11-28T13:00:17.636163+0000","last_fullsized":"2023-11-28T13:00:17.636163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633158+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:00:17.633158+0000","last_peered":"2023-11-28T13:00:17.633158+0000","last_clean":"2023-11-28T13:00:17.633158+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:00:17.633158+0000","last_undegraded":"2023-11-28T13:00:17.633158+0000","last_fullsized":"2023-11-28T13:00:17.633158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215311+0000","last_change":"2023-11-28T13:00:17.239373+0000","last_active":"2023-11-28T13:00:19.215311+0000","last_peered":"2023-11-28T13:00:19.215311+0000","last_clean":"2023-11-28T13:00:19.215311+0000","last_became_active":"2023-11-28T13:00:17.238864+0000","last_became_peered":"2023-11-28T13:00:17.238864+0000","last_unstale":"2023-11-28T13:00:19.215311+0000","last_undegraded":"2023-11-28T13:00:19.215311+0000","last_fullsized":"2023-11-28T13:00:19.215311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:17.763933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330851+0000","last_change":"2023-11-28T13:00:17.649771+0000","last_active":"2023-11-28T13:00:18.330851+0000","last_peered":"2023-11-28T13:00:18.330851+0000","last_clean":"2023-11-28T13:00:18.330851+0000","last_became_active":"2023-11-28T13:00:17.649475+0000","last_became_peered":"2023-11-28T13:00:17.649475+0000","last_unstale":"2023-11-28T13:00:18.330851+0000","last_undegraded":"2023-11-28T13:00:18.330851+0000","last_fullsized":"2023-11-28T13:00:18.330851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633524+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:00:17.633524+0000","last_peered":"2023-11-28T13:00:17.633524+0000","last_clean":"2023-11-28T13:00:17.633524+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:00:17.633524+0000","last_undegraded":"2023-11-28T13:00:17.633524+0000","last_fullsized":"2023-11-28T13:00:17.633524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.630289+0000","last_change":"2023-11-28T13:00:17.251262+0000","last_active":"2023-11-28T13:00:17.630289+0000","last_peered":"2023-11-28T13:00:17.630289+0000","last_clean":"2023-11-28T13:00:17.630289+0000","last_became_active":"2023-11-28T13:00:17.250948+0000","last_became_peered":"2023-11-28T13:00:17.250948+0000","last_unstale":"2023-11-28T13:00:17.630289+0000","last_undegraded":"2023-11-28T13:00:17.630289+0000","last_fullsized":"2023-11-28T13:00:17.630289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:37.311866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648968+0000","last_change":"2023-11-28T13:00:17.640685+0000","last_active":"2023-11-28T13:00:17.648968+0000","last_peered":"2023-11-28T13:00:17.648968+0000","last_clean":"2023-11-28T13:00:17.648968+0000","last_became_active":"2023-11-28T13:00:17.640420+0000","last_became_peered":"2023-11-28T13:00:17.640420+0000","last_unstale":"2023-11-28T13:00:17.648968+0000","last_undegraded":"2023-11-28T13:00:17.648968+0000","last_fullsized":"2023-11-28T13:00:17.648968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.632566+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:00:17.632566+0000","last_peered":"2023-11-28T13:00:17.632566+0000","last_clean":"2023-11-28T13:00:17.632566+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:00:17.632566+0000","last_undegraded":"2023-11-28T13:00:17.632566+0000","last_fullsized":"2023-11-28T13:00:17.632566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636224+0000","last_change":"2023-11-28T13:00:17.237999+0000","last_active":"2023-11-28T13:00:18.636224+0000","last_peered":"2023-11-28T13:00:18.636224+0000","last_clean":"2023-11-28T13:00:18.636224+0000","last_became_active":"2023-11-28T13:00:17.237625+0000","last_became_peered":"2023-11-28T13:00:17.237625+0000","last_unstale":"2023-11-28T13:00:18.636224+0000","last_undegraded":"2023-11-28T13:00:18.636224+0000","last_fullsized":"2023-11-28T13:00:18.636224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650614+0000","last_change":"2023-11-28T13:00:17.644943+0000","last_active":"2023-11-28T13:00:17.650614+0000","last_peered":"2023-11-28T13:00:17.650614+0000","last_clean":"2023-11-28T13:00:17.650614+0000","last_became_active":"2023-11-28T13:00:17.644679+0000","last_became_peered":"2023-11-28T13:00:17.644679+0000","last_unstale":"2023-11-28T13:00:17.650614+0000","last_undegraded":"2023-11-28T13:00:17.650614+0000","last_fullsized":"2023-11-28T13:00:17.650614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215660+0000","last_change":"2023-11-28T13:00:17.239284+0000","last_active":"2023-11-28T13:00:19.215660+0000","last_peered":"2023-11-28T13:00:19.215660+0000","last_clean":"2023-11-28T13:00:19.215660+0000","last_became_active":"2023-11-28T13:00:17.238997+0000","last_became_peered":"2023-11-28T13:00:17.238997+0000","last_unstale":"2023-11-28T13:00:19.215660+0000","last_undegraded":"2023-11-28T13:00:19.215660+0000","last_fullsized":"2023-11-28T13:00:19.215660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:55.423382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633198+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:00:17.633198+0000","last_peered":"2023-11-28T13:00:17.633198+0000","last_clean":"2023-11-28T13:00:17.633198+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:00:17.633198+0000","last_undegraded":"2023-11-28T13:00:17.633198+0000","last_fullsized":"2023-11-28T13:00:17.633198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215108+0000","last_change":"2023-11-28T13:00:17.646149+0000","last_active":"2023-11-28T13:00:19.215108+0000","last_peered":"2023-11-28T13:00:19.215108+0000","last_clean":"2023-11-28T13:00:19.215108+0000","last_became_active":"2023-11-28T13:00:17.645893+0000","last_became_peered":"2023-11-28T13:00:17.645893+0000","last_unstale":"2023-11-28T13:00:19.215108+0000","last_undegraded":"2023-11-28T13:00:19.215108+0000","last_fullsized":"2023-11-28T13:00:19.215108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:24.466207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633216+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:00:17.633216+0000","last_peered":"2023-11-28T13:00:17.633216+0000","last_clean":"2023-11-28T13:00:17.633216+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:00:17.633216+0000","last_undegraded":"2023-11-28T13:00:17.633216+0000","last_fullsized":"2023-11-28T13:00:17.633216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.717720+0000","last_change":"2023-11-28T13:00:17.240379+0000","last_active":"2023-11-28T13:00:18.717720+0000","last_peered":"2023-11-28T13:00:18.717720+0000","last_clean":"2023-11-28T13:00:18.717720+0000","last_became_active":"2023-11-28T13:00:17.239505+0000","last_became_peered":"2023-11-28T13:00:17.239505+0000","last_unstale":"2023-11-28T13:00:18.717720+0000","last_undegraded":"2023-11-28T13:00:18.717720+0000","last_fullsized":"2023-11-28T13:00:18.717720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497157+0000","last_change":"2023-11-28T13:00:17.646003+0000","last_active":"2023-11-28T13:00:18.497157+0000","last_peered":"2023-11-28T13:00:18.497157+0000","last_clean":"2023-11-28T13:00:18.497157+0000","last_became_active":"2023-11-28T13:00:17.645727+0000","last_became_peered":"2023-11-28T13:00:17.645727+0000","last_unstale":"2023-11-28T13:00:18.497157+0000","last_undegraded":"2023-11-28T13:00:18.497157+0000","last_fullsized":"2023-11-28T13:00:18.497157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:30.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656358+0000","last_change":"2023-11-28T13:00:17.251398+0000","last_active":"2023-11-28T13:00:17.656358+0000","last_peered":"2023-11-28T13:00:17.656358+0000","last_clean":"2023-11-28T13:00:17.656358+0000","last_became_active":"2023-11-28T13:00:17.251054+0000","last_became_peered":"2023-11-28T13:00:17.251054+0000","last_unstale":"2023-11-28T13:00:17.656358+0000","last_undegraded":"2023-11-28T13:00:17.656358+0000","last_fullsized":"2023-11-28T13:00:17.656358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217848+0000","last_change":"2023-11-28T13:00:17.641355+0000","last_active":"2023-11-28T13:00:19.217848+0000","last_peered":"2023-11-28T13:00:19.217848+0000","last_clean":"2023-11-28T13:00:19.217848+0000","last_became_active":"2023-11-28T13:00:17.641093+0000","last_became_peered":"2023-11-28T13:00:17.641093+0000","last_unstale":"2023-11-28T13:00:19.217848+0000","last_undegraded":"2023-11-28T13:00:19.217848+0000","last_fullsized":"2023-11-28T13:00:19.217848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:41.816686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338665+0000","last_change":"2023-11-28T13:00:17.246896+0000","last_active":"2023-11-28T13:00:18.338665+0000","last_peered":"2023-11-28T13:00:18.338665+0000","last_clean":"2023-11-28T13:00:18.338665+0000","last_became_active":"2023-11-28T13:00:17.246604+0000","last_became_peered":"2023-11-28T13:00:17.246604+0000","last_unstale":"2023-11-28T13:00:18.338665+0000","last_undegraded":"2023-11-28T13:00:18.338665+0000","last_fullsized":"2023-11-28T13:00:18.338665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497332+0000","last_change":"2023-11-28T13:00:17.648973+0000","last_active":"2023-11-28T13:00:18.497332+0000","last_peered":"2023-11-28T13:00:18.497332+0000","last_clean":"2023-11-28T13:00:18.497332+0000","last_became_active":"2023-11-28T13:00:17.648323+0000","last_became_peered":"2023-11-28T13:00:17.648323+0000","last_unstale":"2023-11-28T13:00:18.497332+0000","last_undegraded":"2023-11-28T13:00:18.497332+0000","last_fullsized":"2023-11-28T13:00:18.497332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338744+0000","last_change":"2023-11-28T13:00:17.247013+0000","last_active":"2023-11-28T13:00:18.338744+0000","last_peered":"2023-11-28T13:00:18.338744+0000","last_clean":"2023-11-28T13:00:18.338744+0000","last_became_active":"2023-11-28T13:00:17.246645+0000","last_became_peered":"2023-11-28T13:00:17.246645+0000","last_unstale":"2023-11-28T13:00:18.338744+0000","last_undegraded":"2023-11-28T13:00:18.338744+0000","last_fullsized":"2023-11-28T13:00:18.338744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497261+0000","last_change":"2023-11-28T13:00:17.649078+0000","last_active":"2023-11-28T13:00:18.497261+0000","last_peered":"2023-11-28T13:00:18.497261+0000","last_clean":"2023-11-28T13:00:18.497261+0000","last_became_active":"2023-11-28T13:00:17.648432+0000","last_became_peered":"2023-11-28T13:00:17.648432+0000","last_unstale":"2023-11-28T13:00:18.497261+0000","last_undegraded":"2023-11-28T13:00:18.497261+0000","last_fullsized":"2023-11-28T13:00:18.497261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215495+0000","last_change":"2023-11-28T13:00:17.234074+0000","last_active":"2023-11-28T13:00:19.215495+0000","last_peered":"2023-11-28T13:00:19.215495+0000","last_clean":"2023-11-28T13:00:19.215495+0000","last_became_active":"2023-11-28T13:00:17.233732+0000","last_became_peered":"2023-11-28T13:00:17.233732+0000","last_unstale":"2023-11-28T13:00:19.215495+0000","last_undegraded":"2023-11-28T13:00:19.215495+0000","last_fullsized":"2023-11-28T13:00:19.215495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:36.950864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215231+0000","last_change":"2023-11-28T13:00:17.641105+0000","last_active":"2023-11-28T13:00:19.215231+0000","last_peered":"2023-11-28T13:00:19.215231+0000","last_clean":"2023-11-28T13:00:19.215231+0000","last_became_active":"2023-11-28T13:00:17.640854+0000","last_became_peered":"2023-11-28T13:00:17.640854+0000","last_unstale":"2023-11-28T13:00:19.215231+0000","last_undegraded":"2023-11-28T13:00:19.215231+0000","last_fullsized":"2023-11-28T13:00:19.215231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652233+0000","last_change":"2023-11-28T13:00:17.242508+0000","last_active":"2023-11-28T13:00:17.652233+0000","last_peered":"2023-11-28T13:00:17.652233+0000","last_clean":"2023-11-28T13:00:17.652233+0000","last_became_active":"2023-11-28T13:00:17.242230+0000","last_became_peered":"2023-11-28T13:00:17.242230+0000","last_unstale":"2023-11-28T13:00:17.652233+0000","last_undegraded":"2023-11-28T13:00:17.652233+0000","last_fullsized":"2023-11-28T13:00:17.652233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497079+0000","last_change":"2023-11-28T13:00:17.650868+0000","last_active":"2023-11-28T13:00:18.497079+0000","last_peered":"2023-11-28T13:00:18.497079+0000","last_clean":"2023-11-28T13:00:18.497079+0000","last_became_active":"2023-11-28T13:00:17.649842+0000","last_became_peered":"2023-11-28T13:00:17.649842+0000","last_unstale":"2023-11-28T13:00:18.497079+0000","last_undegraded":"2023-11-28T13:00:18.497079+0000","last_fullsized":"2023-11-28T13:00:18.497079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330807+0000","last_change":"2023-11-28T13:00:17.641330+0000","last_active":"2023-11-28T13:00:18.330807+0000","last_peered":"2023-11-28T13:00:18.330807+0000","last_clean":"2023-11-28T13:00:18.330807+0000","last_became_active":"2023-11-28T13:00:17.641029+0000","last_became_peered":"2023-11-28T13:00:17.641029+0000","last_unstale":"2023-11-28T13:00:18.330807+0000","last_undegraded":"2023-11-28T13:00:18.330807+0000","last_fullsized":"2023-11-28T13:00:18.330807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:28.607875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215202+0000","last_change":"2023-11-28T13:00:17.643816+0000","last_active":"2023-11-28T13:00:19.215202+0000","last_peered":"2023-11-28T13:00:19.215202+0000","last_clean":"2023-11-28T13:00:19.215202+0000","last_became_active":"2023-11-28T13:00:17.643260+0000","last_became_peered":"2023-11-28T13:00:17.643260+0000","last_unstale":"2023-11-28T13:00:19.215202+0000","last_undegraded":"2023-11-28T13:00:19.215202+0000","last_fullsized":"2023-11-28T13:00:19.215202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497218+0000","last_change":"2023-11-28T13:00:17.649431+0000","last_active":"2023-11-28T13:00:18.497218+0000","last_peered":"2023-11-28T13:00:18.497218+0000","last_clean":"2023-11-28T13:00:18.497218+0000","last_became_active":"2023-11-28T13:00:17.648763+0000","last_became_peered":"2023-11-28T13:00:17.648763+0000","last_unstale":"2023-11-28T13:00:18.497218+0000","last_undegraded":"2023-11-28T13:00:18.497218+0000","last_fullsized":"2023-11-28T13:00:18.497218+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:10.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215472+0000","last_change":"2023-11-28T13:00:17.649148+0000","last_active":"2023-11-28T13:00:19.215472+0000","last_peered":"2023-11-28T13:00:19.215472+0000","last_clean":"2023-11-28T13:00:19.215472+0000","last_became_active":"2023-11-28T13:00:17.648925+0000","last_became_peered":"2023-11-28T13:00:17.648925+0000","last_unstale":"2023-11-28T13:00:19.215472+0000","last_undegraded":"2023-11-28T13:00:19.215472+0000","last_fullsized":"2023-11-28T13:00:19.215472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636138+0000","last_change":"2023-11-28T13:00:17.648409+0000","last_active":"2023-11-28T13:00:18.636138+0000","last_peered":"2023-11-28T13:00:18.636138+0000","last_clean":"2023-11-28T13:00:18.636138+0000","last_became_active":"2023-11-28T13:00:17.648164+0000","last_became_peered":"2023-11-28T13:00:17.648164+0000","last_unstale":"2023-11-28T13:00:18.636138+0000","last_undegraded":"2023-11-28T13:00:18.636138+0000","last_fullsized":"2023-11-28T13:00:18.636138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":96947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.384}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3209999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1029999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7940,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363900,"statfs":{"total":96636764160,"available":96628633600,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8088,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363752,"statfs":{"total":96636764160,"available":96628482048,"internally_reserved":0,"allocated":278528,"data_stored":98461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9096,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362744,"statfs":{"total":96636764160,"available":96627449856,"internally_reserved":0,"allocated":851968,"data_stored":687022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9124,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362716,"statfs":{"total":96636764160,"available":96627421184,"internally_reserved":0,"allocated":880640,"data_stored":688989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8012,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363828,"statfs":{"total":96636764160,"available":96628559872,"internally_reserved":0,"allocated":266240,"data_stored":97493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8004,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363836,"statfs":{"total":96636764160,"available":96628568064,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8068,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363772,"statfs":{"total":96636764160,"available":96628502528,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:29.099 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:29.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.252+0000 7f4d5f32b700 1 Processor -- start 2023-11-28T13:00:29.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.252+0000 7f4d5f32b700 1 -- start start 2023-11-28T13:00:29.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5f32b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4d5806ba90 0x7f4d580af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5f32b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d580cbdf0 0x7f4d580ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5f32b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d580cc5f0 con 0x7f4d5815cef0 2023-11-28T13:00:29.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5f32b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5815d470 con 0x7f4d580cbdf0 2023-11-28T13:00:29.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5f32b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5815d750 con 0x7f4d5806ba90 2023-11-28T13:00:29.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5d8c8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d580cbdf0 0x7f4d580ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5d8c8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d580cbdf0 0x7f4d580ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38722/0 (socket says 172.21.15.153:38722) 2023-11-28T13:00:29.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.253+0000 7f4d5d8c8700 1 -- 172.21.15.153:0/3050265777 learned_addr learned my addr 172.21.15.153:0/3050265777 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5c8c6700 1 --2- 172.21.15.153:0/3050265777 >> v2:172.21.15.177:3300/0 conn(0x7f4d5806ba90 0x7f4d580af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5d8c8700 1 -- 172.21.15.153:0/3050265777 >> v2:172.21.15.177:3300/0 conn(0x7f4d5806ba90 msgr2=0x7f4d580af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5d8c8700 1 --2- 172.21.15.153:0/3050265777 >> v2:172.21.15.177:3300/0 conn(0x7f4d5806ba90 0x7f4d580af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5d8c8700 1 -- 172.21.15.153:0/3050265777 >> v1:172.21.15.177:6789/0 conn(0x7f4d5815cef0 legacy=0x7f4d5806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5d8c8700 1 -- 172.21.15.153:0/3050265777 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5815da30 con 0x7f4d580cbdf0 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5c8c6700 1 --2- 172.21.15.153:0/3050265777 >> v2:172.21.15.177:3300/0 conn(0x7f4d5806ba90 0x7f4d580af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5d8c8700 1 --2- 172.21.15.153:0/3050265777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d580ca650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d5806fbb0 tx=0x7f4d5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d00b7023fdc9438 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d4ffff700 1 -- 172.21.15.153:0/3050265777 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5000e070 con 0x7f4d580cbdf0 2023-11-28T13:00:29.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d4ffff700 1 -- 172.21.15.153:0/3050265777 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d50003a20 con 0x7f4d580cbdf0 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d4ffff700 1 -- 172.21.15.153:0/3050265777 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50003dd0 con 0x7f4d580cbdf0 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5f32b700 1 -- 172.21.15.153:0/3050265777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 msgr2=0x7f4d580ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.254+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/3050265777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d580ca650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d5806fbb0 tx=0x7f4d5000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.255+0000 7f4d5f32b700 1 -- 172.21.15.153:0/3050265777 shutdown_connections 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.255+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/3050265777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d580ca650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.255+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/3050265777 >> v2:172.21.15.177:3300/0 conn(0x7f4d5806ba90 0x7f4d580af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.255+0000 7f4d5f32b700 1 -- 172.21.15.153:0/3050265777 >> 172.21.15.153:0/3050265777 conn(0x7f4d580d0000 msgr2=0x7f4d580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.255+0000 7f4d5f32b700 1 -- 172.21.15.153:0/3050265777 shutdown_connections 2023-11-28T13:00:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.255+0000 7f4d5f32b700 1 -- 172.21.15.153:0/3050265777 wait complete. 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.256+0000 7f4d5f32b700 1 Processor -- start 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.256+0000 7f4d5f32b700 1 -- start start 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.256+0000 7f4d5f32b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d5806ba90 0x7f4d58152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.256+0000 7f4d5f32b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d58156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5f32b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815cef0 0x7f4d58157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5f32b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58158540 con 0x7f4d580cbdf0 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5f32b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58157930 con 0x7f4d5806ba90 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5f32b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58157c10 con 0x7f4d5815cef0 2023-11-28T13:00:29.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d58156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d58156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38726/0 (socket says 172.21.15.153:38726) 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 -- 172.21.15.153:0/4126027165 learned_addr learned my addr 172.21.15.153:0/4126027165 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5d0c7700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d5806ba90 0x7f4d58152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5d8c8700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815cef0 0x7f4d58157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 -- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815cef0 msgr2=0x7f4d58157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815cef0 0x7f4d58157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 -- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d5806ba90 msgr2=0x7f4d58152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d5806ba90 0x7f4d58152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 -- 172.21.15.153:0/4126027165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50009b30 con 0x7f4d580cbdf0 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5c8c6700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d58156de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c750 tx=0x7f4d5400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:29.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d4dffb700 1 -- 172.21.15.153:0/4126027165 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54011800 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d4dffb700 1 -- 172.21.15.153:0/4126027165 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54011960 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d4dffb700 1 -- 172.21.15.153:0/4126027165 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54011cd0 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5815dff0 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.257+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5815dd10 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.258+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d580cab90 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.259+0000 7f4d4dffb700 1 -- 172.21.15.153:0/4126027165 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d54020070 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.259+0000 7f4d4dffb700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d3c0682f0 0x7f4d3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.259+0000 7f4d4dffb700 1 -- 172.21.15.153:0/4126027165 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d54085970 con 0x7f4d580cbdf0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.259+0000 7f4d5d0c7700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d3c0682f0 0x7f4d3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.259+0000 7f4d5d0c7700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d3c0682f0 0x7f4d3c06a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009960 tx=0x7f4d480090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:29.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.263+0000 7f4d4dffb700 1 -- 172.21.15.153:0/4126027165 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d540521d0 con 0x7f4d580cbdf0 2023-11-28T13:00:29.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.441+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d580d0e60 con 0x7f4d3c0682f0 2023-11-28T13:00:29.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.449+0000 7f4d4dffb700 1 -- 172.21.15.153:0/4126027165 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286618 (secure 0 0 0) 0x7f4d580d0e60 con 0x7f4d3c0682f0 2023-11-28T13:00:29.454 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:29.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d3c0682f0 msgr2=0x7f4d3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d3c0682f0 0x7f4d3c06a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009960 tx=0x7f4d480090e0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 msgr2=0x7f4d58156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d58156de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c750 tx=0x7f4d5400ca60 comp rx=0 tx=0).stop 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 shutdown_connections 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815cef0 0x7f4d58157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d5806ba90 0x7f4d58152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d3c0682f0 0x7f4d3c06a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 --2- 172.21.15.153:0/4126027165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d580cbdf0 0x7f4d58156de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 >> 172.21.15.153:0/4126027165 conn(0x7f4d580d0000 msgr2=0x7f4d580afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 shutdown_connections 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.452+0000 7f4d5f32b700 1 -- 172.21.15.153:0/4126027165 wait complete. 2023-11-28T13:00:29.459 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:00:29.466 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-28T13:00:28.144657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66404,"kb_used_data":3236,"kb_used_omap":0,"kb_used_meta":63104,"kb_avail":754908316,"statfs":{"total":773094113280,"available":773026115584,"internally_reserved":0,"allocated":3313664,"data_stored":1959483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64618496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009156"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217825+0000","last_change":"2023-11-28T13:00:17.635428+0000","last_active":"2023-11-28T13:00:19.217825+0000","last_peered":"2023-11-28T13:00:19.217825+0000","last_clean":"2023-11-28T13:00:19.217825+0000","last_became_active":"2023-11-28T13:00:17.635148+0000","last_became_peered":"2023-11-28T13:00:17.635148+0000","last_unstale":"2023-11-28T13:00:19.217825+0000","last_undegraded":"2023-11-28T13:00:19.217825+0000","last_fullsized":"2023-11-28T13:00:19.217825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.262249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338757+0000","last_change":"2023-11-28T13:00:17.645017+0000","last_active":"2023-11-28T13:00:18.338757+0000","last_peered":"2023-11-28T13:00:18.338757+0000","last_clean":"2023-11-28T13:00:18.338757+0000","last_became_active":"2023-11-28T13:00:17.644524+0000","last_became_peered":"2023-11-28T13:00:17.644524+0000","last_unstale":"2023-11-28T13:00:18.338757+0000","last_undegraded":"2023-11-28T13:00:18.338757+0000","last_fullsized":"2023-11-28T13:00:18.338757+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497428+0000","last_change":"2023-11-28T13:00:17.648662+0000","last_active":"2023-11-28T13:00:18.497428+0000","last_peered":"2023-11-28T13:00:18.497428+0000","last_clean":"2023-11-28T13:00:18.497428+0000","last_became_active":"2023-11-28T13:00:17.648271+0000","last_became_peered":"2023-11-28T13:00:17.648271+0000","last_unstale":"2023-11-28T13:00:18.497428+0000","last_undegraded":"2023-11-28T13:00:18.497428+0000","last_fullsized":"2023-11-28T13:00:18.497428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:37.103258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330841+0000","last_change":"2023-11-28T13:00:17.643733+0000","last_active":"2023-11-28T13:00:18.330841+0000","last_peered":"2023-11-28T13:00:18.330841+0000","last_clean":"2023-11-28T13:00:18.330841+0000","last_became_active":"2023-11-28T13:00:17.643428+0000","last_became_peered":"2023-11-28T13:00:17.643428+0000","last_unstale":"2023-11-28T13:00:18.330841+0000","last_undegraded":"2023-11-28T13:00:18.330841+0000","last_fullsized":"2023-11-28T13:00:18.330841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652247+0000","last_change":"2023-11-28T13:00:17.650740+0000","last_active":"2023-11-28T13:00:17.652247+0000","last_peered":"2023-11-28T13:00:17.652247+0000","last_clean":"2023-11-28T13:00:17.652247+0000","last_became_active":"2023-11-28T13:00:17.650427+0000","last_became_peered":"2023-11-28T13:00:17.650427+0000","last_unstale":"2023-11-28T13:00:17.652247+0000","last_undegraded":"2023-11-28T13:00:17.652247+0000","last_fullsized":"2023-11-28T13:00:17.652247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497199+0000","last_change":"2023-11-28T13:00:17.652679+0000","last_active":"2023-11-28T13:00:18.497199+0000","last_peered":"2023-11-28T13:00:18.497199+0000","last_clean":"2023-11-28T13:00:18.497199+0000","last_became_active":"2023-11-28T13:00:17.648169+0000","last_became_peered":"2023-11-28T13:00:17.648169+0000","last_unstale":"2023-11-28T13:00:18.497199+0000","last_undegraded":"2023-11-28T13:00:18.497199+0000","last_fullsized":"2023-11-28T13:00:18.497199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:58.762522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331071+0000","last_change":"2023-11-28T13:00:17.647769+0000","last_active":"2023-11-28T13:00:18.331071+0000","last_peered":"2023-11-28T13:00:18.331071+0000","last_clean":"2023-11-28T13:00:18.331071+0000","last_became_active":"2023-11-28T13:00:17.647476+0000","last_became_peered":"2023-11-28T13:00:17.647476+0000","last_unstale":"2023-11-28T13:00:18.331071+0000","last_undegraded":"2023-11-28T13:00:18.331071+0000","last_fullsized":"2023-11-28T13:00:18.331071+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338605+0000","last_change":"2023-11-28T13:00:17.653233+0000","last_active":"2023-11-28T13:00:18.338605+0000","last_peered":"2023-11-28T13:00:18.338605+0000","last_clean":"2023-11-28T13:00:18.338605+0000","last_became_active":"2023-11-28T13:00:17.652957+0000","last_became_peered":"2023-11-28T13:00:17.652957+0000","last_unstale":"2023-11-28T13:00:18.338605+0000","last_undegraded":"2023-11-28T13:00:18.338605+0000","last_fullsized":"2023-11-28T13:00:18.338605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652321+0000","last_change":"2023-11-28T13:00:17.650826+0000","last_active":"2023-11-28T13:00:17.652321+0000","last_peered":"2023-11-28T13:00:17.652321+0000","last_clean":"2023-11-28T13:00:17.652321+0000","last_became_active":"2023-11-28T13:00:17.650572+0000","last_became_peered":"2023-11-28T13:00:17.650572+0000","last_unstale":"2023-11-28T13:00:17.652321+0000","last_undegraded":"2023-11-28T13:00:17.652321+0000","last_fullsized":"2023-11-28T13:00:17.652321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338780+0000","last_change":"2023-11-28T13:00:17.642144+0000","last_active":"2023-11-28T13:00:18.338780+0000","last_peered":"2023-11-28T13:00:18.338780+0000","last_clean":"2023-11-28T13:00:18.338780+0000","last_became_active":"2023-11-28T13:00:17.641814+0000","last_became_peered":"2023-11-28T13:00:17.641814+0000","last_unstale":"2023-11-28T13:00:18.338780+0000","last_undegraded":"2023-11-28T13:00:18.338780+0000","last_fullsized":"2023-11-28T13:00:18.338780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497444+0000","last_change":"2023-11-28T13:00:17.643190+0000","last_active":"2023-11-28T13:00:18.497444+0000","last_peered":"2023-11-28T13:00:18.497444+0000","last_clean":"2023-11-28T13:00:18.497444+0000","last_became_active":"2023-11-28T13:00:17.642741+0000","last_became_peered":"2023-11-28T13:00:17.642741+0000","last_unstale":"2023-11-28T13:00:18.497444+0000","last_undegraded":"2023-11-28T13:00:18.497444+0000","last_fullsized":"2023-11-28T13:00:18.497444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:33.241895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217739+0000","last_change":"2023-11-28T13:00:17.654503+0000","last_active":"2023-11-28T13:00:19.217739+0000","last_peered":"2023-11-28T13:00:19.217739+0000","last_clean":"2023-11-28T13:00:19.217739+0000","last_became_active":"2023-11-28T13:00:17.654204+0000","last_became_peered":"2023-11-28T13:00:17.654204+0000","last_unstale":"2023-11-28T13:00:19.217739+0000","last_undegraded":"2023-11-28T13:00:19.217739+0000","last_fullsized":"2023-11-28T13:00:19.217739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:27.440586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217678+0000","last_change":"2023-11-28T13:00:17.634718+0000","last_active":"2023-11-28T13:00:19.217678+0000","last_peered":"2023-11-28T13:00:19.217678+0000","last_clean":"2023-11-28T13:00:19.217678+0000","last_became_active":"2023-11-28T13:00:17.634420+0000","last_became_peered":"2023-11-28T13:00:17.634420+0000","last_unstale":"2023-11-28T13:00:19.217678+0000","last_undegraded":"2023-11-28T13:00:19.217678+0000","last_fullsized":"2023-11-28T13:00:19.217678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:07.567102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338634+0000","last_change":"2023-11-28T13:00:17.641361+0000","last_active":"2023-11-28T13:00:18.338634+0000","last_peered":"2023-11-28T13:00:18.338634+0000","last_clean":"2023-11-28T13:00:18.338634+0000","last_became_active":"2023-11-28T13:00:17.641078+0000","last_became_peered":"2023-11-28T13:00:17.641078+0000","last_unstale":"2023-11-28T13:00:18.338634+0000","last_undegraded":"2023-11-28T13:00:18.338634+0000","last_fullsized":"2023-11-28T13:00:18.338634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:38.545553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331110+0000","last_change":"2023-11-28T13:00:17.647800+0000","last_active":"2023-11-28T13:00:18.331110+0000","last_peered":"2023-11-28T13:00:18.331110+0000","last_clean":"2023-11-28T13:00:18.331110+0000","last_became_active":"2023-11-28T13:00:17.647519+0000","last_became_peered":"2023-11-28T13:00:17.647519+0000","last_unstale":"2023-11-28T13:00:18.331110+0000","last_undegraded":"2023-11-28T13:00:18.331110+0000","last_fullsized":"2023-11-28T13:00:18.331110+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497110+0000","last_change":"2023-11-28T13:00:17.651090+0000","last_active":"2023-11-28T13:00:18.497110+0000","last_peered":"2023-11-28T13:00:18.497110+0000","last_clean":"2023-11-28T13:00:18.497110+0000","last_became_active":"2023-11-28T13:00:17.649174+0000","last_became_peered":"2023-11-28T13:00:17.649174+0000","last_unstale":"2023-11-28T13:00:18.497110+0000","last_undegraded":"2023-11-28T13:00:18.497110+0000","last_fullsized":"2023-11-28T13:00:18.497110+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:32.070272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652338+0000","last_change":"2023-11-28T13:00:17.650764+0000","last_active":"2023-11-28T13:00:17.652338+0000","last_peered":"2023-11-28T13:00:17.652338+0000","last_clean":"2023-11-28T13:00:17.652338+0000","last_became_active":"2023-11-28T13:00:17.650252+0000","last_became_peered":"2023-11-28T13:00:17.650252+0000","last_unstale":"2023-11-28T13:00:17.652338+0000","last_undegraded":"2023-11-28T13:00:17.652338+0000","last_fullsized":"2023-11-28T13:00:17.652338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497486+0000","last_change":"2023-11-28T13:00:17.648771+0000","last_active":"2023-11-28T13:00:18.497486+0000","last_peered":"2023-11-28T13:00:18.497486+0000","last_clean":"2023-11-28T13:00:18.497486+0000","last_became_active":"2023-11-28T13:00:17.646367+0000","last_became_peered":"2023-11-28T13:00:17.646367+0000","last_unstale":"2023-11-28T13:00:18.497486+0000","last_undegraded":"2023-11-28T13:00:18.497486+0000","last_fullsized":"2023-11-28T13:00:18.497486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:33.549492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217793+0000","last_change":"2023-11-28T13:00:17.654191+0000","last_active":"2023-11-28T13:00:19.217793+0000","last_peered":"2023-11-28T13:00:19.217793+0000","last_clean":"2023-11-28T13:00:19.217793+0000","last_became_active":"2023-11-28T13:00:17.653922+0000","last_became_peered":"2023-11-28T13:00:17.653922+0000","last_unstale":"2023-11-28T13:00:19.217793+0000","last_undegraded":"2023-11-28T13:00:19.217793+0000","last_fullsized":"2023-11-28T13:00:19.217793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:59.066202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497368+0000","last_change":"2023-11-28T13:00:17.653882+0000","last_active":"2023-11-28T13:00:18.497368+0000","last_peered":"2023-11-28T13:00:18.497368+0000","last_clean":"2023-11-28T13:00:18.497368+0000","last_became_active":"2023-11-28T13:00:17.653497+0000","last_became_peered":"2023-11-28T13:00:17.653497+0000","last_unstale":"2023-11-28T13:00:18.497368+0000","last_undegraded":"2023-11-28T13:00:18.497368+0000","last_fullsized":"2023-11-28T13:00:18.497368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:59.591407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331153+0000","last_change":"2023-11-28T13:00:17.652105+0000","last_active":"2023-11-28T13:00:18.331153+0000","last_peered":"2023-11-28T13:00:18.331153+0000","last_clean":"2023-11-28T13:00:18.331153+0000","last_became_active":"2023-11-28T13:00:17.651804+0000","last_became_peered":"2023-11-28T13:00:17.651804+0000","last_unstale":"2023-11-28T13:00:18.331153+0000","last_undegraded":"2023-11-28T13:00:18.331153+0000","last_fullsized":"2023-11-28T13:00:18.331153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650622+0000","last_change":"2023-11-28T13:00:17.647496+0000","last_active":"2023-11-28T13:00:17.650622+0000","last_peered":"2023-11-28T13:00:17.650622+0000","last_clean":"2023-11-28T13:00:17.650622+0000","last_became_active":"2023-11-28T13:00:17.647234+0000","last_became_peered":"2023-11-28T13:00:17.647234+0000","last_unstale":"2023-11-28T13:00:17.650622+0000","last_undegraded":"2023-11-28T13:00:17.650622+0000","last_fullsized":"2023-11-28T13:00:17.650622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215336+0000","last_change":"2023-11-28T13:00:17.645773+0000","last_active":"2023-11-28T13:00:19.215336+0000","last_peered":"2023-11-28T13:00:19.215336+0000","last_clean":"2023-11-28T13:00:19.215336+0000","last_became_active":"2023-11-28T13:00:17.643545+0000","last_became_peered":"2023-11-28T13:00:17.643545+0000","last_unstale":"2023-11-28T13:00:19.215336+0000","last_undegraded":"2023-11-28T13:00:19.215336+0000","last_fullsized":"2023-11-28T13:00:19.215336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:36.504863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331033+0000","last_change":"2023-11-28T13:00:17.651625+0000","last_active":"2023-11-28T13:00:18.331033+0000","last_peered":"2023-11-28T13:00:18.331033+0000","last_clean":"2023-11-28T13:00:18.331033+0000","last_became_active":"2023-11-28T13:00:17.651210+0000","last_became_peered":"2023-11-28T13:00:17.651210+0000","last_unstale":"2023-11-28T13:00:18.331033+0000","last_undegraded":"2023-11-28T13:00:18.331033+0000","last_fullsized":"2023-11-28T13:00:18.331033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635880+0000","last_change":"2023-11-28T13:00:17.646690+0000","last_active":"2023-11-28T13:00:18.635880+0000","last_peered":"2023-11-28T13:00:18.635880+0000","last_clean":"2023-11-28T13:00:18.635880+0000","last_became_active":"2023-11-28T13:00:17.646438+0000","last_became_peered":"2023-11-28T13:00:17.646438+0000","last_unstale":"2023-11-28T13:00:18.635880+0000","last_undegraded":"2023-11-28T13:00:18.635880+0000","last_fullsized":"2023-11-28T13:00:18.635880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:52.870009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636047+0000","last_change":"2023-11-28T13:00:17.641404+0000","last_active":"2023-11-28T13:00:18.636047+0000","last_peered":"2023-11-28T13:00:18.636047+0000","last_clean":"2023-11-28T13:00:18.636047+0000","last_became_active":"2023-11-28T13:00:17.641140+0000","last_became_peered":"2023-11-28T13:00:17.641140+0000","last_unstale":"2023-11-28T13:00:18.636047+0000","last_undegraded":"2023-11-28T13:00:18.636047+0000","last_fullsized":"2023-11-28T13:00:18.636047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338691+0000","last_change":"2023-11-28T13:00:17.645292+0000","last_active":"2023-11-28T13:00:18.338691+0000","last_peered":"2023-11-28T13:00:18.338691+0000","last_clean":"2023-11-28T13:00:18.338691+0000","last_became_active":"2023-11-28T13:00:17.645001+0000","last_became_peered":"2023-11-28T13:00:17.645001+0000","last_unstale":"2023-11-28T13:00:18.338691+0000","last_undegraded":"2023-11-28T13:00:18.338691+0000","last_fullsized":"2023-11-28T13:00:18.338691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:05.072866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497240+0000","last_change":"2023-11-28T13:00:17.245858+0000","last_active":"2023-11-28T13:00:18.497240+0000","last_peered":"2023-11-28T13:00:18.497240+0000","last_clean":"2023-11-28T13:00:18.497240+0000","last_became_active":"2023-11-28T13:00:17.245568+0000","last_became_peered":"2023-11-28T13:00:17.245568+0000","last_unstale":"2023-11-28T13:00:18.497240+0000","last_undegraded":"2023-11-28T13:00:18.497240+0000","last_fullsized":"2023-11-28T13:00:18.497240+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215475+0000","last_change":"2023-11-28T13:00:17.644504+0000","last_active":"2023-11-28T13:00:19.215475+0000","last_peered":"2023-11-28T13:00:19.215475+0000","last_clean":"2023-11-28T13:00:19.215475+0000","last_became_active":"2023-11-28T13:00:17.644099+0000","last_became_peered":"2023-11-28T13:00:17.644099+0000","last_unstale":"2023-11-28T13:00:19.215475+0000","last_undegraded":"2023-11-28T13:00:19.215475+0000","last_fullsized":"2023-11-28T13:00:19.215475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635846+0000","last_change":"2023-11-28T13:00:17.236124+0000","last_active":"2023-11-28T13:00:18.635846+0000","last_peered":"2023-11-28T13:00:18.635846+0000","last_clean":"2023-11-28T13:00:18.635846+0000","last_became_active":"2023-11-28T13:00:17.235500+0000","last_became_peered":"2023-11-28T13:00:17.235500+0000","last_unstale":"2023-11-28T13:00:18.635846+0000","last_undegraded":"2023-11-28T13:00:18.635846+0000","last_fullsized":"2023-11-28T13:00:18.635846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:19.252999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648987+0000","last_change":"2023-11-28T13:00:17.642697+0000","last_active":"2023-11-28T13:00:17.648987+0000","last_peered":"2023-11-28T13:00:17.648987+0000","last_clean":"2023-11-28T13:00:17.648987+0000","last_became_active":"2023-11-28T13:00:17.642400+0000","last_became_peered":"2023-11-28T13:00:17.642400+0000","last_unstale":"2023-11-28T13:00:17.648987+0000","last_undegraded":"2023-11-28T13:00:17.648987+0000","last_fullsized":"2023-11-28T13:00:17.648987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:16:07.582318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215390+0000","last_change":"2023-11-28T13:00:17.240445+0000","last_active":"2023-11-28T13:00:19.215390+0000","last_peered":"2023-11-28T13:00:19.215390+0000","last_clean":"2023-11-28T13:00:19.215390+0000","last_became_active":"2023-11-28T13:00:17.240095+0000","last_became_peered":"2023-11-28T13:00:17.240095+0000","last_unstale":"2023-11-28T13:00:19.215390+0000","last_undegraded":"2023-11-28T13:00:19.215390+0000","last_fullsized":"2023-11-28T13:00:19.215390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330869+0000","last_change":"2023-11-28T13:00:17.644329+0000","last_active":"2023-11-28T13:00:18.330869+0000","last_peered":"2023-11-28T13:00:18.330869+0000","last_clean":"2023-11-28T13:00:18.330869+0000","last_became_active":"2023-11-28T13:00:17.644046+0000","last_became_peered":"2023-11-28T13:00:17.644046+0000","last_unstale":"2023-11-28T13:00:18.330869+0000","last_undegraded":"2023-11-28T13:00:18.330869+0000","last_fullsized":"2023-11-28T13:00:18.330869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.849042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.631440+0000","last_change":"2023-11-28T13:00:17.242514+0000","last_active":"2023-11-28T13:00:17.631440+0000","last_peered":"2023-11-28T13:00:17.631440+0000","last_clean":"2023-11-28T13:00:17.631440+0000","last_became_active":"2023-11-28T13:00:17.242258+0000","last_became_peered":"2023-11-28T13:00:17.242258+0000","last_unstale":"2023-11-28T13:00:17.631440+0000","last_undegraded":"2023-11-28T13:00:17.631440+0000","last_fullsized":"2023-11-28T13:00:17.631440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215295+0000","last_change":"2023-11-28T13:00:17.642442+0000","last_active":"2023-11-28T13:00:19.215295+0000","last_peered":"2023-11-28T13:00:19.215295+0000","last_clean":"2023-11-28T13:00:19.215295+0000","last_became_active":"2023-11-28T13:00:17.642162+0000","last_became_peered":"2023-11-28T13:00:17.642162+0000","last_unstale":"2023-11-28T13:00:19.215295+0000","last_undegraded":"2023-11-28T13:00:19.215295+0000","last_fullsized":"2023-11-28T13:00:19.215295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338768+0000","last_change":"2023-11-28T13:00:17.238867+0000","last_active":"2023-11-28T13:00:18.338768+0000","last_peered":"2023-11-28T13:00:18.338768+0000","last_clean":"2023-11-28T13:00:18.338768+0000","last_became_active":"2023-11-28T13:00:17.238545+0000","last_became_peered":"2023-11-28T13:00:17.238545+0000","last_unstale":"2023-11-28T13:00:18.338768+0000","last_undegraded":"2023-11-28T13:00:18.338768+0000","last_fullsized":"2023-11-28T13:00:18.338768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215335+0000","last_change":"2023-11-28T13:00:17.645527+0000","last_active":"2023-11-28T13:00:19.215335+0000","last_peered":"2023-11-28T13:00:19.215335+0000","last_clean":"2023-11-28T13:00:19.215335+0000","last_became_active":"2023-11-28T13:00:17.644057+0000","last_became_peered":"2023-11-28T13:00:17.644057+0000","last_unstale":"2023-11-28T13:00:19.215335+0000","last_undegraded":"2023-11-28T13:00:19.215335+0000","last_fullsized":"2023-11-28T13:00:19.215335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:55.013485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"24'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331175+0000","last_change":"2023-11-28T13:00:17.242375+0000","last_active":"2023-11-28T13:00:18.331175+0000","last_peered":"2023-11-28T13:00:18.331175+0000","last_clean":"2023-11-28T13:00:18.331175+0000","last_became_active":"2023-11-28T13:00:17.242010+0000","last_became_peered":"2023-11-28T13:00:17.242010+0000","last_unstale":"2023-11-28T13:00:18.331175+0000","last_undegraded":"2023-11-28T13:00:18.331175+0000","last_fullsized":"2023-11-28T13:00:18.331175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.522601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635882+0000","last_change":"2023-11-28T13:00:17.647427+0000","last_active":"2023-11-28T13:00:18.635882+0000","last_peered":"2023-11-28T13:00:18.635882+0000","last_clean":"2023-11-28T13:00:18.635882+0000","last_became_active":"2023-11-28T13:00:17.646853+0000","last_became_peered":"2023-11-28T13:00:17.646853+0000","last_unstale":"2023-11-28T13:00:18.635882+0000","last_undegraded":"2023-11-28T13:00:18.635882+0000","last_fullsized":"2023-11-28T13:00:18.635882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"24'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497407+0000","last_change":"2023-11-28T13:00:17.243823+0000","last_active":"2023-11-28T13:00:18.497407+0000","last_peered":"2023-11-28T13:00:18.497407+0000","last_clean":"2023-11-28T13:00:18.497407+0000","last_became_active":"2023-11-28T13:00:17.241238+0000","last_became_peered":"2023-11-28T13:00:17.241238+0000","last_unstale":"2023-11-28T13:00:18.497407+0000","last_undegraded":"2023-11-28T13:00:18.497407+0000","last_fullsized":"2023-11-28T13:00:18.497407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497280+0000","last_change":"2023-11-28T13:00:17.649986+0000","last_active":"2023-11-28T13:00:18.497280+0000","last_peered":"2023-11-28T13:00:18.497280+0000","last_clean":"2023-11-28T13:00:18.497280+0000","last_became_active":"2023-11-28T13:00:17.649402+0000","last_became_peered":"2023-11-28T13:00:17.649402+0000","last_unstale":"2023-11-28T13:00:18.497280+0000","last_undegraded":"2023-11-28T13:00:18.497280+0000","last_fullsized":"2023-11-28T13:00:18.497280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:15.471790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331132+0000","last_change":"2023-11-28T13:00:17.247047+0000","last_active":"2023-11-28T13:00:18.331132+0000","last_peered":"2023-11-28T13:00:18.331132+0000","last_clean":"2023-11-28T13:00:18.331132+0000","last_became_active":"2023-11-28T13:00:17.246742+0000","last_became_peered":"2023-11-28T13:00:17.246742+0000","last_unstale":"2023-11-28T13:00:18.331132+0000","last_undegraded":"2023-11-28T13:00:18.331132+0000","last_fullsized":"2023-11-28T13:00:18.331132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:50.559956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215527+0000","last_change":"2023-11-28T13:00:17.641605+0000","last_active":"2023-11-28T13:00:19.215527+0000","last_peered":"2023-11-28T13:00:19.215527+0000","last_clean":"2023-11-28T13:00:19.215527+0000","last_became_active":"2023-11-28T13:00:17.641350+0000","last_became_peered":"2023-11-28T13:00:17.641350+0000","last_unstale":"2023-11-28T13:00:19.215527+0000","last_undegraded":"2023-11-28T13:00:19.215527+0000","last_fullsized":"2023-11-28T13:00:19.215527+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:34.900070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.630517+0000","last_change":"2023-11-28T13:00:17.240432+0000","last_active":"2023-11-28T13:00:17.630517+0000","last_peered":"2023-11-28T13:00:17.630517+0000","last_clean":"2023-11-28T13:00:17.630517+0000","last_became_active":"2023-11-28T13:00:17.239192+0000","last_became_peered":"2023-11-28T13:00:17.239192+0000","last_unstale":"2023-11-28T13:00:17.630517+0000","last_undegraded":"2023-11-28T13:00:17.630517+0000","last_fullsized":"2023-11-28T13:00:17.630517+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:38.430703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338702+0000","last_change":"2023-11-28T13:00:17.648774+0000","last_active":"2023-11-28T13:00:18.338702+0000","last_peered":"2023-11-28T13:00:18.338702+0000","last_clean":"2023-11-28T13:00:18.338702+0000","last_became_active":"2023-11-28T13:00:17.647403+0000","last_became_peered":"2023-11-28T13:00:17.647403+0000","last_unstale":"2023-11-28T13:00:18.338702+0000","last_undegraded":"2023-11-28T13:00:18.338702+0000","last_fullsized":"2023-11-28T13:00:18.338702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:17.592744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331091+0000","last_change":"2023-11-28T13:00:17.245977+0000","last_active":"2023-11-28T13:00:18.331091+0000","last_peered":"2023-11-28T13:00:18.331091+0000","last_clean":"2023-11-28T13:00:18.331091+0000","last_became_active":"2023-11-28T13:00:17.245651+0000","last_became_peered":"2023-11-28T13:00:17.245651+0000","last_unstale":"2023-11-28T13:00:18.331091+0000","last_undegraded":"2023-11-28T13:00:18.331091+0000","last_fullsized":"2023-11-28T13:00:18.331091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:30.760573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.649006+0000","last_change":"2023-11-28T13:00:17.644029+0000","last_active":"2023-11-28T13:00:17.649006+0000","last_peered":"2023-11-28T13:00:17.649006+0000","last_clean":"2023-11-28T13:00:17.649006+0000","last_became_active":"2023-11-28T13:00:17.643789+0000","last_became_peered":"2023-11-28T13:00:17.643789+0000","last_unstale":"2023-11-28T13:00:17.649006+0000","last_undegraded":"2023-11-28T13:00:17.649006+0000","last_fullsized":"2023-11-28T13:00:17.649006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:13.404172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"24'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330786+0000","last_change":"2023-11-28T13:00:17.240716+0000","last_active":"2023-11-28T13:00:18.330786+0000","last_peered":"2023-11-28T13:00:18.330786+0000","last_clean":"2023-11-28T13:00:18.330786+0000","last_became_active":"2023-11-28T13:00:17.239928+0000","last_became_peered":"2023-11-28T13:00:17.239928+0000","last_unstale":"2023-11-28T13:00:18.330786+0000","last_undegraded":"2023-11-28T13:00:18.330786+0000","last_fullsized":"2023-11-28T13:00:18.330786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:32.086174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215680+0000","last_change":"2023-11-28T13:00:17.643923+0000","last_active":"2023-11-28T13:00:19.215680+0000","last_peered":"2023-11-28T13:00:19.215680+0000","last_clean":"2023-11-28T13:00:19.215680+0000","last_became_active":"2023-11-28T13:00:17.643579+0000","last_became_peered":"2023-11-28T13:00:17.643579+0000","last_unstale":"2023-11-28T13:00:19.215680+0000","last_undegraded":"2023-11-28T13:00:19.215680+0000","last_fullsized":"2023-11-28T13:00:19.215680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331252+0000","last_change":"2023-11-28T13:00:17.247259+0000","last_active":"2023-11-28T13:00:18.331252+0000","last_peered":"2023-11-28T13:00:18.331252+0000","last_clean":"2023-11-28T13:00:18.331252+0000","last_became_active":"2023-11-28T13:00:17.246943+0000","last_became_peered":"2023-11-28T13:00:17.246943+0000","last_unstale":"2023-11-28T13:00:18.331252+0000","last_undegraded":"2023-11-28T13:00:18.331252+0000","last_fullsized":"2023-11-28T13:00:18.331252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:02.643388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338621+0000","last_change":"2023-11-28T13:00:17.644322+0000","last_active":"2023-11-28T13:00:18.338621+0000","last_peered":"2023-11-28T13:00:18.338621+0000","last_clean":"2023-11-28T13:00:18.338621+0000","last_became_active":"2023-11-28T13:00:17.644054+0000","last_became_peered":"2023-11-28T13:00:17.644054+0000","last_unstale":"2023-11-28T13:00:18.338621+0000","last_undegraded":"2023-11-28T13:00:18.338621+0000","last_fullsized":"2023-11-28T13:00:18.338621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:46.098419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331230+0000","last_change":"2023-11-28T13:00:17.247274+0000","last_active":"2023-11-28T13:00:18.331230+0000","last_peered":"2023-11-28T13:00:18.331230+0000","last_clean":"2023-11-28T13:00:18.331230+0000","last_became_active":"2023-11-28T13:00:17.246851+0000","last_became_peered":"2023-11-28T13:00:17.246851+0000","last_unstale":"2023-11-28T13:00:18.331230+0000","last_undegraded":"2023-11-28T13:00:18.331230+0000","last_fullsized":"2023-11-28T13:00:18.331230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:17.621082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497137+0000","last_change":"2023-11-28T13:00:17.640395+0000","last_active":"2023-11-28T13:00:18.497137+0000","last_peered":"2023-11-28T13:00:18.497137+0000","last_clean":"2023-11-28T13:00:18.497137+0000","last_became_active":"2023-11-28T13:00:17.640195+0000","last_became_peered":"2023-11-28T13:00:17.640195+0000","last_unstale":"2023-11-28T13:00:18.497137+0000","last_undegraded":"2023-11-28T13:00:18.497137+0000","last_fullsized":"2023-11-28T13:00:18.497137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:13:14.878366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.331208+0000","last_change":"2023-11-28T13:00:17.240698+0000","last_active":"2023-11-28T13:00:18.331208+0000","last_peered":"2023-11-28T13:00:18.331208+0000","last_clean":"2023-11-28T13:00:18.331208+0000","last_became_active":"2023-11-28T13:00:17.240387+0000","last_became_peered":"2023-11-28T13:00:17.240387+0000","last_unstale":"2023-11-28T13:00:18.331208+0000","last_undegraded":"2023-11-28T13:00:18.331208+0000","last_fullsized":"2023-11-28T13:00:18.331208+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:05.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330832+0000","last_change":"2023-11-28T13:00:17.647318+0000","last_active":"2023-11-28T13:00:18.330832+0000","last_peered":"2023-11-28T13:00:18.330832+0000","last_clean":"2023-11-28T13:00:18.330832+0000","last_became_active":"2023-11-28T13:00:17.647043+0000","last_became_peered":"2023-11-28T13:00:17.647043+0000","last_unstale":"2023-11-28T13:00:18.330832+0000","last_undegraded":"2023-11-28T13:00:18.330832+0000","last_fullsized":"2023-11-28T13:00:18.330832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:38.321116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338680+0000","last_change":"2023-11-28T13:00:17.235843+0000","last_active":"2023-11-28T13:00:18.338680+0000","last_peered":"2023-11-28T13:00:18.338680+0000","last_clean":"2023-11-28T13:00:18.338680+0000","last_became_active":"2023-11-28T13:00:17.235495+0000","last_became_peered":"2023-11-28T13:00:17.235495+0000","last_unstale":"2023-11-28T13:00:18.338680+0000","last_undegraded":"2023-11-28T13:00:18.338680+0000","last_fullsized":"2023-11-28T13:00:18.338680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:33.975916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648938+0000","last_change":"2023-11-28T13:00:17.640727+0000","last_active":"2023-11-28T13:00:17.648938+0000","last_peered":"2023-11-28T13:00:17.648938+0000","last_clean":"2023-11-28T13:00:17.648938+0000","last_became_active":"2023-11-28T13:00:17.640438+0000","last_became_peered":"2023-11-28T13:00:17.640438+0000","last_unstale":"2023-11-28T13:00:17.648938+0000","last_undegraded":"2023-11-28T13:00:17.648938+0000","last_fullsized":"2023-11-28T13:00:17.648938+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215456+0000","last_change":"2023-11-28T13:00:17.240709+0000","last_active":"2023-11-28T13:00:19.215456+0000","last_peered":"2023-11-28T13:00:19.215456+0000","last_clean":"2023-11-28T13:00:19.215456+0000","last_became_active":"2023-11-28T13:00:17.239893+0000","last_became_peered":"2023-11-28T13:00:17.239893+0000","last_unstale":"2023-11-28T13:00:19.215456+0000","last_undegraded":"2023-11-28T13:00:19.215456+0000","last_fullsized":"2023-11-28T13:00:19.215456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497386+0000","last_change":"2023-11-28T13:00:17.653235+0000","last_active":"2023-11-28T13:00:18.497386+0000","last_peered":"2023-11-28T13:00:18.497386+0000","last_clean":"2023-11-28T13:00:18.497386+0000","last_became_active":"2023-11-28T13:00:17.650249+0000","last_became_peered":"2023-11-28T13:00:17.650249+0000","last_unstale":"2023-11-28T13:00:18.497386+0000","last_undegraded":"2023-11-28T13:00:18.497386+0000","last_fullsized":"2023-11-28T13:00:18.497386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.503594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330820+0000","last_change":"2023-11-28T13:00:17.245224+0000","last_active":"2023-11-28T13:00:18.330820+0000","last_peered":"2023-11-28T13:00:18.330820+0000","last_clean":"2023-11-28T13:00:18.330820+0000","last_became_active":"2023-11-28T13:00:17.244913+0000","last_became_peered":"2023-11-28T13:00:17.244913+0000","last_unstale":"2023-11-28T13:00:18.330820+0000","last_undegraded":"2023-11-28T13:00:18.330820+0000","last_fullsized":"2023-11-28T13:00:18.330820+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:24.336186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497463+0000","last_change":"2023-11-28T13:00:17.643015+0000","last_active":"2023-11-28T13:00:18.497463+0000","last_peered":"2023-11-28T13:00:18.497463+0000","last_clean":"2023-11-28T13:00:18.497463+0000","last_became_active":"2023-11-28T13:00:17.642470+0000","last_became_peered":"2023-11-28T13:00:17.642470+0000","last_unstale":"2023-11-28T13:00:18.497463+0000","last_undegraded":"2023-11-28T13:00:18.497463+0000","last_fullsized":"2023-11-28T13:00:18.497463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:41.679867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497180+0000","last_change":"2023-11-28T13:00:17.244207+0000","last_active":"2023-11-28T13:00:18.497180+0000","last_peered":"2023-11-28T13:00:18.497180+0000","last_clean":"2023-11-28T13:00:18.497180+0000","last_became_active":"2023-11-28T13:00:17.243861+0000","last_became_peered":"2023-11-28T13:00:17.243861+0000","last_unstale":"2023-11-28T13:00:18.497180+0000","last_undegraded":"2023-11-28T13:00:18.497180+0000","last_fullsized":"2023-11-28T13:00:18.497180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497504+0000","last_change":"2023-11-28T13:00:17.651269+0000","last_active":"2023-11-28T13:00:18.497504+0000","last_peered":"2023-11-28T13:00:18.497504+0000","last_clean":"2023-11-28T13:00:18.497504+0000","last_became_active":"2023-11-28T13:00:17.650883+0000","last_became_peered":"2023-11-28T13:00:17.650883+0000","last_unstale":"2023-11-28T13:00:18.497504+0000","last_undegraded":"2023-11-28T13:00:18.497504+0000","last_fullsized":"2023-11-28T13:00:18.497504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:13.856922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215191+0000","last_change":"2023-11-28T13:00:17.238617+0000","last_active":"2023-11-28T13:00:19.215191+0000","last_peered":"2023-11-28T13:00:19.215191+0000","last_clean":"2023-11-28T13:00:19.215191+0000","last_became_active":"2023-11-28T13:00:17.238344+0000","last_became_peered":"2023-11-28T13:00:17.238344+0000","last_unstale":"2023-11-28T13:00:19.215191+0000","last_undegraded":"2023-11-28T13:00:19.215191+0000","last_fullsized":"2023-11-28T13:00:19.215191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:46.225565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636017+0000","last_change":"2023-11-28T13:00:17.649564+0000","last_active":"2023-11-28T13:00:18.636017+0000","last_peered":"2023-11-28T13:00:18.636017+0000","last_clean":"2023-11-28T13:00:18.636017+0000","last_became_active":"2023-11-28T13:00:17.649312+0000","last_became_peered":"2023-11-28T13:00:17.649312+0000","last_unstale":"2023-11-28T13:00:18.636017+0000","last_undegraded":"2023-11-28T13:00:18.636017+0000","last_fullsized":"2023-11-28T13:00:18.636017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634764+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:00:17.634764+0000","last_peered":"2023-11-28T13:00:17.634764+0000","last_clean":"2023-11-28T13:00:17.634764+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:00:17.634764+0000","last_undegraded":"2023-11-28T13:00:17.634764+0000","last_fullsized":"2023-11-28T13:00:17.634764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338730+0000","last_change":"2023-11-28T13:00:17.240503+0000","last_active":"2023-11-28T13:00:18.338730+0000","last_peered":"2023-11-28T13:00:18.338730+0000","last_clean":"2023-11-28T13:00:18.338730+0000","last_became_active":"2023-11-28T13:00:17.240256+0000","last_became_peered":"2023-11-28T13:00:17.240256+0000","last_unstale":"2023-11-28T13:00:18.338730+0000","last_undegraded":"2023-11-28T13:00:18.338730+0000","last_fullsized":"2023-11-28T13:00:18.338730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635946+0000","last_change":"2023-11-28T13:00:17.645964+0000","last_active":"2023-11-28T13:00:18.635946+0000","last_peered":"2023-11-28T13:00:18.635946+0000","last_clean":"2023-11-28T13:00:18.635946+0000","last_became_active":"2023-11-28T13:00:17.645384+0000","last_became_peered":"2023-11-28T13:00:17.645384+0000","last_unstale":"2023-11-28T13:00:18.635946+0000","last_undegraded":"2023-11-28T13:00:18.635946+0000","last_fullsized":"2023-11-28T13:00:18.635946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.770251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.634535+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:00:17.634535+0000","last_peered":"2023-11-28T13:00:17.634535+0000","last_clean":"2023-11-28T13:00:17.634535+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:00:17.634535+0000","last_undegraded":"2023-11-28T13:00:17.634535+0000","last_fullsized":"2023-11-28T13:00:17.634535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497313+0000","last_change":"2023-11-28T13:00:17.245773+0000","last_active":"2023-11-28T13:00:18.497313+0000","last_peered":"2023-11-28T13:00:18.497313+0000","last_clean":"2023-11-28T13:00:18.497313+0000","last_became_active":"2023-11-28T13:00:17.245409+0000","last_became_peered":"2023-11-28T13:00:17.245409+0000","last_unstale":"2023-11-28T13:00:18.497313+0000","last_undegraded":"2023-11-28T13:00:18.497313+0000","last_fullsized":"2023-11-28T13:00:18.497313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:00.519142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636216+0000","last_change":"2023-11-28T13:00:17.641395+0000","last_active":"2023-11-28T13:00:18.636216+0000","last_peered":"2023-11-28T13:00:18.636216+0000","last_clean":"2023-11-28T13:00:18.636216+0000","last_became_active":"2023-11-28T13:00:17.641149+0000","last_became_peered":"2023-11-28T13:00:17.641149+0000","last_unstale":"2023-11-28T13:00:18.636216+0000","last_undegraded":"2023-11-28T13:00:18.636216+0000","last_fullsized":"2023-11-28T13:00:18.636216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.628022+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:00:17.628022+0000","last_peered":"2023-11-28T13:00:17.628022+0000","last_clean":"2023-11-28T13:00:17.628022+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:00:17.628022+0000","last_undegraded":"2023-11-28T13:00:17.628022+0000","last_fullsized":"2023-11-28T13:00:17.628022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635610+0000","last_change":"2023-11-28T13:00:17.237802+0000","last_active":"2023-11-28T13:00:18.635610+0000","last_peered":"2023-11-28T13:00:18.635610+0000","last_clean":"2023-11-28T13:00:18.635610+0000","last_became_active":"2023-11-28T13:00:17.237487+0000","last_became_peered":"2023-11-28T13:00:17.237487+0000","last_unstale":"2023-11-28T13:00:18.635610+0000","last_undegraded":"2023-11-28T13:00:18.635610+0000","last_fullsized":"2023-11-28T13:00:18.635610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:12:06.557853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.635877+0000","last_change":"2023-11-28T13:00:17.644511+0000","last_active":"2023-11-28T13:00:18.635877+0000","last_peered":"2023-11-28T13:00:18.635877+0000","last_clean":"2023-11-28T13:00:18.635877+0000","last_became_active":"2023-11-28T13:00:17.644095+0000","last_became_peered":"2023-11-28T13:00:17.644095+0000","last_unstale":"2023-11-28T13:00:18.635877+0000","last_undegraded":"2023-11-28T13:00:18.635877+0000","last_fullsized":"2023-11-28T13:00:18.635877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:22.904547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.636163+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:00:17.636163+0000","last_peered":"2023-11-28T13:00:17.636163+0000","last_clean":"2023-11-28T13:00:17.636163+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:00:17.636163+0000","last_undegraded":"2023-11-28T13:00:17.636163+0000","last_fullsized":"2023-11-28T13:00:17.636163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633158+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:00:17.633158+0000","last_peered":"2023-11-28T13:00:17.633158+0000","last_clean":"2023-11-28T13:00:17.633158+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:00:17.633158+0000","last_undegraded":"2023-11-28T13:00:17.633158+0000","last_fullsized":"2023-11-28T13:00:17.633158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215311+0000","last_change":"2023-11-28T13:00:17.239373+0000","last_active":"2023-11-28T13:00:19.215311+0000","last_peered":"2023-11-28T13:00:19.215311+0000","last_clean":"2023-11-28T13:00:19.215311+0000","last_became_active":"2023-11-28T13:00:17.238864+0000","last_became_peered":"2023-11-28T13:00:17.238864+0000","last_unstale":"2023-11-28T13:00:19.215311+0000","last_undegraded":"2023-11-28T13:00:19.215311+0000","last_fullsized":"2023-11-28T13:00:19.215311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:17.763933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330851+0000","last_change":"2023-11-28T13:00:17.649771+0000","last_active":"2023-11-28T13:00:18.330851+0000","last_peered":"2023-11-28T13:00:18.330851+0000","last_clean":"2023-11-28T13:00:18.330851+0000","last_became_active":"2023-11-28T13:00:17.649475+0000","last_became_peered":"2023-11-28T13:00:17.649475+0000","last_unstale":"2023-11-28T13:00:18.330851+0000","last_undegraded":"2023-11-28T13:00:18.330851+0000","last_fullsized":"2023-11-28T13:00:18.330851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:31.769822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633524+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:00:17.633524+0000","last_peered":"2023-11-28T13:00:17.633524+0000","last_clean":"2023-11-28T13:00:17.633524+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:00:17.633524+0000","last_undegraded":"2023-11-28T13:00:17.633524+0000","last_fullsized":"2023-11-28T13:00:17.633524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.630289+0000","last_change":"2023-11-28T13:00:17.251262+0000","last_active":"2023-11-28T13:00:17.630289+0000","last_peered":"2023-11-28T13:00:17.630289+0000","last_clean":"2023-11-28T13:00:17.630289+0000","last_became_active":"2023-11-28T13:00:17.250948+0000","last_became_peered":"2023-11-28T13:00:17.250948+0000","last_unstale":"2023-11-28T13:00:17.630289+0000","last_undegraded":"2023-11-28T13:00:17.630289+0000","last_fullsized":"2023-11-28T13:00:17.630289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:37.311866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.648968+0000","last_change":"2023-11-28T13:00:17.640685+0000","last_active":"2023-11-28T13:00:17.648968+0000","last_peered":"2023-11-28T13:00:17.648968+0000","last_clean":"2023-11-28T13:00:17.648968+0000","last_became_active":"2023-11-28T13:00:17.640420+0000","last_became_peered":"2023-11-28T13:00:17.640420+0000","last_unstale":"2023-11-28T13:00:17.648968+0000","last_undegraded":"2023-11-28T13:00:17.648968+0000","last_fullsized":"2023-11-28T13:00:17.648968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:14.562781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.632566+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:00:17.632566+0000","last_peered":"2023-11-28T13:00:17.632566+0000","last_clean":"2023-11-28T13:00:17.632566+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:00:17.632566+0000","last_undegraded":"2023-11-28T13:00:17.632566+0000","last_fullsized":"2023-11-28T13:00:17.632566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636224+0000","last_change":"2023-11-28T13:00:17.237999+0000","last_active":"2023-11-28T13:00:18.636224+0000","last_peered":"2023-11-28T13:00:18.636224+0000","last_clean":"2023-11-28T13:00:18.636224+0000","last_became_active":"2023-11-28T13:00:17.237625+0000","last_became_peered":"2023-11-28T13:00:17.237625+0000","last_unstale":"2023-11-28T13:00:18.636224+0000","last_undegraded":"2023-11-28T13:00:18.636224+0000","last_fullsized":"2023-11-28T13:00:18.636224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:58.056953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.650614+0000","last_change":"2023-11-28T13:00:17.644943+0000","last_active":"2023-11-28T13:00:17.650614+0000","last_peered":"2023-11-28T13:00:17.650614+0000","last_clean":"2023-11-28T13:00:17.650614+0000","last_became_active":"2023-11-28T13:00:17.644679+0000","last_became_peered":"2023-11-28T13:00:17.644679+0000","last_unstale":"2023-11-28T13:00:17.650614+0000","last_undegraded":"2023-11-28T13:00:17.650614+0000","last_fullsized":"2023-11-28T13:00:17.650614+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"24'2","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215660+0000","last_change":"2023-11-28T13:00:17.239284+0000","last_active":"2023-11-28T13:00:19.215660+0000","last_peered":"2023-11-28T13:00:19.215660+0000","last_clean":"2023-11-28T13:00:19.215660+0000","last_became_active":"2023-11-28T13:00:17.238997+0000","last_became_peered":"2023-11-28T13:00:17.238997+0000","last_unstale":"2023-11-28T13:00:19.215660+0000","last_undegraded":"2023-11-28T13:00:19.215660+0000","last_fullsized":"2023-11-28T13:00:19.215660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:56:55.423382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633198+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:00:17.633198+0000","last_peered":"2023-11-28T13:00:17.633198+0000","last_clean":"2023-11-28T13:00:17.633198+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:00:17.633198+0000","last_undegraded":"2023-11-28T13:00:17.633198+0000","last_fullsized":"2023-11-28T13:00:17.633198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215108+0000","last_change":"2023-11-28T13:00:17.646149+0000","last_active":"2023-11-28T13:00:19.215108+0000","last_peered":"2023-11-28T13:00:19.215108+0000","last_clean":"2023-11-28T13:00:19.215108+0000","last_became_active":"2023-11-28T13:00:17.645893+0000","last_became_peered":"2023-11-28T13:00:17.645893+0000","last_unstale":"2023-11-28T13:00:19.215108+0000","last_undegraded":"2023-11-28T13:00:19.215108+0000","last_fullsized":"2023-11-28T13:00:19.215108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:24.466207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.633216+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:00:17.633216+0000","last_peered":"2023-11-28T13:00:17.633216+0000","last_clean":"2023-11-28T13:00:17.633216+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:00:17.633216+0000","last_undegraded":"2023-11-28T13:00:17.633216+0000","last_fullsized":"2023-11-28T13:00:17.633216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.717720+0000","last_change":"2023-11-28T13:00:17.240379+0000","last_active":"2023-11-28T13:00:18.717720+0000","last_peered":"2023-11-28T13:00:18.717720+0000","last_clean":"2023-11-28T13:00:18.717720+0000","last_became_active":"2023-11-28T13:00:17.239505+0000","last_became_peered":"2023-11-28T13:00:17.239505+0000","last_unstale":"2023-11-28T13:00:18.717720+0000","last_undegraded":"2023-11-28T13:00:18.717720+0000","last_fullsized":"2023-11-28T13:00:18.717720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497157+0000","last_change":"2023-11-28T13:00:17.646003+0000","last_active":"2023-11-28T13:00:18.497157+0000","last_peered":"2023-11-28T13:00:18.497157+0000","last_clean":"2023-11-28T13:00:18.497157+0000","last_became_active":"2023-11-28T13:00:17.645727+0000","last_became_peered":"2023-11-28T13:00:17.645727+0000","last_unstale":"2023-11-28T13:00:18.497157+0000","last_undegraded":"2023-11-28T13:00:18.497157+0000","last_fullsized":"2023-11-28T13:00:18.497157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:30.565015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.656358+0000","last_change":"2023-11-28T13:00:17.251398+0000","last_active":"2023-11-28T13:00:17.656358+0000","last_peered":"2023-11-28T13:00:17.656358+0000","last_clean":"2023-11-28T13:00:17.656358+0000","last_became_active":"2023-11-28T13:00:17.251054+0000","last_became_peered":"2023-11-28T13:00:17.251054+0000","last_unstale":"2023-11-28T13:00:17.656358+0000","last_undegraded":"2023-11-28T13:00:17.656358+0000","last_fullsized":"2023-11-28T13:00:17.656358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.217848+0000","last_change":"2023-11-28T13:00:17.641355+0000","last_active":"2023-11-28T13:00:19.217848+0000","last_peered":"2023-11-28T13:00:19.217848+0000","last_clean":"2023-11-28T13:00:19.217848+0000","last_became_active":"2023-11-28T13:00:17.641093+0000","last_became_peered":"2023-11-28T13:00:17.641093+0000","last_unstale":"2023-11-28T13:00:19.217848+0000","last_undegraded":"2023-11-28T13:00:19.217848+0000","last_fullsized":"2023-11-28T13:00:19.217848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:41.816686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338665+0000","last_change":"2023-11-28T13:00:17.246896+0000","last_active":"2023-11-28T13:00:18.338665+0000","last_peered":"2023-11-28T13:00:18.338665+0000","last_clean":"2023-11-28T13:00:18.338665+0000","last_became_active":"2023-11-28T13:00:17.246604+0000","last_became_peered":"2023-11-28T13:00:17.246604+0000","last_unstale":"2023-11-28T13:00:18.338665+0000","last_undegraded":"2023-11-28T13:00:18.338665+0000","last_fullsized":"2023-11-28T13:00:18.338665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.044919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497332+0000","last_change":"2023-11-28T13:00:17.648973+0000","last_active":"2023-11-28T13:00:18.497332+0000","last_peered":"2023-11-28T13:00:18.497332+0000","last_clean":"2023-11-28T13:00:18.497332+0000","last_became_active":"2023-11-28T13:00:17.648323+0000","last_became_peered":"2023-11-28T13:00:17.648323+0000","last_unstale":"2023-11-28T13:00:18.497332+0000","last_undegraded":"2023-11-28T13:00:18.497332+0000","last_fullsized":"2023-11-28T13:00:18.497332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:48.280571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.338744+0000","last_change":"2023-11-28T13:00:17.247013+0000","last_active":"2023-11-28T13:00:18.338744+0000","last_peered":"2023-11-28T13:00:18.338744+0000","last_clean":"2023-11-28T13:00:18.338744+0000","last_became_active":"2023-11-28T13:00:17.246645+0000","last_became_peered":"2023-11-28T13:00:17.246645+0000","last_unstale":"2023-11-28T13:00:18.338744+0000","last_undegraded":"2023-11-28T13:00:18.338744+0000","last_fullsized":"2023-11-28T13:00:18.338744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:41.726419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497261+0000","last_change":"2023-11-28T13:00:17.649078+0000","last_active":"2023-11-28T13:00:18.497261+0000","last_peered":"2023-11-28T13:00:18.497261+0000","last_clean":"2023-11-28T13:00:18.497261+0000","last_became_active":"2023-11-28T13:00:17.648432+0000","last_became_peered":"2023-11-28T13:00:17.648432+0000","last_unstale":"2023-11-28T13:00:18.497261+0000","last_undegraded":"2023-11-28T13:00:18.497261+0000","last_fullsized":"2023-11-28T13:00:18.497261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:22:12.163075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"24'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215495+0000","last_change":"2023-11-28T13:00:17.234074+0000","last_active":"2023-11-28T13:00:19.215495+0000","last_peered":"2023-11-28T13:00:19.215495+0000","last_clean":"2023-11-28T13:00:19.215495+0000","last_became_active":"2023-11-28T13:00:17.233732+0000","last_became_peered":"2023-11-28T13:00:17.233732+0000","last_unstale":"2023-11-28T13:00:19.215495+0000","last_undegraded":"2023-11-28T13:00:19.215495+0000","last_fullsized":"2023-11-28T13:00:19.215495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:36.950864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215231+0000","last_change":"2023-11-28T13:00:17.641105+0000","last_active":"2023-11-28T13:00:19.215231+0000","last_peered":"2023-11-28T13:00:19.215231+0000","last_clean":"2023-11-28T13:00:19.215231+0000","last_became_active":"2023-11-28T13:00:17.640854+0000","last_became_peered":"2023-11-28T13:00:17.640854+0000","last_unstale":"2023-11-28T13:00:19.215231+0000","last_undegraded":"2023-11-28T13:00:19.215231+0000","last_fullsized":"2023-11-28T13:00:19.215231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:08.077069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"24'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:17.652233+0000","last_change":"2023-11-28T13:00:17.242508+0000","last_active":"2023-11-28T13:00:17.652233+0000","last_peered":"2023-11-28T13:00:17.652233+0000","last_clean":"2023-11-28T13:00:17.652233+0000","last_became_active":"2023-11-28T13:00:17.242230+0000","last_became_peered":"2023-11-28T13:00:17.242230+0000","last_unstale":"2023-11-28T13:00:17.652233+0000","last_undegraded":"2023-11-28T13:00:17.652233+0000","last_fullsized":"2023-11-28T13:00:17.652233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:16.205791+0000","last_clean_scrub_stamp":"2023-11-28T13:00:16.205791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:44.757088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497079+0000","last_change":"2023-11-28T13:00:17.650868+0000","last_active":"2023-11-28T13:00:18.497079+0000","last_peered":"2023-11-28T13:00:18.497079+0000","last_clean":"2023-11-28T13:00:18.497079+0000","last_became_active":"2023-11-28T13:00:17.649842+0000","last_became_peered":"2023-11-28T13:00:17.649842+0000","last_unstale":"2023-11-28T13:00:18.497079+0000","last_undegraded":"2023-11-28T13:00:18.497079+0000","last_fullsized":"2023-11-28T13:00:18.497079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:13.604429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.330807+0000","last_change":"2023-11-28T13:00:17.641330+0000","last_active":"2023-11-28T13:00:18.330807+0000","last_peered":"2023-11-28T13:00:18.330807+0000","last_clean":"2023-11-28T13:00:18.330807+0000","last_became_active":"2023-11-28T13:00:17.641029+0000","last_became_peered":"2023-11-28T13:00:17.641029+0000","last_unstale":"2023-11-28T13:00:18.330807+0000","last_undegraded":"2023-11-28T13:00:18.330807+0000","last_fullsized":"2023-11-28T13:00:18.330807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:28.607875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215202+0000","last_change":"2023-11-28T13:00:17.643816+0000","last_active":"2023-11-28T13:00:19.215202+0000","last_peered":"2023-11-28T13:00:19.215202+0000","last_clean":"2023-11-28T13:00:19.215202+0000","last_became_active":"2023-11-28T13:00:17.643260+0000","last_became_peered":"2023-11-28T13:00:17.643260+0000","last_unstale":"2023-11-28T13:00:19.215202+0000","last_undegraded":"2023-11-28T13:00:19.215202+0000","last_fullsized":"2023-11-28T13:00:19.215202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:29.688166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.497218+0000","last_change":"2023-11-28T13:00:17.649431+0000","last_active":"2023-11-28T13:00:18.497218+0000","last_peered":"2023-11-28T13:00:18.497218+0000","last_clean":"2023-11-28T13:00:18.497218+0000","last_became_active":"2023-11-28T13:00:17.648763+0000","last_became_peered":"2023-11-28T13:00:17.648763+0000","last_unstale":"2023-11-28T13:00:18.497218+0000","last_undegraded":"2023-11-28T13:00:18.497218+0000","last_fullsized":"2023-11-28T13:00:18.497218+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:10.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:19.215472+0000","last_change":"2023-11-28T13:00:17.649148+0000","last_active":"2023-11-28T13:00:19.215472+0000","last_peered":"2023-11-28T13:00:19.215472+0000","last_clean":"2023-11-28T13:00:19.215472+0000","last_became_active":"2023-11-28T13:00:17.648925+0000","last_became_peered":"2023-11-28T13:00:17.648925+0000","last_unstale":"2023-11-28T13:00:19.215472+0000","last_undegraded":"2023-11-28T13:00:19.215472+0000","last_fullsized":"2023-11-28T13:00:19.215472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:31.555004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:18.636138+0000","last_change":"2023-11-28T13:00:17.648409+0000","last_active":"2023-11-28T13:00:18.636138+0000","last_peered":"2023-11-28T13:00:18.636138+0000","last_clean":"2023-11-28T13:00:18.636138+0000","last_became_active":"2023-11-28T13:00:17.648164+0000","last_became_peered":"2023-11-28T13:00:17.648164+0000","last_unstale":"2023-11-28T13:00:18.636138+0000","last_undegraded":"2023-11-28T13:00:18.636138+0000","last_fullsized":"2023-11-28T13:00:18.636138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:17.215040+0000","last_clean_scrub_stamp":"2023-11-28T13:00:17.215040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:55.516084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":96947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.384}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3209999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1029999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7940,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363900,"statfs":{"total":96636764160,"available":96628633600,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8088,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363752,"statfs":{"total":96636764160,"available":96628482048,"internally_reserved":0,"allocated":278528,"data_stored":98461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9096,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362744,"statfs":{"total":96636764160,"available":96627449856,"internally_reserved":0,"allocated":851968,"data_stored":687022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9124,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362716,"statfs":{"total":96636764160,"available":96627421184,"internally_reserved":0,"allocated":880640,"data_stored":688989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8012,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363828,"statfs":{"total":96636764160,"available":96628559872,"internally_reserved":0,"allocated":266240,"data_stored":97493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8004,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363836,"statfs":{"total":96636764160,"available":96628568064,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8068,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363772,"statfs":{"total":96636764160,"available":96628502528,"internally_reserved":0,"allocated":258048,"data_stored":96857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:29.471 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T13:00:29.471 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T13:00:29.471 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T13:00:29.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:00:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.624+0000 7f960c383700 1 Processor -- start 2023-11-28T13:00:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.624+0000 7f960c383700 1 -- start start 2023-11-28T13:00:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960c383700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f96040d0a00 0x7f960415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960c383700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f96040b2930 0x7f96040b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960c383700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960406b420 con 0x7f96040ce040 2023-11-28T13:00:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960c383700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96040b32f0 con 0x7f96040b2930 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960c383700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960406ff60 con 0x7f96040d0a00 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f96040b2930 0x7f96040b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f96040b2930 0x7f96040b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38736/0 (socket says 172.21.15.153:38736) 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 -- 172.21.15.153:0/2947706454 learned_addr learned my addr 172.21.15.153:0/2947706454 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960991e700 1 --2- 172.21.15.153:0/2947706454 >> v2:172.21.15.177:3300/0 conn(0x7f96040d0a00 0x7f960415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 -- 172.21.15.153:0/2947706454 >> v2:172.21.15.177:3300/0 conn(0x7f96040d0a00 msgr2=0x7f960415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 --2- 172.21.15.153:0/2947706454 >> v2:172.21.15.177:3300/0 conn(0x7f96040d0a00 0x7f960415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 -- 172.21.15.153:0/2947706454 >> v1:172.21.15.177:6789/0 conn(0x7f96040ce040 legacy=0x7f960406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 -- 172.21.15.153:0/2947706454 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960406b700 con 0x7f96040b2930 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960a920700 1 --2- 172.21.15.153:0/2947706454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040b2930 0x7f96040b2db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f960406ed50 tx=0x7f95fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed873403d8b6daf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:29.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.625+0000 7f960911d700 1 -- 172.21.15.153:0/2947706454 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00e070 con 0x7f96040b2930 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960911d700 1 -- 172.21.15.153:0/2947706454 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc00b910 con 0x7f96040b2930 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960911d700 1 -- 172.21.15.153:0/2947706454 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00bcc0 con 0x7f96040b2930 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 -- 172.21.15.153:0/2947706454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040b2930 msgr2=0x7f96040b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 --2- 172.21.15.153:0/2947706454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040b2930 0x7f96040b2db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f960406ed50 tx=0x7f95fc00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 -- 172.21.15.153:0/2947706454 shutdown_connections 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 --2- 172.21.15.153:0/2947706454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040b2930 0x7f96040b2db0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 --2- 172.21.15.153:0/2947706454 >> v2:172.21.15.177:3300/0 conn(0x7f96040d0a00 0x7f960415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 -- 172.21.15.153:0/2947706454 >> 172.21.15.153:0/2947706454 conn(0x7f960406aed0 msgr2=0x7f960406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 -- 172.21.15.153:0/2947706454 shutdown_connections 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.626+0000 7f960c383700 1 -- 172.21.15.153:0/2947706454 wait complete. 2023-11-28T13:00:29.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.627+0000 7f960c383700 1 Processor -- start 2023-11-28T13:00:29.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.627+0000 7f960c383700 1 -- start start 2023-11-28T13:00:29.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960c383700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96040b2930 0x7f96040b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960c383700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96040ce040 0x7f96040c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960c383700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040d0a00 0x7f96041f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960c383700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040b6d90 con 0x7f96040d0a00 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960c383700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9604152800 con 0x7f96040ce040 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960c383700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96040b6430 con 0x7f96040b2930 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040d0a00 0x7f96041f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040d0a00 0x7f96041f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38742/0 (socket says 172.21.15.153:38742) 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 -- 172.21.15.153:0/2909831042 learned_addr learned my addr 172.21.15.153:0/2909831042 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960991e700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96040ce040 0x7f96040c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a11f700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96040b2930 0x7f96040b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 -- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96040b2930 msgr2=0x7f96040b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96040b2930 0x7f96040b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 -- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96040ce040 msgr2=0x7f96040c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96040ce040 0x7f96040c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960a920700 1 -- 172.21.15.153:0/2909831042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc009b30 con 0x7f96040d0a00 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.628+0000 7f960991e700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96040ce040 0x7f96040c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.629+0000 7f960a920700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040d0a00 0x7f96041f5a10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f96040b3aa0 tx=0x7f95fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.629+0000 7f95faffd700 1 -- 172.21.15.153:0/2909831042 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0099b0 con 0x7f96040d0a00 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.629+0000 7f95faffd700 1 -- 172.21.15.153:0/2909831042 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc0092e0 con 0x7f96040d0a00 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.629+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960406bcc0 con 0x7f96040d0a00 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.629+0000 7f95faffd700 1 -- 172.21.15.153:0/2909831042 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc01b630 con 0x7f96040d0a00 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.629+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960406b9e0 con 0x7f96040d0a00 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.630+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960406bfa0 con 0x7f96040d0a00 2023-11-28T13:00:29.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.630+0000 7f95faffd700 1 -- 172.21.15.153:0/2909831042 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f95fc021030 con 0x7f96040d0a00 2023-11-28T13:00:29.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.630+0000 7f95faffd700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f95e80682f0 0x7f95e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.631+0000 7f95faffd700 1 -- 172.21.15.153:0/2909831042 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f95fc0871a0 con 0x7f96040d0a00 2023-11-28T13:00:29.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.631+0000 7f960a11f700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f95e80682f0 0x7f95e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.631+0000 7f960a11f700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f95e80682f0 0x7f95e806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9600000fb0 tx=0x7f9600009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:29.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.635+0000 7f95faffd700 1 -- 172.21.15.153:0/2909831042 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95fc053980 con 0x7f96040d0a00 2023-11-28T13:00:29.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.859+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9604068690 con 0x7f96040d0a00 2023-11-28T13:00:29.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.859+0000 7f95faffd700 1 -- 172.21.15.153:0/2909831042 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f95fc056fa0 con 0x7f96040d0a00 2023-11-28T13:00:29.861 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:29.861 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T13:00:29.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f95e80682f0 msgr2=0x7f95e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f95e80682f0 0x7f95e806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9600000fb0 tx=0x7f9600009040 comp rx=0 tx=0).stop 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040d0a00 msgr2=0x7f96041f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040d0a00 0x7f96041f5a10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f96040b3aa0 tx=0x7f95fc004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 shutdown_connections 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f95e80682f0 0x7f95e806a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96040d0a00 0x7f96041f5a10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96040b2930 0x7f96040b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 --2- 172.21.15.153:0/2909831042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96040ce040 0x7f96040c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 >> 172.21.15.153:0/2909831042 conn(0x7f960406aed0 msgr2=0x7f96040cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 shutdown_connections 2023-11-28T13:00:29.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.861+0000 7f960c383700 1 -- 172.21.15.153:0/2909831042 wait complete. 2023-11-28T13:00:29.875 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T13:00:29.875 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:29.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.993+0000 7f6ff46bb700 1 Processor -- start 2023-11-28T13:00:29.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.993+0000 7f6ff46bb700 1 -- start start 2023-11-28T13:00:29.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.994+0000 7f6ff46bb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6fec140950 0x7f6fec0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.994+0000 7f6ff46bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.994+0000 7f6ff46bb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec13ff00 con 0x7f6fec0cd8c0 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.994+0000 7f6ff46bb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec1404c0 con 0x7f6fec140950 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.994+0000 7f6ff46bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec140dc0 con 0x7f6fec0b2e70 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6fec140950 0x7f6fec0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6fec140950 0x7f6fec0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38748/0 (socket says 172.21.15.153:38748) 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 -- 172.21.15.153:0/2676602984 learned_addr learned my addr 172.21.15.153:0/2676602984 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff2c58700 1 --2- 172.21.15.153:0/2676602984 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 -- 172.21.15.153:0/2676602984 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 msgr2=0x7f6fec06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 --2- 172.21.15.153:0/2676602984 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 -- 172.21.15.153:0/2676602984 >> v1:172.21.15.177:6789/0 conn(0x7f6fec0cd8c0 legacy=0x7f6fec0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 -- 172.21.15.153:0/2676602984 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec15aa80 con 0x7f6fec140950 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1c56700 1 --2- 172.21.15.153:0/2676602984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec140950 0x7f6fec0b2930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80098f0 tx=0x7f6fe8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9eef5a6e6cb395e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1455700 1 -- 172.21.15.153:0/2676602984 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800b8a0 con 0x7f6fec140950 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1455700 1 -- 172.21.15.153:0/2676602984 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe800ba00 con 0x7f6fec140950 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.995+0000 7f6ff1455700 1 -- 172.21.15.153:0/2676602984 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800bd90 con 0x7f6fec140950 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 -- 172.21.15.153:0/2676602984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec140950 msgr2=0x7f6fec0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/2676602984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec140950 0x7f6fec0b2930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80098f0 tx=0x7f6fe8009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 -- 172.21.15.153:0/2676602984 shutdown_connections 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/2676602984 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/2676602984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec140950 0x7f6fec0b2930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 -- 172.21.15.153:0/2676602984 >> 172.21.15.153:0/2676602984 conn(0x7f6fec06aed0 msgr2=0x7f6fec06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:29.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 -- 172.21.15.153:0/2676602984 shutdown_connections 2023-11-28T13:00:30.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.996+0000 7f6ff46bb700 1 -- 172.21.15.153:0/2676602984 wait complete. 2023-11-28T13:00:30.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.997+0000 7f6ff46bb700 1 Processor -- start 2023-11-28T13:00:30.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.997+0000 7f6ff46bb700 1 -- start start 2023-11-28T13:00:30.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec0cd8c0 0x7f6fec0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fec140950 0x7f6fec0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1594d0 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1429a0 con 0x7f6fec140950 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec142c80 con 0x7f6fec0b2e70 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec0cd8c0 0x7f6fec0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec0cd8c0 0x7f6fec0cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38754/0 (socket says 172.21.15.153:38754) 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 -- 172.21.15.153:0/195605864 learned_addr learned my addr 172.21.15.153:0/195605864 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff2457700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff2c58700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fec140950 0x7f6fec0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 -- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 msgr2=0x7f6fec158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 -- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fec140950 msgr2=0x7f6fec0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fec140950 0x7f6fec0cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 -- 172.21.15.153:0/195605864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8009550 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff2c58700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fec140950 0x7f6fec0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff1c56700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec0cd8c0 0x7f6fec0cf000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009c40 tx=0x7f6fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6fe2ffd700 1 -- 172.21.15.153:0/195605864 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80036a0 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6fe2ffd700 1 -- 172.21.15.153:0/195605864 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe801e070 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec0b35f0 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6fe2ffd700 1 -- 172.21.15.153:0/195605864 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8003a60 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.998+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec0b3310 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:29.999+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec140150 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.000+0000 7f6fe2ffd700 1 -- 172.21.15.153:0/195605864 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6fe800f050 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.000+0000 7f6fe2ffd700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6fd00682f0 0x7f6fd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.000+0000 7f6fe2ffd700 1 -- 172.21.15.153:0/195605864 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6fe8020070 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.000+0000 7f6ff2457700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6fd00682f0 0x7f6fd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.000+0000 7f6ff2457700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6fd00682f0 0x7f6fd006a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6fec13fe70 tx=0x7f6fdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.004+0000 7f6fe2ffd700 1 -- 172.21.15.153:0/195605864 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6fe80539d0 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.188+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fec068690 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.189+0000 7f6fe2ffd700 1 -- 172.21.15.153:0/195605864 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f6fe8023380 con 0x7f6fec0cd8c0 2023-11-28T13:00:30.191 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:30.191 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:30.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6fd00682f0 msgr2=0x7f6fd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6fd00682f0 0x7f6fd006a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6fec13fe70 tx=0x7f6fdc009040 comp rx=0 tx=0).stop 2023-11-28T13:00:30.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec0cd8c0 msgr2=0x7f6fec0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec0cd8c0 0x7f6fec0cf000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009c40 tx=0x7f6fe8004750 comp rx=0 tx=0).stop 2023-11-28T13:00:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 shutdown_connections 2023-11-28T13:00:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fec140950 0x7f6fec0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6fec0b2e70 0x7f6fec158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6fd00682f0 0x7f6fd006a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 --2- 172.21.15.153:0/195605864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6fec0cd8c0 0x7f6fec0cf000 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 >> 172.21.15.153:0/195605864 conn(0x7f6fec06aed0 msgr2=0x7f6fec0ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 shutdown_connections 2023-11-28T13:00:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.191+0000 7f6ff46bb700 1 -- 172.21.15.153:0/195605864 wait complete. 2023-11-28T13:00:30.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:30.206 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:30.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.328+0000 7fe5c6ea7700 1 Processor -- start 2023-11-28T13:00:30.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.328+0000 7fe5c6ea7700 1 -- start start 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5c6ea7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe5c0071040 0x7fe5c0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5c6ea7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe5c006dee0 0x7fe5c006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5c6ea7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c0071950 con 0x7fe5c006cdc0 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5c6ea7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c0161550 con 0x7fe5c006dee0 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5c6ea7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c015d060 con 0x7fe5c0071040 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe5c006dee0 0x7fe5c006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe5c006dee0 0x7fe5c006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38762/0 (socket says 172.21.15.153:38762) 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 -- 172.21.15.153:0/3858610467 learned_addr learned my addr 172.21.15.153:0/3858610467 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5c4c43700 1 --2- 172.21.15.153:0/3858610467 >> v2:172.21.15.177:3300/0 conn(0x7fe5c0071040 0x7fe5c0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 -- 172.21.15.153:0/3858610467 >> v2:172.21.15.177:3300/0 conn(0x7fe5c0071040 msgr2=0x7fe5c0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 --2- 172.21.15.153:0/3858610467 >> v2:172.21.15.177:3300/0 conn(0x7fe5c0071040 0x7fe5c0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 -- 172.21.15.153:0/3858610467 >> v1:172.21.15.177:6789/0 conn(0x7fe5c006cdc0 legacy=0x7fe5c006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 -- 172.21.15.153:0/3858610467 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c015d340 con 0x7fe5c006dee0 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bffff700 1 --2- 172.21.15.153:0/3858610467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006dee0 0x7fe5c006e330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00d0f50 tx=0x7fe5ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9278c43a237d0b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bf7fe700 1 -- 172.21.15.153:0/3858610467 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac011830 con 0x7fe5c006dee0 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bf7fe700 1 -- 172.21.15.153:0/3858610467 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac011990 con 0x7fe5c006dee0 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.329+0000 7fe5bf7fe700 1 -- 172.21.15.153:0/3858610467 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac011d20 con 0x7fe5c006dee0 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/3858610467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006dee0 msgr2=0x7fe5c006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/3858610467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006dee0 0x7fe5c006e330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00d0f50 tx=0x7fe5ac009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/3858610467 shutdown_connections 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/3858610467 >> v2:172.21.15.177:3300/0 conn(0x7fe5c0071040 0x7fe5c0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/3858610467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006dee0 0x7fe5c006e330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/3858610467 >> 172.21.15.153:0/3858610467 conn(0x7fe5c00d0000 msgr2=0x7fe5c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/3858610467 shutdown_connections 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.330+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/3858610467 wait complete. 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.332+0000 7fe5c6ea7700 1 Processor -- start 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.332+0000 7fe5c6ea7700 1 -- start start 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006cdc0 0x7fe5c00b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c006dee0 0x7fe5c014a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0071040 0x7fe5c00ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00ae4f0 con 0x7fe5c006cdc0 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c014a7e0 con 0x7fe5c0071040 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00ad8e0 con 0x7fe5c006dee0 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006cdc0 0x7fe5c00b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006cdc0 0x7fe5c00b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38766/0 (socket says 172.21.15.153:38766) 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 -- 172.21.15.153:0/1832717037 learned_addr learned my addr 172.21.15.153:0/1832717037 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c5444700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0071040 0x7fe5c00ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 -- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c006dee0 msgr2=0x7fe5c014a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5bffff700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c006dee0 0x7fe5c014a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c006dee0 0x7fe5c014a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 -- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0071040 msgr2=0x7fe5c00ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0071040 0x7fe5c00ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 -- 172.21.15.153:0/1832717037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac009550 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c5444700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0071040 0x7fe5c00ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c4c43700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006cdc0 0x7fe5c00b40c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e870 tx=0x7fe5b400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5bd7fa700 1 -- 172.21.15.153:0/1832717037 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4009590 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5bd7fa700 1 -- 172.21.15.153:0/1832717037 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b400ee70 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c015d900 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5bd7fa700 1 -- 172.21.15.153:0/1832717037 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400ca50 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.333+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c015d620 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.334+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c006e870 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.335+0000 7fe5bd7fa700 1 -- 172.21.15.153:0/1832717037 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe5b4019030 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.335+0000 7fe5bd7fa700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5a40682f0 0x7fe5a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.335+0000 7fe5bd7fa700 1 -- 172.21.15.153:0/1832717037 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5b4085ad0 con 0x7fe5c006cdc0 2023-11-28T13:00:30.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.335+0000 7fe5bffff700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5a40682f0 0x7fe5a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.340 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.335+0000 7fe5bffff700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5a40682f0 0x7fe5a406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac014040 tx=0x7fe5ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.339+0000 7fe5bd7fa700 1 -- 172.21.15.153:0/1832717037 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5b4052330 con 0x7fe5c006cdc0 2023-11-28T13:00:30.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.524+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5c00720a0 con 0x7fe5c006cdc0 2023-11-28T13:00:30.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.525+0000 7fe5bd7fa700 1 -- 172.21.15.153:0/1832717037 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5b4017020 con 0x7fe5c006cdc0 2023-11-28T13:00:30.527 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:30.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5a40682f0 msgr2=0x7fe5a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5a40682f0 0x7fe5a406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac014040 tx=0x7fe5ac008030 comp rx=0 tx=0).stop 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006cdc0 msgr2=0x7fe5c00b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006cdc0 0x7fe5c00b40c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e870 tx=0x7fe5b400ec30 comp rx=0 tx=0).stop 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 shutdown_connections 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5a40682f0 0x7fe5a406a7a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5c006cdc0 0x7fe5c00b40c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0071040 0x7fe5c00ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 --2- 172.21.15.153:0/1832717037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c006dee0 0x7fe5c014a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 >> 172.21.15.153:0/1832717037 conn(0x7fe5c00d0000 msgr2=0x7fe5c0157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 shutdown_connections 2023-11-28T13:00:30.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.527+0000 7fe5c6ea7700 1 -- 172.21.15.153:0/1832717037 wait complete. 2023-11-28T13:00:30.541 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:30.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.661+0000 7f1afcd36700 1 Processor -- start 2023-11-28T13:00:30.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.661+0000 7f1afcd36700 1 -- start start 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.662+0000 7f1afcd36700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1af806ec90 0x7f1af8065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.662+0000 7f1afcd36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80cec70 0x7f1af80ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.662+0000 7f1afcd36700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af806e240 con 0x7f1af80c9f10 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.662+0000 7f1afcd36700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af8066580 con 0x7f1af806ec90 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.662+0000 7f1afcd36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80cded0 con 0x7f1af80cec70 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.662+0000 7f1af6d9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1af80c9f10 0x7f1af815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56940/0 (socket says 172.21.15.153:56940) 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.662+0000 7f1af6d9d700 1 -- 172.21.15.153:0/415280878 learned_addr learned my addr 172.21.15.153:0/415280878 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af659c700 1 --2- 172.21.15.153:0/415280878 >> v2:172.21.15.177:3300/0 conn(0x7f1af806ec90 0x7f1af8065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af5d9b700 1 --2- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80cec70 0x7f1af80ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af559a700 1 -- 172.21.15.153:0/415280878 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828703508 0 0) 0x7f1af0001490 con 0x7f1af80c9f10 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af559a700 1 -- 172.21.15.153:0/415280878 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae00035b0 con 0x7f1af80c9f10 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af659c700 1 -- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80cec70 msgr2=0x7f1af80ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af659c700 1 --2- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80cec70 0x7f1af80ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af659c700 1 -- 172.21.15.153:0/415280878 >> v1:172.21.15.153:6789/0 conn(0x7f1af80c9f10 legacy=0x7f1af815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af659c700 1 -- 172.21.15.153:0/415280878 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80ce1b0 con 0x7f1af806ec90 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af659c700 1 --2- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 0x7f1af8065ef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009840 tx=0x7f1ae8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d219127d938315 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af559a700 1 -- 172.21.15.153:0/415280878 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8011830 con 0x7f1af806ec90 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af559a700 1 -- 172.21.15.153:0/415280878 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae8011990 con 0x7f1af806ec90 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.663+0000 7f1af559a700 1 -- 172.21.15.153:0/415280878 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8011d20 con 0x7f1af806ec90 2023-11-28T13:00:30.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 -- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 msgr2=0x7f1af8065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 --2- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 0x7f1af8065ef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009840 tx=0x7f1ae8009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 -- 172.21.15.153:0/415280878 shutdown_connections 2023-11-28T13:00:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 --2- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 0x7f1af8065ef0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 --2- 172.21.15.153:0/415280878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80cec70 0x7f1af80ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 -- 172.21.15.153:0/415280878 >> 172.21.15.153:0/415280878 conn(0x7f1af80b12b0 msgr2=0x7f1af80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 -- 172.21.15.153:0/415280878 shutdown_connections 2023-11-28T13:00:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.664+0000 7f1afcd36700 1 -- 172.21.15.153:0/415280878 wait complete. 2023-11-28T13:00:30.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.665+0000 7f1afcd36700 1 Processor -- start 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.665+0000 7f1afcd36700 1 -- start start 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1afcd36700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 0x7f1af81524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1afcd36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80c9f10 0x7f1af814cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1afcd36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1af80cec70 0x7f1af80c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1afcd36700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80c5ea0 con 0x7f1af80cec70 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1afcd36700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af8157220 con 0x7f1af806ec90 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1afcd36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af8157500 con 0x7f1af80c9f10 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1af80cec70 0x7f1af80c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1af80cec70 0x7f1af80c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38780/0 (socket says 172.21.15.153:38780) 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 -- 172.21.15.153:0/4009396991 learned_addr learned my addr 172.21.15.153:0/4009396991 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:30.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af659c700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 0x7f1af81524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af5d9b700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80c9f10 0x7f1af814cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 -- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80c9f10 msgr2=0x7f1af814cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80c9f10 0x7f1af814cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 -- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 msgr2=0x7f1af81524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 0x7f1af81524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 -- 172.21.15.153:0/4009396991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80094a0 con 0x7f1af80cec70 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af5d9b700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80c9f10 0x7f1af814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.666+0000 7f1af6d9d700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1af80cec70 0x7f1af80c8a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1af000cb40 tx=0x7f1af000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.667+0000 7f1ae6ffd700 1 -- 172.21.15.153:0/4009396991 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af000aaa0 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.667+0000 7f1ae6ffd700 1 -- 172.21.15.153:0/4009396991 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af000ac00 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.667+0000 7f1ae6ffd700 1 -- 172.21.15.153:0/4009396991 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0012400 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.667+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af80ce770 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.667+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af80ce490 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.668+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af80af140 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.668+0000 7f1ae6ffd700 1 -- 172.21.15.153:0/4009396991 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1af0019030 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.668+0000 7f1ae6ffd700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1ad80682f0 0x7f1ad806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.668+0000 7f1ae6ffd700 1 -- 172.21.15.153:0/4009396991 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1af00856e0 con 0x7f1af80cec70 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.668+0000 7f1af659c700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1ad80682f0 0x7f1ad806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.669+0000 7f1af659c700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1ad80682f0 0x7f1ad806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8000c00 tx=0x7f1ae801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.673+0000 7f1ae6ffd700 1 -- 172.21.15.153:0/4009396991 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1af0051f40 con 0x7f1af80cec70 2023-11-28T13:00:30.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.853+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1af806e240 con 0x7f1af80cec70 2023-11-28T13:00:30.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.853+0000 7f1ae6ffd700 1 -- 172.21.15.153:0/4009396991 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1af806e240 con 0x7f1af80cec70 2023-11-28T13:00:30.855 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:30.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1ad80682f0 msgr2=0x7f1ad806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1ad80682f0 0x7f1ad806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8000c00 tx=0x7f1ae801c040 comp rx=0 tx=0).stop 2023-11-28T13:00:30.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1af80cec70 msgr2=0x7f1af80c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1af80cec70 0x7f1af80c8a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1af000cb40 tx=0x7f1af000ce50 comp rx=0 tx=0).stop 2023-11-28T13:00:30.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 shutdown_connections 2023-11-28T13:00:30.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1ad80682f0 0x7f1ad806a7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1af80cec70 0x7f1af80c8a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af806ec90 0x7f1af81524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 --2- 172.21.15.153:0/4009396991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af80c9f10 0x7f1af814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 >> 172.21.15.153:0/4009396991 conn(0x7f1af80b12b0 msgr2=0x7f1af80b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 shutdown_connections 2023-11-28T13:00:30.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.855+0000 7f1afcd36700 1 -- 172.21.15.153:0/4009396991 wait complete. 2023-11-28T13:00:30.869 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:30.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.990+0000 7f34e83ac700 1 Processor -- start 2023-11-28T13:00:30.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.990+0000 7f34e83ac700 1 -- start start 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.991+0000 7f34e83ac700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34e00cdbd0 0x7f34e0071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e83ac700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e0071980 con 0x7f34e0071580 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e83ac700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e0155240 con 0x7f34e0154e70 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e83ac700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e00ce050 con 0x7f34e00cdbd0 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5947700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34e00cdbd0 0x7f34e0071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e6949700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f34e0071580 0x7f34e015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:56954/0 (socket says 172.21.15.153:56954) 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5947700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f34e00cdbd0 0x7f34e0071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38786/0 (socket says 172.21.15.153:38786) 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e6949700 1 -- 172.21.15.153:0/1943255649 learned_addr learned my addr 172.21.15.153:0/1943255649 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5947700 1 -- 172.21.15.153:0/1943255649 >> v1:172.21.15.177:6789/0 conn(0x7f34e0154e70 legacy=0x7f34e00cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:30.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5947700 1 -- 172.21.15.153:0/1943255649 >> v1:172.21.15.153:6789/0 conn(0x7f34e0071580 legacy=0x7f34e015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5146700 1 -- 172.21.15.153:0/1943255649 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038399866 0 0) 0x7f34e0071980 con 0x7f34e0071580 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5947700 1 -- 172.21.15.153:0/1943255649 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e015b100 con 0x7f34e00cdbd0 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5947700 1 --2- 172.21.15.153:0/1943255649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e00cdbd0 0x7f34e0071040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009840 tx=0x7f34dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b254f4248580042a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5146700 1 -- 172.21.15.153:0/1943255649 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc004230 con 0x7f34e00cdbd0 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5146700 1 -- 172.21.15.153:0/1943255649 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc004390 con 0x7f34e00cdbd0 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.992+0000 7f34e5146700 1 -- 172.21.15.153:0/1943255649 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0036a0 con 0x7f34e00cdbd0 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.993+0000 7f34e83ac700 1 -- 172.21.15.153:0/1943255649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e00cdbd0 msgr2=0x7f34e0071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.993+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1943255649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e00cdbd0 0x7f34e0071040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009840 tx=0x7f34dc009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.993+0000 7f34e83ac700 1 -- 172.21.15.153:0/1943255649 shutdown_connections 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.993+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1943255649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e00cdbd0 0x7f34e0071040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.993+0000 7f34e83ac700 1 -- 172.21.15.153:0/1943255649 >> 172.21.15.153:0/1943255649 conn(0x7f34e00d0000 msgr2=0x7f34e00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.993+0000 7f34e83ac700 1 -- 172.21.15.153:0/1943255649 shutdown_connections 2023-11-28T13:00:30.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.993+0000 7f34e83ac700 1 -- 172.21.15.153:0/1943255649 wait complete. 2023-11-28T13:00:30.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.994+0000 7f34e83ac700 1 Processor -- start 2023-11-28T13:00:30.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.994+0000 7f34e83ac700 1 -- start start 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e83ac700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e0071580 0x7f34e015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e83ac700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34e00cdbd0 0x7f34e00c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e83ac700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34e0154e70 0x7f34e01efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e83ac700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00c54b0 con 0x7f34e0071580 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e83ac700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00b5790 con 0x7f34e0154e70 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e83ac700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e015a850 con 0x7f34e00cdbd0 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e0071580 0x7f34e015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e0071580 0x7f34e015ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38792/0 (socket says 172.21.15.153:38792) 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 -- 172.21.15.153:0/1032351532 learned_addr learned my addr 172.21.15.153:0/1032351532 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6949700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34e0154e70 0x7f34e01efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 -- 172.21.15.153:0/1032351532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34e00cdbd0 msgr2=0x7f34e00c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34e00cdbd0 0x7f34e00c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 -- 172.21.15.153:0/1032351532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34e0154e70 msgr2=0x7f34e01efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34e0154e70 0x7f34e01efe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 -- 172.21.15.153:0/1032351532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0094a0 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e6148700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e0071580 0x7f34e015ab20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b4d0 tx=0x7f34d000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34d6ffd700 1 -- 172.21.15.153:0/1032351532 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0011830 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34d6ffd700 1 -- 172.21.15.153:0/1032351532 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d0011990 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e015b6c0 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.995+0000 7f34d6ffd700 1 -- 172.21.15.153:0/1032351532 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0011d40 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.996+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e015b3e0 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.996+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e015b9a0 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.997+0000 7f34d6ffd700 1 -- 172.21.15.153:0/1032351532 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f34d000f050 con 0x7f34e0071580 2023-11-28T13:00:31.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.997+0000 7f34d6ffd700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34c40682f0 0x7f34c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.997+0000 7f34d6ffd700 1 -- 172.21.15.153:0/1032351532 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f34d00857e0 con 0x7f34e0071580 2023-11-28T13:00:31.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.997+0000 7f34e5947700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34c40682f0 0x7f34c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:30.998+0000 7f34e5947700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34c40682f0 0x7f34c406a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34e015dff0 tx=0x7f34dc010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.001+0000 7f34d6ffd700 1 -- 172.21.15.153:0/1032351532 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34d0051fe0 con 0x7f34e0071580 2023-11-28T13:00:31.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.183+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34e015a630 con 0x7f34e0071580 2023-11-28T13:00:31.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.183+0000 7f34d6ffd700 1 -- 172.21.15.153:0/1032351532 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34d0055600 con 0x7f34e0071580 2023-11-28T13:00:31.185 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:31.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34c40682f0 msgr2=0x7f34c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34c40682f0 0x7f34c406a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34e015dff0 tx=0x7f34dc010000 comp rx=0 tx=0).stop 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e0071580 msgr2=0x7f34e015ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e0071580 0x7f34e015ab20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b4d0 tx=0x7f34d000b890 comp rx=0 tx=0).stop 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 shutdown_connections 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f34c40682f0 0x7f34c406a7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34e0071580 0x7f34e015ab20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34e0154e70 0x7f34e01efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 --2- 172.21.15.153:0/1032351532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34e00cdbd0 0x7f34e00c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 >> 172.21.15.153:0/1032351532 conn(0x7f34e00d0000 msgr2=0x7f34e00cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.185+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 shutdown_connections 2023-11-28T13:00:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.186+0000 7f34e83ac700 1 -- 172.21.15.153:0/1032351532 wait complete. 2023-11-28T13:00:31.199 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:31.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.321+0000 7f7c9ec31700 1 Processor -- start 2023-11-28T13:00:31.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.321+0000 7f7c9ec31700 1 -- start start 2023-11-28T13:00:31.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.321+0000 7f7c9ec31700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c980cd8c0 0x7f7c98140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.321+0000 7f7c9ec31700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c9806f850 0x7f7c9806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.321+0000 7f7c9ec31700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9813ff00 con 0x7f7c980cdc90 2023-11-28T13:00:31.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.321+0000 7f7c9ec31700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c981404c0 con 0x7f7c980cd8c0 2023-11-28T13:00:31.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.321+0000 7f7c9ec31700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9815aa80 con 0x7f7c9806f850 2023-11-28T13:00:31.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c980cd8c0 0x7f7c98140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c980cd8c0 0x7f7c98140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38800/0 (socket says 172.21.15.153:38800) 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 -- 172.21.15.153:0/4219281651 learned_addr learned my addr 172.21.15.153:0/4219281651 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9d1ce700 1 --2- 172.21.15.153:0/4219281651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c9806f850 0x7f7c9806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 -- 172.21.15.153:0/4219281651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c9806f850 msgr2=0x7f7c9806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 --2- 172.21.15.153:0/4219281651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c9806f850 0x7f7c9806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 -- 172.21.15.153:0/4219281651 >> v1:172.21.15.177:6789/0 conn(0x7f7c980cdc90 legacy=0x7f7c980b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 -- 172.21.15.153:0/4219281651 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980b3040 con 0x7f7c980cd8c0 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c977fe700 1 -- 172.21.15.153:0/4219281651 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253675637 0 0) 0x7f7c9813ff00 con 0x7f7c980cdc90 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c9c9cd700 1 --2- 172.21.15.153:0/4219281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cd8c0 0x7f7c98140950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fd0 tx=0x7f7c8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160f004d8d7e172e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c977fe700 1 -- 172.21.15.153:0/4219281651 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c005e30 con 0x7f7c980cd8c0 2023-11-28T13:00:31.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c977fe700 1 -- 172.21.15.153:0/4219281651 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c004030 con 0x7f7c980cd8c0 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.322+0000 7f7c977fe700 1 -- 172.21.15.153:0/4219281651 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0043e0 con 0x7f7c980cd8c0 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 -- 172.21.15.153:0/4219281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cd8c0 msgr2=0x7f7c98140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/4219281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cd8c0 0x7f7c98140950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fd0 tx=0x7f7c8c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 -- 172.21.15.153:0/4219281651 shutdown_connections 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/4219281651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c9806f850 0x7f7c9806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/4219281651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cd8c0 0x7f7c98140950 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 -- 172.21.15.153:0/4219281651 >> 172.21.15.153:0/4219281651 conn(0x7f7c9806aed0 msgr2=0x7f7c9806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 -- 172.21.15.153:0/4219281651 shutdown_connections 2023-11-28T13:00:31.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.323+0000 7f7c9ec31700 1 -- 172.21.15.153:0/4219281651 wait complete. 2023-11-28T13:00:31.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.324+0000 7f7c9ec31700 1 Processor -- start 2023-11-28T13:00:31.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.324+0000 7f7c9ec31700 1 -- start start 2023-11-28T13:00:31.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9ec31700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c9806f850 0x7f7c980c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9ec31700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c980cd8c0 0x7f7c98142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9ec31700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cdc90 0x7f7c98142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9ec31700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980b49f0 con 0x7f7c980cdc90 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9ec31700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980b4cd0 con 0x7f7c9806f850 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9ec31700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980b4fb0 con 0x7f7c980cd8c0 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cdc90 0x7f7c98142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cdc90 0x7f7c98142bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38810/0 (socket says 172.21.15.153:38810) 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 -- 172.21.15.153:0/99234111 learned_addr learned my addr 172.21.15.153:0/99234111 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:31.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9c9cd700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c9806f850 0x7f7c980c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c97fff700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c980cd8c0 0x7f7c98142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 -- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c980cd8c0 msgr2=0x7f7c98142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c980cd8c0 0x7f7c98142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 -- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c9806f850 msgr2=0x7f7c980c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c9806f850 0x7f7c980c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 -- 172.21.15.153:0/99234111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c009500 con 0x7f7c980cdc90 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c97fff700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c980cd8c0 0x7f7c98142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.325+0000 7f7c9d1ce700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cdc90 0x7f7c98142bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000efc0 tx=0x7f7c9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.326+0000 7f7c957fa700 1 -- 172.21.15.153:0/99234111 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90009ea0 con 0x7f7c980cdc90 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.326+0000 7f7c957fa700 1 -- 172.21.15.153:0/99234111 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9000f040 con 0x7f7c980cdc90 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.326+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c980b35f0 con 0x7f7c980cdc90 2023-11-28T13:00:31.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.326+0000 7f7c957fa700 1 -- 172.21.15.153:0/99234111 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90014480 con 0x7f7c980cdc90 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.326+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c980b3310 con 0x7f7c980cdc90 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.326+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c98140150 con 0x7f7c980cdc90 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.327+0000 7f7c957fa700 1 -- 172.21.15.153:0/99234111 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7c900145e0 con 0x7f7c980cdc90 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.327+0000 7f7c957fa700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c7c0682f0 0x7f7c7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.327+0000 7f7c957fa700 1 -- 172.21.15.153:0/99234111 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7c90086600 con 0x7f7c980cdc90 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.327+0000 7f7c9c9cd700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c7c0682f0 0x7f7c7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.328+0000 7f7c9c9cd700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c7c0682f0 0x7f7c7c06a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fd0 tx=0x7f7c8c008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.332+0000 7f7c957fa700 1 -- 172.21.15.153:0/99234111 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7c90052e60 con 0x7f7c980cdc90 2023-11-28T13:00:31.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.516+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c98068690 con 0x7f7c980cdc90 2023-11-28T13:00:31.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.517+0000 7f7c957fa700 1 -- 172.21.15.153:0/99234111 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c90056480 con 0x7f7c980cdc90 2023-11-28T13:00:31.519 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c7c0682f0 msgr2=0x7f7c7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c7c0682f0 0x7f7c7c06a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fd0 tx=0x7f7c8c008070 comp rx=0 tx=0).stop 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cdc90 msgr2=0x7f7c98142bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cdc90 0x7f7c98142bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000efc0 tx=0x7f7c9000c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 shutdown_connections 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c7c0682f0 0x7f7c7c06a7a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c980cdc90 0x7f7c98142bc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c9806f850 0x7f7c980c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 --2- 172.21.15.153:0/99234111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c980cd8c0 0x7f7c98142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 >> 172.21.15.153:0/99234111 conn(0x7f7c9806aed0 msgr2=0x7f7c980cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 shutdown_connections 2023-11-28T13:00:31.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.519+0000 7f7c9ec31700 1 -- 172.21.15.153:0/99234111 wait complete. 2023-11-28T13:00:31.533 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:31.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.651+0000 7fae4e068700 1 Processor -- start 2023-11-28T13:00:31.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.651+0000 7fae4e068700 1 -- start start 2023-11-28T13:00:31.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae4e068700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fae4806ba90 0x7fae480af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae4e068700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fae480cbdf0 0x7fae480ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae4e068700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae480cc5f0 con 0x7fae4815cef0 2023-11-28T13:00:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae4e068700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4815d470 con 0x7fae480cbdf0 2023-11-28T13:00:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae4e068700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4815d750 con 0x7fae4806ba90 2023-11-28T13:00:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae47fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fae480cbdf0 0x7fae480ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae47fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fae480cbdf0 0x7fae480ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38818/0 (socket says 172.21.15.153:38818) 2023-11-28T13:00:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.652+0000 7fae47fff700 1 -- 172.21.15.153:0/1518082547 learned_addr learned my addr 172.21.15.153:0/1518082547 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae47fff700 1 -- 172.21.15.153:0/1518082547 >> v2:172.21.15.177:3300/0 conn(0x7fae4806ba90 msgr2=0x7fae480af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae47fff700 1 --2- 172.21.15.153:0/1518082547 >> v2:172.21.15.177:3300/0 conn(0x7fae4806ba90 0x7fae480af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae47fff700 1 -- 172.21.15.153:0/1518082547 >> v1:172.21.15.177:6789/0 conn(0x7fae4815cef0 legacy=0x7fae4806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae47fff700 1 -- 172.21.15.153:0/1518082547 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4815da30 con 0x7fae480cbdf0 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae47fff700 1 --2- 172.21.15.153:0/1518082547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480ca650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae4806fbb0 tx=0x7fae4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75cd970e41f6d1f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae46ffd700 1 -- 172.21.15.153:0/1518082547 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4000e070 con 0x7fae480cbdf0 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae46ffd700 1 -- 172.21.15.153:0/1518082547 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40003a20 con 0x7fae480cbdf0 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae46ffd700 1 -- 172.21.15.153:0/1518082547 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40003dd0 con 0x7fae480cbdf0 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae4e068700 1 -- 172.21.15.153:0/1518082547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 msgr2=0x7fae480ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae4e068700 1 --2- 172.21.15.153:0/1518082547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480ca650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae4806fbb0 tx=0x7fae4000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae4e068700 1 -- 172.21.15.153:0/1518082547 shutdown_connections 2023-11-28T13:00:31.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae4e068700 1 --2- 172.21.15.153:0/1518082547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480ca650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae4e068700 1 --2- 172.21.15.153:0/1518082547 >> v2:172.21.15.177:3300/0 conn(0x7fae4806ba90 0x7fae480af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.653+0000 7fae4e068700 1 -- 172.21.15.153:0/1518082547 >> 172.21.15.153:0/1518082547 conn(0x7fae480d0000 msgr2=0x7fae480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.654+0000 7fae4e068700 1 -- 172.21.15.153:0/1518082547 shutdown_connections 2023-11-28T13:00:31.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.654+0000 7fae4e068700 1 -- 172.21.15.153:0/1518082547 wait complete. 2023-11-28T13:00:31.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 Processor -- start 2023-11-28T13:00:31.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 -- start start 2023-11-28T13:00:31.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae4806ba90 0x7fae48153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae4815cef0 0x7fae480c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae480b36c0 con 0x7fae480cbdf0 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae480c9b10 con 0x7fae4806ba90 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.655+0000 7fae4e068700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae480c9df0 con 0x7fae4815cef0 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38822/0 (socket says 172.21.15.153:38822) 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 -- 172.21.15.153:0/3721445557 learned_addr learned my addr 172.21.15.153:0/3721445557 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae477fe700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae4806ba90 0x7fae48153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae47fff700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae4815cef0 0x7fae480c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 -- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae4815cef0 msgr2=0x7fae480c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae4815cef0 0x7fae480c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 -- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae4806ba90 msgr2=0x7fae48153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae4806ba90 0x7fae48153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 -- 172.21.15.153:0/3721445557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40009b30 con 0x7fae480cbdf0 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae3ffff700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480c92c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b530 tx=0x7fae3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae44ff9700 1 -- 172.21.15.153:0/3721445557 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30011830 con 0x7fae480cbdf0 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae44ff9700 1 -- 172.21.15.153:0/3721445557 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30011990 con 0x7fae480cbdf0 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4815dff0 con 0x7fae480cbdf0 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae44ff9700 1 -- 172.21.15.153:0/3721445557 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30011d40 con 0x7fae480cbdf0 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.656+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4815dd10 con 0x7fae480cbdf0 2023-11-28T13:00:31.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.657+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae480cab90 con 0x7fae480cbdf0 2023-11-28T13:00:31.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.658+0000 7fae44ff9700 1 -- 172.21.15.153:0/3721445557 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fae30021070 con 0x7fae480cbdf0 2023-11-28T13:00:31.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.658+0000 7fae44ff9700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fae280682f0 0x7fae2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.658+0000 7fae44ff9700 1 -- 172.21.15.153:0/3721445557 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae30085fa0 con 0x7fae480cbdf0 2023-11-28T13:00:31.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.658+0000 7fae477fe700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fae280682f0 0x7fae2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.658+0000 7fae477fe700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fae280682f0 0x7fae2806a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae48159ba0 tx=0x7fae380090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.662+0000 7fae44ff9700 1 -- 172.21.15.153:0/3721445557 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae30052960 con 0x7fae480cbdf0 2023-11-28T13:00:31.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.887+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae48072000 con 0x7fae480cbdf0 2023-11-28T13:00:31.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.887+0000 7fae44ff9700 1 -- 172.21.15.153:0/3721445557 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7fae30055f80 con 0x7fae480cbdf0 2023-11-28T13:00:31.889 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:31.889 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2499459281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2499459281},{"type":"v1","addr":"172.21.15.177:6834","nonce":2499459281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2708083457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2708083457},{"type":"v1","addr":"172.21.15.153:6837","nonce":2708083457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2730194389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2730194389},{"type":"v1","addr":"172.21.15.153:6836","nonce":2730194389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:17.624622+0000","modified":"2023-11-28T13:00:20.642491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/923823631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":923823631},{"type":"v1","addr":"172.21.15.177:6835","nonce":923823631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fae280682f0 msgr2=0x7fae2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fae280682f0 0x7fae2806a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae48159ba0 tx=0x7fae380090e0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 msgr2=0x7fae480c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480c92c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae3000b530 tx=0x7fae3000b840 comp rx=0 tx=0).stop 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 shutdown_connections 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae4815cef0 0x7fae480c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae4806ba90 0x7fae48153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fae280682f0 0x7fae2806a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 --2- 172.21.15.153:0/3721445557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae480cbdf0 0x7fae480c92c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 >> 172.21.15.153:0/3721445557 conn(0x7fae480d0000 msgr2=0x7fae480af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 shutdown_connections 2023-11-28T13:00:31.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:31.889+0000 7fae4e068700 1 -- 172.21.15.153:0/3721445557 wait complete. 2023-11-28T13:00:31.893 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 8 2023-11-28T13:00:32.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:00:17.624622+0000', 'modified': '2023-11-28T13:00:20.642491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/923823631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6833', 'nonce': 923823631}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 923823631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:00:32.306 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:00:32.306 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T13:00:32.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.427+0000 7fb4eb4ef700 1 Processor -- start 2023-11-28T13:00:32.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.428+0000 7fb4eb4ef700 1 -- start start 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.428+0000 7fb4eb4ef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40ce040 0x7fb4e415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.428+0000 7fb4eb4ef700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb4e40d0a00 0x7fb4e406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.428+0000 7fb4eb4ef700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e406b420 con 0x7fb4e40b2930 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.428+0000 7fb4eb4ef700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e406b700 con 0x7fb4e40d0a00 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.428+0000 7fb4eb4ef700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e406b9e0 con 0x7fb4e40ce040 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.428+0000 7fb4e8a8a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb4e40d0a00 0x7fb4e406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e8a8a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb4e40d0a00 0x7fb4e406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38828/0 (socket says 172.21.15.153:38828) 2023-11-28T13:00:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/2753706720 learned_addr learned my addr 172.21.15.153:0/2753706720 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e928b700 1 --2- 172.21.15.153:0/2753706720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40ce040 0x7fb4e415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/2753706720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40ce040 msgr2=0x7fb4e415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/2753706720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40ce040 0x7fb4e415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/2753706720 >> v1:172.21.15.177:6789/0 conn(0x7fb4e40b2930 legacy=0x7fb4e406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/2753706720 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e406bcc0 con 0x7fb4e40d0a00 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/2753706720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e406f850 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00098f0 tx=0x7fb4e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61b2e4aec8e7a41 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4dbfff700 1 -- 172.21.15.153:0/2753706720 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000b8a0 con 0x7fb4e40d0a00 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4dbfff700 1 -- 172.21.15.153:0/2753706720 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000ba00 con 0x7fb4e40d0a00 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.429+0000 7fb4dbfff700 1 -- 172.21.15.153:0/2753706720 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000bd90 con 0x7fb4e40d0a00 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2753706720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 msgr2=0x7fb4e406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2753706720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e406f850 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00098f0 tx=0x7fb4e0009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:32.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2753706720 shutdown_connections 2023-11-28T13:00:32.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2753706720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40ce040 0x7fb4e415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2753706720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e406f850 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2753706720 >> 172.21.15.153:0/2753706720 conn(0x7fb4e406aed0 msgr2=0x7fb4e406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2753706720 shutdown_connections 2023-11-28T13:00:32.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.430+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2753706720 wait complete. 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.431+0000 7fb4eb4ef700 1 Processor -- start 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.431+0000 7fb4eb4ef700 1 -- start start 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4eb4ef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40b2930 0x7fb4e414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4eb4ef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4e40ce040 0x7fb4e40b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4eb4ef700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e40c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4eb4ef700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40c9020 con 0x7fb4e40d0a00 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4eb4ef700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40c8410 con 0x7fb4e40ce040 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4eb4ef700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40c86f0 con 0x7fb4e40b2930 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e40c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e40c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38836/0 (socket says 172.21.15.153:38836) 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 -- 172.21.15.153:0/2425394193 learned_addr learned my addr 172.21.15.153:0/2425394193 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4e40ce040 0x7fb4e40b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e928b700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40b2930 0x7fb4e414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 -- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40b2930 msgr2=0x7fb4e414a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40b2930 0x7fb4e414a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 -- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4e40ce040 msgr2=0x7fb4e40b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4e40ce040 0x7fb4e40b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 -- 172.21.15.153:0/2425394193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009550 con 0x7fb4e40d0a00 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4e9a8c700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e40c4d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00cb50 tx=0x7fb4dc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4d9ffb700 1 -- 172.21.15.153:0/2425394193 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc003c00 con 0x7fb4e40d0a00 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4d9ffb700 1 -- 172.21.15.153:0/2425394193 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc003d60 con 0x7fb4e40d0a00 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4d9ffb700 1 -- 172.21.15.153:0/2425394193 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00aa20 con 0x7fb4e40d0a00 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.432+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e406fd90 con 0x7fb4e40d0a00 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.433+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e406bfa0 con 0x7fb4e40d0a00 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.434+0000 7fb4d9ffb700 1 -- 172.21.15.153:0/2425394193 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb4dc010050 con 0x7fb4e40d0a00 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.434+0000 7fb4d9ffb700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4c80682f0 0x7fb4c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.434+0000 7fb4d9ffb700 1 -- 172.21.15.153:0/2425394193 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4dc085b20 con 0x7fb4e40d0a00 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.434+0000 7fb4e928b700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4c80682f0 0x7fb4c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.435+0000 7fb4e928b700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4c80682f0 0x7fb4c806a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4006fd0 tx=0x7fb4d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.436+0000 7fb4f7678b80 1 Processor -- start 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4f7678b80 1 -- start start 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9c9130 0x55c5fa9cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa88bbc0 con 0x55c5fa9cb810 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa8fa350 con 0x55c5fa9c9130 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa8f3880 con 0x55c5fa9cc120 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4e928b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4e928b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38840/0 (socket says 172.21.15.153:38840) 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.437+0000 7fb4e928b700 1 -- 172.21.15.153:0/4222086064 learned_addr learned my addr 172.21.15.153:0/4222086064 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9c9130 0x55c5fa9cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4e928b700 1 -- 172.21.15.153:0/4222086064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 msgr2=0x55c5fa9e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4e928b700 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4e928b700 1 -- 172.21.15.153:0/4222086064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9c9130 msgr2=0x55c5fa9cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4e928b700 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9c9130 0x55c5fa9cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4e928b700 1 -- 172.21.15.153:0/4222086064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5fa7755e0 con 0x55c5fa9cb810 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4e928b700 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9a1a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4010990 tx=0x7fb4d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1599d103746b8d52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4c6ffd700 1 -- 172.21.15.153:0/4222086064 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4005a20 con 0x55c5fa9cb810 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4c6ffd700 1 -- 172.21.15.153:0/4222086064 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d401a070 con 0x55c5fa9cb810 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.438+0000 7fb4c6ffd700 1 -- 172.21.15.153:0/4222086064 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4005dc0 con 0x55c5fa9cb810 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 -- 172.21.15.153:0/4222086064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 msgr2=0x55c5fa9a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9a1a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4010990 tx=0x7fb4d4013000 comp rx=0 tx=0).stop 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 -- 172.21.15.153:0/4222086064 shutdown_connections 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9a1a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/4222086064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9c9130 0x55c5fa9cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 -- 172.21.15.153:0/4222086064 >> 172.21.15.153:0/4222086064 conn(0x55c5fa99ed60 msgr2=0x55c5fa99f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 -- 172.21.15.153:0/4222086064 shutdown_connections 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 -- 172.21.15.153:0/4222086064 wait complete. 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 -- start start 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.439+0000 7fb4f7678b80 1 Processor -- start 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9c9130 0x55c5fa9dd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9c1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cc120 0x55c5fa9c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa7778d0 con 0x55c5fa9cb810 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa9ddd40 con 0x55c5fa9cc120 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa9becb0 con 0x55c5fa9c9130 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4f7678b80 10 client.0 _ll_register_callbacks cb 0x55c5fa799240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4e8a8a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4e8a8a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9c1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38850/0 (socket says 172.21.15.153:38850) 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.440+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/3509406030 learned_addr learned my addr 172.21.15.153:0/3509406030 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9c9130 0x55c5fa9dd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e9a8c700 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cc120 0x55c5fa9c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9c9130 msgr2=0x55c5fa9dd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9c9130 0x55c5fa9dd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cc120 msgr2=0x55c5fa9c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cc120 0x55c5fa9c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e8a8a700 1 -- 172.21.15.153:0/3509406030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d400e040 con 0x55c5fa9cb810 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9c1110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0013000 tx=0x7fb4e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d20214f5c57bc32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4bffff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4bffff700 1 -- 172.21.15.153:0/3509406030 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0021070 con 0x55c5fa9cb810 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4bffff700 1 -- 172.21.15.153:0/3509406030 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4e0004040 con 0x55c5fa9cb810 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5fa8c1140 con 0x55c5fa9cb810 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4bffff700 1 -- 172.21.15.153:0/3509406030 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4e0004240 con 0x55c5fa9cb810 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 10 client.4617 fetch_fsmap learned FSMap version 8 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c5fa721d90 con 0x55c5fa9cb810 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4bffff700 1 -- 172.21.15.153:0/3509406030 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c5fa8c1140 con 0x55c5fa9cb810 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4bffff700 1 -- 172.21.15.153:0/3509406030 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb4e00043a0 con 0x55c5fa9cb810 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 10 client.4617 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 10 client.4617 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 20 client.4617 populate_metadata read hostname 'smithi153' 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9bc0e0 0x55c5fa9be720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 4 client.4617 mds_command: new command op to 4300 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.441+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 --> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5fa777f10 con 0x55c5fa9bc0e0 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.442+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9bc0e0 0x55c5fa9be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.442+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9bc0e0 0x55c5fa9be720 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ff38346d8c94fb server_cookie=d2a9e434d4e67ee6 in_seq=0 out_seq=0 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.442+0000 7fb4bffff700 10 client.4617 ms_handle_connect on v2:172.21.15.177:6833/923823631 2023-11-28T13:00:32.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4bffff700 1 -- 172.21.15.153:0/3509406030 <== mds.0 v2:172.21.15.177:6833/923823631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55c5fa777f10 con 0x55c5fa9bc0e0 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4bffff700 10 client.4617 handle_command_reply: tid=0 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 client.4617 shutdown 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 msgr2=0x55c5fa9c1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9c1110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0013000 tx=0x7fb4e00047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 shutdown_connections 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cc120 0x55c5fa9c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9c9130 0x55c5fa9dd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9bc0e0 0x55c5fa9be720 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3509406030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9cb810 0x55c5fa9c1110 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4bffff700 0 client.4617 ms_handle_reset on v2:172.21.15.177:6833/923823631 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 >> 172.21.15.153:0/3509406030 conn(0x55c5fa99ed60 msgr2=0x55c5fa9a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 shutdown_connections 2023-11-28T13:00:32.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3509406030 wait complete. 2023-11-28T13:00:32.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.443+0000 7fb4f7678b80 20 client.4617 trim_cache size 0 max 16384 2023-11-28T13:00:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.460+0000 7fb4f7678b80 1 Processor -- start 2023-11-28T13:00:32.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.460+0000 7fb4f7678b80 1 -- start start 2023-11-28T13:00:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa99c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 0x55c5fa99b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa99a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa9c8d80 con 0x55c5fa9c9130 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa9dd600 con 0x55c5fa9cb810 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa9e0020 con 0x55c5fa9cc120 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa99c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa99c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38854/0 (socket says 172.21.15.153:38854) 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 -- 172.21.15.153:0/3601894043 learned_addr learned my addr 172.21.15.153:0/3601894043 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 -- 172.21.15.153:0/3601894043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 msgr2=0x55c5fa99a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3601894043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa99a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 -- 172.21.15.153:0/3601894043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 msgr2=0x55c5fa99b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3601894043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 0x55c5fa99b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 -- 172.21.15.153:0/3601894043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5fa8ba140 con 0x55c5fa9c9130 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.461+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3601894043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa99c650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4010990 tx=0x7fb4d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937ac2a27318d3f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4c77fe700 1 -- 172.21.15.153:0/3601894043 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4005e20 con 0x55c5fa9c9130 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4c77fe700 1 -- 172.21.15.153:0/3601894043 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d401a070 con 0x55c5fa9c9130 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4c77fe700 1 -- 172.21.15.153:0/3601894043 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4016da0 con 0x55c5fa9c9130 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3601894043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 msgr2=0x55c5fa99c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3601894043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa99c650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4010990 tx=0x7fb4d4015000 comp rx=0 tx=0).stop 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3601894043 shutdown_connections 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3601894043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa99c650 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3601894043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa99a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3601894043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 0x55c5fa99b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3601894043 >> 172.21.15.153:0/3601894043 conn(0x55c5fa99ed60 msgr2=0x55c5fa99be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3601894043 shutdown_connections 2023-11-28T13:00:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.462+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3601894043 wait complete. 2023-11-28T13:00:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.463+0000 7fb4f7678b80 1 -- start start 2023-11-28T13:00:32.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.463+0000 7fb4f7678b80 1 Processor -- start 2023-11-28T13:00:32.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa9c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 0x55c5faa31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5faa323f0 con 0x55c5fa9c9130 2023-11-28T13:00:32.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fa9b9730 con 0x55c5fa9cb810 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5faa34ed0 con 0x55c5fa9cc120 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 10 client.0 _ll_register_callbacks cb 0x55c5fa799240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa9c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa9c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38860/0 (socket says 172.21.15.153:38860) 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 -- 172.21.15.153:0/3305634535 learned_addr learned my addr 172.21.15.153:0/3305634535 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 0x55c5faa31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e9a8c700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 -- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 msgr2=0x55c5fa9a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 -- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 msgr2=0x55c5faa31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 0x55c5faa31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 -- 172.21.15.153:0/3305634535 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5fa7755e0 con 0x55c5fa9c9130 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e9a8c700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4e928b700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa9c2d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c5fa99d0d0 tx=0x7fb4d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2990b6a8914db424 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4bffff700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4bffff700 1 -- 172.21.15.153:0/3305634535 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4032050 con 0x55c5fa9c9130 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4bffff700 1 -- 172.21.15.153:0/3305634535 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4d403a460 con 0x55c5fa9c9130 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5fa8b1160 con 0x55c5fa9c9130 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4bffff700 1 -- 172.21.15.153:0/3305634535 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d403a620 con 0x55c5fa9c9130 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 10 client.4623 fetch_fsmap learned FSMap version 8 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.464+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c5fa748200 con 0x55c5fa9c9130 2023-11-28T13:00:32.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4bffff700 1 -- 172.21.15.153:0/3305634535 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c5fa8b1160 con 0x55c5fa9c9130 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4bffff700 1 -- 172.21.15.153:0/3305634535 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb4d403a780 con 0x55c5fa9c9130 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4f7678b80 10 client.4623 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4f7678b80 10 client.4623 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4f7678b80 20 client.4623 populate_metadata read hostname 'smithi153' 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9b8da0 0x55c5fa9dde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4f7678b80 4 client.4623 mds_command: new command op to 4300 tid=0[{"prefix": "status"}] 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 --> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c5fa89b170 con 0x55c5fa9b8da0 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9b8da0 0x55c5fa9dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4bffff700 10 client.4623 ms_handle_connect on v2:172.21.15.177:6833/923823631 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4e8a8a700 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9b8da0 0x55c5fa9dde80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd428331be65796 server_cookie=7bfd23b0e071ab97 in_seq=0 out_seq=0 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4bffff700 1 -- 172.21.15.153:0/3305634535 <== mds.0 v2:172.21.15.177:6833/923823631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55c5fa89b170 con 0x55c5fa9b8da0 2023-11-28T13:00:32.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4bffff700 10 client.4623 handle_command_reply: tid=0 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.465+0000 7fb4f7678b80 1 client.4623 shutdown 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 msgr2=0x55c5fa9c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa9c2d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c5fa99d0d0 tx=0x7fb4d4017000 comp rx=0 tx=0).stop 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 shutdown_connections 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:6833/923823631,v1:172.21.15.177:6835/923823631] conn(0x55c5fa9b8da0 0x55c5fa9dde80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55c5fa9c9130 0x55c5fa9c2d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4bffff700 0 client.4623 ms_handle_reset on v2:172.21.15.177:6833/923823631 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c5fa9cc120 0x55c5fa9a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 --2- 172.21.15.153:0/3305634535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5fa9cb810 0x55c5faa31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 >> 172.21.15.153:0/3305634535 conn(0x55c5fa99ed60 msgr2=0x55c5fa9bf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 shutdown_connections 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 1 -- 172.21.15.153:0/3305634535 wait complete. 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4f7678b80 20 client.4623 trim_cache size 0 max 16384 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: "cluster_fsid": "47bfaa00-a151-4537-aa4c-32b0866eb248", 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: "whoami": 0, 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 4300, 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: "want_state": "up:active", 2023-11-28T13:00:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: "state": "up:active", 2023-11-28T13:00:32.476 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs", 2023-11-28T13:00:32.476 INFO:teuthology.orchestra.run.smithi153.stdout: "rank_uptime": 14.840015228, 2023-11-28T13:00:32.476 INFO:teuthology.orchestra.run.smithi153.stdout: "mdsmap_epoch": 8, 2023-11-28T13:00:32.476 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch": 25, 2023-11-28T13:00:32.476 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch_barrier": 25, 2023-11-28T13:00:32.476 INFO:teuthology.orchestra.run.smithi153.stdout: "uptime": 20.144397398999999 2023-11-28T13:00:32.476 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:00:32.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4c80682f0 msgr2=0x7fb4c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4c80682f0 0x7fb4c806a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4006fd0 tx=0x7fb4d4008040 comp rx=0 tx=0).stop 2023-11-28T13:00:32.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 msgr2=0x7fb4e40c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e40c4d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00cb50 tx=0x7fb4dc00ce60 comp rx=0 tx=0).stop 2023-11-28T13:00:32.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 shutdown_connections 2023-11-28T13:00:32.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4c80682f0 0x7fb4c806a7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb4e40d0a00 0x7fb4e40c4d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4e40b2930 0x7fb4e414a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 --2- 172.21.15.153:0/2425394193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4e40ce040 0x7fb4e40b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 >> 172.21.15.153:0/2425394193 conn(0x7fb4e406aed0 msgr2=0x7fb4e40b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 shutdown_connections 2023-11-28T13:00:32.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:32.466+0000 7fb4eb4ef700 1 -- 172.21.15.153:0/2425394193 wait complete. 2023-11-28T13:00:32.482 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T13:00:32.503 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T13:00:32.504 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T13:00:32.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.504 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:00:32.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.521 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-11-28T13:00:32.590 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2023-11-28T13:00:32.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:00:32.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.592 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:00:32.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.650 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-11-28T13:00:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2023-11-28T13:00:32.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:00:32.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.718 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T13:00:32.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.740 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T13:00:32.813 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-11-28T13:00:32.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:00:32.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.815 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T13:00:32.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:32.873 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T13:00:32.943 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-11-28T13:00:32.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:00:32.945 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-28T13:00:32.945 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:32.945 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:32.946 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.064+0000 7f2a19e14700 1 Processor -- start 2023-11-28T13:00:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.064+0000 7f2a19e14700 1 -- start start 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a19e14700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 0x7f2a1406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a19e14700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1406bf10 con 0x7f2a1406fb30 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a19e14700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a141590b0 con 0x7f2a14158ce0 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a19e14700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140cb0d0 con 0x7f2a140ce4b0 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a12ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2a1406fb30 0x7f2a140cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57034/0 (socket says 172.21.15.153:57034) 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a12ffd700 1 -- 172.21.15.153:0/3646194155 learned_addr learned my addr 172.21.15.153:0/3646194155 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630666808 0 0) 0x7f2a1406bf10 con 0x7f2a1406fb30 2023-11-28T13:00:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a13fff700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 0x7f2a1406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a000035b0 con 0x7f2a1406fb30 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496801822 0 0) 0x7f2a000035b0 con 0x7f2a1406fb30 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 msgr2=0x7f2a1406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a127fc700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 0x7f2a1406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 >> v1:172.21.15.177:6789/0 conn(0x7f2a14158ce0 legacy=0x7f2a1415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.065+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a140cb3b0 con 0x7f2a1406fb30 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f2a04001670 con 0x7f2a1406fb30 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 >> v1:172.21.15.153:6789/0 conn(0x7f2a1406fb30 legacy=0x7f2a140cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 0x7f2a00003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00003fb0 0x7f2a00004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140cb0d0 con 0x7f2a00004d90 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0000baf0 con 0x7f2a00003fb0 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0000dec0 con 0x7f2a140ce4b0 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a13fff700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a137fe700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 0x7f2a00003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a12ffd700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00003fb0 0x7f2a00004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a13fff700 1 -- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 msgr2=0x7f2a00003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a13fff700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140ce4b0 0x7f2a00003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a13fff700 1 -- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00003fb0 msgr2=0x7f2a00004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a13fff700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00003fb0 0x7f2a00004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a13fff700 1 -- 172.21.15.153:0/3646194155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a000035b0 con 0x7f2a00004d90 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a13fff700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2a140ce8a0 tx=0x7f2a00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa749a5270af8f62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c008e60 con 0x7f2a00004d90 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c010040 con 0x7f2a00004d90 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.066+0000 7f2a127fc700 1 -- 172.21.15.153:0/3646194155 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c014630 con 0x7f2a00004d90 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.067+0000 7f2a19e14700 1 -- 172.21.15.153:0/3646194155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 msgr2=0x7f2a00005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.067+0000 7f2a19e14700 1 --2- 172.21.15.153:0/3646194155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2a140ce8a0 tx=0x7f2a00005880 comp rx=0 tx=0).stop 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.067+0000 7f2a13fff700 1 -- 172.21.15.153:0/3646194155 reap_dead start 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.067+0000 7f2a19e14700 1 -- 172.21.15.153:0/3646194155 shutdown_connections 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.067+0000 7f2a19e14700 1 -- 172.21.15.153:0/3646194155 >> 172.21.15.153:0/3646194155 conn(0x7f2a140cb670 msgr2=0x7f2a140cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.067+0000 7f2a19e14700 1 -- 172.21.15.153:0/3646194155 shutdown_connections 2023-11-28T13:00:33.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.067+0000 7f2a19e14700 1 -- 172.21.15.153:0/3646194155 wait complete. 2023-11-28T13:00:33.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.068+0000 7f2a19e14700 1 Processor -- start 2023-11-28T13:00:33.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.068+0000 7f2a19e14700 1 -- start start 2023-11-28T13:00:33.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a19e14700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a1414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a19e14700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a1414d630 0x7f2a140b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a19e14700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140b3450 0x7f2a141592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a19e14700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140b38f0 con 0x7f2a00004d90 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a19e14700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141f2760 con 0x7f2a1414d630 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a19e14700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141f1be0 con 0x7f2a140b3450 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a1414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a1414fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38880/0 (socket says 172.21.15.153:38880) 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 -- 172.21.15.153:0/2019369276 learned_addr learned my addr 172.21.15.153:0/2019369276 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a12ffd700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a1414d630 0x7f2a140b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a13fff700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140b3450 0x7f2a141592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 -- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140b3450 msgr2=0x7f2a141592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140b3450 0x7f2a141592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 -- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a1414d630 msgr2=0x7f2a140b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a1414d630 0x7f2a140b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 -- 172.21.15.153:0/2019369276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1406bba0 con 0x7f2a00004d90 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a13fff700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140b3450 0x7f2a141592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a12ffd700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a1414d630 0x7f2a140b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.069+0000 7f2a137fe700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a1414fd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00f7f0 tx=0x7f29fc00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.070+0000 7f2a0bfff700 1 -- 172.21.15.153:0/2019369276 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0052e0 con 0x7f2a00004d90 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.070+0000 7f2a0bfff700 1 -- 172.21.15.153:0/2019369276 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc005440 con 0x7f2a00004d90 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.070+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1406b8c0 con 0x7f2a00004d90 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.070+0000 7f2a0bfff700 1 -- 172.21.15.153:0/2019369276 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0038c0 con 0x7f2a00004d90 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.070+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a140c9370 con 0x7f2a00004d90 2023-11-28T13:00:33.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.070+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a140b43f0 con 0x7f2a00004d90 2023-11-28T13:00:33.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.071+0000 7f2a0bfff700 1 -- 172.21.15.153:0/2019369276 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f29fc003a20 con 0x7f2a00004d90 2023-11-28T13:00:33.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.071+0000 7f2a0bfff700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29f40682f0 0x7f29f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.071+0000 7f2a0bfff700 1 -- 172.21.15.153:0/2019369276 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29fc085c50 con 0x7f2a00004d90 2023-11-28T13:00:33.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.071+0000 7f2a12ffd700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29f40682f0 0x7f29f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.072+0000 7f2a12ffd700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29f40682f0 0x7f29f406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a04007e10 tx=0x7f2a04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.076+0000 7f2a0bfff700 1 -- 172.21.15.153:0/2019369276 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f29fc0524b0 con 0x7f2a00004d90 2023-11-28T13:00:33.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.302+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a140745c0 con 0x7f2a00004d90 2023-11-28T13:00:33.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.303+0000 7f2a0bfff700 1 -- 172.21.15.153:0/2019369276 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29fc003cb0 con 0x7f2a00004d90 2023-11-28T13:00:33.305 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29f40682f0 msgr2=0x7f29f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29f40682f0 0x7f29f406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a04007e10 tx=0x7f2a04009040 comp rx=0 tx=0).stop 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 msgr2=0x7f2a1414fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a1414fd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00f7f0 tx=0x7f29fc00fbb0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 shutdown_connections 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29f40682f0 0x7f29f406a7a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a00004d90 0x7f2a1414fd30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a140b3450 0x7f2a141592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 --2- 172.21.15.153:0/2019369276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a1414d630 0x7f2a140b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 >> 172.21.15.153:0/2019369276 conn(0x7f2a140cb670 msgr2=0x7f2a140b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 shutdown_connections 2023-11-28T13:00:33.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:33.305+0000 7f2a19e14700 1 -- 172.21.15.153:0/2019369276 wait complete. 2023-11-28T13:00:33.319 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:33.319 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:33.319 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-11-28T13:00:33.319 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:00:33.319 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:33.319 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:33.319 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:33.320 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:33.320 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:00:33.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.320 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:33.337 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42347sec preferred_lft 42347sec 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ad7d:b13e:f07:e265/64 scope link noprefixroute 2023-11-28T13:00:33.338 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:33.339 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T13:00:33.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.339 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:33.339 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:33.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add name ceph-brx type bridge 2023-11-28T13:00:33.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr flush dev ceph-brx 2023-11-28T13:00:33.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set ceph-brx up 2023-11-28T13:00:33.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T13:00:33.339 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:33.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.519 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:33.546 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-11-28T13:00:33.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.547 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-11-28T13:00:33.618 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-11-28T13:00:33.618 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:33.618 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:00:33.618 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:00:33.618 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:33.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.619 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:33.619 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:33.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:00:33.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:00:33.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:00:33.619 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:33.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.861 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:00:33.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.878 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2023-11-28T13:00:33.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:33.934 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:33.934 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:33.934 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:00:33.934 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T13:00:33.935 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:34.029 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T13:00:34.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T13:00:34.030 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:34.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.267 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:34.267 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:34.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-11-28T13:00:34.267 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T13:00:34.267 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:34.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:34.374 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:34.391 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:00:34.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:34.392 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:34.448 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:00:34.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.448 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:00:34.545 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:00:34.545 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:00:34.545 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:00:34.647 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:00:34.647 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:00:34.647 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:00:34.647 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:34.647 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:34.672 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:00:34.672 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:00:34.673 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:00:34.673 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:34.673 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:34.673 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T13:00:34.673 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T13:00:34.673 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=47bfaa00-a151-4537-aa4c-32b0866eb248" will pass to kernel 2023-11-28T13:00:34.676 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:00:34.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.676 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:34.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.707 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:34.798 INFO:teuthology.orchestra.run.smithi153.stdout:client4549 2023-11-28T13:00:34.798 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:34.798 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:34.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.816 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:34.901 INFO:teuthology.orchestra.run.smithi153.stdout:client4549 2023-11-28T13:00:34.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.902 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:34.988 INFO:teuthology.orchestra.run.smithi153.stdout:client4549 2023-11-28T13:00:34.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:34.988 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/47bfaa00-a151-4537-aa4c-32b0866eb248.client4549/status') 2023-11-28T13:00:35.041 INFO:teuthology.orchestra.run.smithi153.stdout:instance: client.4549 (3)172.21.15.153:0/1884536243 2023-11-28T13:00:35.041 INFO:teuthology.orchestra.run.smithi153.stdout:blocklisted: false 2023-11-28T13:00:35.042 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records in 2023-11-28T13:00:35.042 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records out 2023-11-28T13:00:35.042 INFO:teuthology.orchestra.run.smithi153.stderr:71 bytes copied, 3.5881e-05 s, 2.0 MB/s 2023-11-28T13:00:35.043 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-28T13:00:35.043 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:35.043 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:35.043 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:35.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.161+0000 7f526be36700 1 Processor -- start 2023-11-28T13:00:35.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.161+0000 7f526be36700 1 -- start start 2023-11-28T13:00:35.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526be36700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5264140950 0x7f52640b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526be36700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52640b2e70 0x7f526406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526be36700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526413ff00 con 0x7f52640cd8c0 2023-11-28T13:00:35.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526be36700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52641404c0 con 0x7f52640b2e70 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526be36700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5264140dc0 con 0x7f5264140950 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52640b2e70 0x7f526406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f52640b2e70 0x7f526406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38892/0 (socket says 172.21.15.153:38892) 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 -- 172.21.15.153:0/2161015071 learned_addr learned my addr 172.21.15.153:0/2161015071 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f52693d1700 1 --2- 172.21.15.153:0/2161015071 >> v2:172.21.15.177:3300/0 conn(0x7f5264140950 0x7f52640b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 -- 172.21.15.153:0/2161015071 >> v2:172.21.15.177:3300/0 conn(0x7f5264140950 msgr2=0x7f52640b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 --2- 172.21.15.153:0/2161015071 >> v2:172.21.15.177:3300/0 conn(0x7f5264140950 0x7f52640b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 -- 172.21.15.153:0/2161015071 >> v1:172.21.15.177:6789/0 conn(0x7f52640cd8c0 legacy=0x7f52640cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 -- 172.21.15.153:0/2161015071 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526415aa80 con 0x7f52640b2e70 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.162+0000 7f526a3d3700 1 --2- 172.21.15.153:0/2161015071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f526406f850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5264071c40 tx=0x7f525c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc456417ee6dec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f5268bd0700 1 -- 172.21.15.153:0/2161015071 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c00e070 con 0x7f52640b2e70 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f5268bd0700 1 -- 172.21.15.153:0/2161015071 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c003a20 con 0x7f52640b2e70 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f5268bd0700 1 -- 172.21.15.153:0/2161015071 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c003dd0 con 0x7f52640b2e70 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 -- 172.21.15.153:0/2161015071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 msgr2=0x7f526406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 --2- 172.21.15.153:0/2161015071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f526406f850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5264071c40 tx=0x7f525c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 -- 172.21.15.153:0/2161015071 shutdown_connections 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 --2- 172.21.15.153:0/2161015071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f526406f850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 --2- 172.21.15.153:0/2161015071 >> v2:172.21.15.177:3300/0 conn(0x7f5264140950 0x7f52640b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 -- 172.21.15.153:0/2161015071 >> 172.21.15.153:0/2161015071 conn(0x7f526406aed0 msgr2=0x7f526406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 -- 172.21.15.153:0/2161015071 shutdown_connections 2023-11-28T13:00:35.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.163+0000 7f526be36700 1 -- 172.21.15.153:0/2161015071 wait complete. 2023-11-28T13:00:35.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.165+0000 7f526be36700 1 Processor -- start 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.165+0000 7f526be36700 1 -- start start 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f526be36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f5264159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f526be36700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52640cd8c0 0x7f52640c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f526be36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5264140950 0x7f526415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f526be36700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52640c92a0 con 0x7f52640b2e70 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f526be36700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52640c8990 con 0x7f52640cd8c0 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f526be36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52640c6f70 con 0x7f5264140950 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f5264159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f5264159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38896/0 (socket says 172.21.15.153:38896) 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 -- 172.21.15.153:0/3130270251 learned_addr learned my addr 172.21.15.153:0/3130270251 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f52693d1700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52640cd8c0 0x7f52640c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f526a3d3700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5264140950 0x7f526415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 -- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5264140950 msgr2=0x7f526415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5264140950 0x7f526415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 -- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52640cd8c0 msgr2=0x7f52640c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52640cd8c0 0x7f52640c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 -- 172.21.15.153:0/3130270251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c009b30 con 0x7f52640b2e70 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f52693d1700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52640cd8c0 0x7f52640c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.166+0000 7f5269bd2700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f5264159090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f526000c4b0 tx=0x7f526000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.167+0000 7f525a7fc700 1 -- 172.21.15.153:0/3130270251 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260011790 con 0x7f52640b2e70 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.167+0000 7f525a7fc700 1 -- 172.21.15.153:0/3130270251 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52600118f0 con 0x7f52640b2e70 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.167+0000 7f525a7fc700 1 -- 172.21.15.153:0/3130270251 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260011ca0 con 0x7f52640b2e70 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.167+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52640b35f0 con 0x7f52640b2e70 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.167+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52640b3310 con 0x7f52640b2e70 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.167+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5264140150 con 0x7f52640b2e70 2023-11-28T13:00:35.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.168+0000 7f525a7fc700 1 -- 172.21.15.153:0/3130270251 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5260010050 con 0x7f52640b2e70 2023-11-28T13:00:35.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.168+0000 7f525a7fc700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52480682f0 0x7f524806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.168+0000 7f525a7fc700 1 -- 172.21.15.153:0/3130270251 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5260085700 con 0x7f52640b2e70 2023-11-28T13:00:35.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.168+0000 7f52693d1700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52480682f0 0x7f524806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.169+0000 7f52693d1700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52480682f0 0x7f524806a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f52640d1f40 tx=0x7f5254008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.172+0000 7f525a7fc700 1 -- 172.21.15.153:0/3130270251 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5260051f00 con 0x7f52640b2e70 2023-11-28T13:00:35.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.402+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f52640c9510 con 0x7f52640b2e70 2023-11-28T13:00:35.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.402+0000 7f525a7fc700 1 -- 172.21.15.153:0/3130270251 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5260055520 con 0x7f52640b2e70 2023-11-28T13:00:35.404 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:00:35.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52480682f0 msgr2=0x7f524806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52480682f0 0x7f524806a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f52640d1f40 tx=0x7f5254008040 comp rx=0 tx=0).stop 2023-11-28T13:00:35.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 msgr2=0x7f5264159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f5264159090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f526000c4b0 tx=0x7f526000c870 comp rx=0 tx=0).stop 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 shutdown_connections 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52480682f0 0x7f524806a7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52640b2e70 0x7f5264159090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5264140950 0x7f526415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 --2- 172.21.15.153:0/3130270251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52640cd8c0 0x7f52640c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 >> 172.21.15.153:0/3130270251 conn(0x7f526406aed0 msgr2=0x7f526413e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 shutdown_connections 2023-11-28T13:00:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:35.404+0000 7f526be36700 1 -- 172.21.15.153:0/3130270251 wait complete. 2023-11-28T13:00:35.418 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:35.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:35.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-11-28T13:00:35.418 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T13:00:35.419 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:35.419 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:35.419 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:35.419 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:35.419 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T13:00:35.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.419 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-11-28T13:00:35.436 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:35.437 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42345sec preferred_lft 42345sec 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ad7d:b13e:f07:e265/64 scope link noprefixroute 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether a2:10:9d:7d:8b:89 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.438 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::6cbb:9eff:fe64:da81/64 scope link 2023-11-28T13:00:35.439 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.439 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:00:35.439 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether a2:10:9d:7d:8b:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:00:35.439 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::a010:9dff:fe7d:8b89/64 scope link tentative 2023-11-28T13:00:35.439 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.439 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:35.502 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-11-28T13:00:35.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.504 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-11-28T13:00:35.563 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-11-28T13:00:35.563 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:35.563 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:00:35.564 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:00:35.564 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:35.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.564 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:35.564 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:35.564 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:00:35.565 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:00:35.565 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:00:35.565 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:35.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.668 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:00:35.684 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:00:35.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.685 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2023-11-28T13:00:35.741 INFO:teuthology.orchestra.run.smithi153.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T13:00:35.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.742 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:35.742 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:35.742 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:00:35.742 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T13:00:35.742 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:35.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.835 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T13:00:35.876 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:35.876 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:35.876 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:35.876 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.877 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:35.877 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.877 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:35.877 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f6:d7:85:a4:b0:92 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:35.877 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:35.877 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.878 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f4d7:85ff:fea4:b092/64 scope link tentative 2023-11-28T13:00:35.878 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.896 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T13:00:35.973 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:35.973 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:35.973 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether f6:d7:85:a4:b0:92 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::f4d7:85ff:fea4:b092/64 scope link 2023-11-28T13:00:35.974 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:35.988 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T13:00:35.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.988 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:35.988 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:35.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T13:00:35.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:35.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T13:00:35.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T13:00:35.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T13:00:35.989 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:36.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:36.223 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:00:36.223 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:00:36.223 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.1 up 2023-11-28T13:00:36.223 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T13:00:36.223 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:00:36.329 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:36.329 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:36.351 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T13:00:36.352 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:36.352 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:36.408 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:00:36.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:36.408 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:00:36.493 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:00:36.493 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T13:00:36.493 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:00:36.501 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:00:36.502 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T13:00:36.502 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:00:36.502 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:36.502 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:36.544 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T13:00:36.544 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T13:00:36.544 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:00:36.545 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:36.545 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:36.545 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T13:00:36.545 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T13:00:36.545 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=47bfaa00-a151-4537-aa4c-32b0866eb248" will pass to kernel 2023-11-28T13:00:36.560 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:00:36.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:36.560 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:36.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:36.587 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:36.659 INFO:teuthology.orchestra.run.smithi153.stdout:client4638 2023-11-28T13:00:36.659 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:36.659 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:36.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:36.717 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:36.788 INFO:teuthology.orchestra.run.smithi153.stdout:client4638 2023-11-28T13:00:36.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:36.789 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:36.860 INFO:teuthology.orchestra.run.smithi153.stdout:client4638 2023-11-28T13:00:36.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:36.861 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/47bfaa00-a151-4537-aa4c-32b0866eb248.client4638/status') 2023-11-28T13:00:36.941 INFO:teuthology.orchestra.run.smithi153.stdout:instance: client.4638 (3)192.168.0.2:0/3118288305 2023-11-28T13:00:36.941 INFO:teuthology.orchestra.run.smithi153.stdout:blocklisted: false 2023-11-28T13:00:36.942 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records in 2023-11-28T13:00:36.942 INFO:teuthology.orchestra.run.smithi153.stderr:0+1 records out 2023-11-28T13:00:36.942 INFO:teuthology.orchestra.run.smithi153.stderr:69 bytes copied, 2.2369e-05 s, 3.1 MB/s 2023-11-28T13:00:36.943 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-28T13:00:36.943 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:36.943 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:36.943 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:37.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.101+0000 7fc31f87f700 1 Processor -- start 2023-11-28T13:00:37.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.101+0000 7fc31f87f700 1 -- start start 2023-11-28T13:00:37.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31f87f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc3181658c0 0x7fc318167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31f87f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31815d060 con 0x7fc318071040 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31f87f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31815d340 con 0x7fc31806dee0 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31f87f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc318161550 con 0x7fc3181658c0 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31de1c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc3181658c0 0x7fc318167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31d61b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc318071040 0x7fc318071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57074/0 (socket says 172.21.15.153:57074) 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31d61b700 1 -- 172.21.15.153:0/843196855 learned_addr learned my addr 172.21.15.153:0/843196855 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc30ffff700 1 -- 172.21.15.153:0/843196855 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458351940 0 0) 0x7fc31815d060 con 0x7fc318071040 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc30ffff700 1 -- 172.21.15.153:0/843196855 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3040035b0 con 0x7fc318071040 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31de1c700 1 -- 172.21.15.153:0/843196855 >> v1:172.21.15.177:6789/0 conn(0x7fc31806dee0 legacy=0x7fc31806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31de1c700 1 -- 172.21.15.153:0/843196855 >> v1:172.21.15.153:6789/0 conn(0x7fc318071040 legacy=0x7fc318071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.102+0000 7fc31de1c700 1 -- 172.21.15.153:0/843196855 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31815d620 con 0x7fc3181658c0 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31de1c700 1 --2- 172.21.15.153:0/843196855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3181658c0 0x7fc318167ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc310009d90 tx=0x7fc31000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e205eaf05f4ddb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:37.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc30ffff700 1 -- 172.21.15.153:0/843196855 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000e070 con 0x7fc3181658c0 2023-11-28T13:00:37.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc30ffff700 1 -- 172.21.15.153:0/843196855 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31000bb20 con 0x7fc3181658c0 2023-11-28T13:00:37.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc30ffff700 1 -- 172.21.15.153:0/843196855 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310012430 con 0x7fc3181658c0 2023-11-28T13:00:37.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31f87f700 1 -- 172.21.15.153:0/843196855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3181658c0 msgr2=0x7fc318167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31f87f700 1 --2- 172.21.15.153:0/843196855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3181658c0 0x7fc318167ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc310009d90 tx=0x7fc31000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31f87f700 1 -- 172.21.15.153:0/843196855 shutdown_connections 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31f87f700 1 --2- 172.21.15.153:0/843196855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3181658c0 0x7fc318167ca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31f87f700 1 -- 172.21.15.153:0/843196855 >> 172.21.15.153:0/843196855 conn(0x7fc3180d0000 msgr2=0x7fc3180d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31f87f700 1 -- 172.21.15.153:0/843196855 shutdown_connections 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.103+0000 7fc31f87f700 1 -- 172.21.15.153:0/843196855 wait complete. 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 Processor -- start 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 -- start start 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc31806dee0 0x7fc3180ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318071040 0x7fc3180b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3181658c0 0x7fc318142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180adbd0 con 0x7fc31806dee0 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3181571e0 con 0x7fc318071040 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31f87f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180ad8a0 con 0x7fc3181658c0 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31d61b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc31806dee0 0x7fc3180ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31d61b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc31806dee0 0x7fc3180ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38916/0 (socket says 172.21.15.153:38916) 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31d61b700 1 -- 172.21.15.153:0/3833050168 learned_addr learned my addr 172.21.15.153:0/3833050168 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.105+0000 7fc31ce1a700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318071040 0x7fc3180b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31de1c700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3181658c0 0x7fc318142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31d61b700 1 -- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3181658c0 msgr2=0x7fc318142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31d61b700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3181658c0 0x7fc318142850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31d61b700 1 -- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318071040 msgr2=0x7fc3180b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31d61b700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318071040 0x7fc3180b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31d61b700 1 -- 172.21.15.153:0/3833050168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310009a70 con 0x7fc31806dee0 2023-11-28T13:00:37.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31de1c700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3181658c0 0x7fc318142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31ce1a700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318071040 0x7fc3180b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31d61b700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc31806dee0 0x7fc3180ae060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c4d0 tx=0x7fc30800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc30dffb700 1 -- 172.21.15.153:0/3833050168 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308005d70 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc30dffb700 1 -- 172.21.15.153:0/3833050168 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3080070d0 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc30dffb700 1 -- 172.21.15.153:0/3833050168 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308007460 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31815ccd0 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.106+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31815d900 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.107+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31806a270 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.108+0000 7fc30dffb700 1 -- 172.21.15.153:0/3833050168 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc308019030 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.108+0000 7fc30dffb700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc2fc0682f0 0x7fc2fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.108+0000 7fc30dffb700 1 -- 172.21.15.153:0/3833050168 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc308085080 con 0x7fc31806dee0 2023-11-28T13:00:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.108+0000 7fc31ce1a700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc2fc0682f0 0x7fc2fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.108+0000 7fc31ce1a700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc2fc0682f0 0x7fc2fc06a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc318142720 tx=0x7fc314009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:37.114 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.112+0000 7fc30dffb700 1 -- 172.21.15.153:0/3833050168 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc3080518e0 con 0x7fc31806dee0 2023-11-28T13:00:37.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.336+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc3180720a0 con 0x7fc31806dee0 2023-11-28T13:00:37.338 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.336+0000 7fc30dffb700 1 -- 172.21.15.153:0/3833050168 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc308017060 con 0x7fc31806dee0 2023-11-28T13:00:37.338 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc2fc0682f0 msgr2=0x7fc2fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc2fc0682f0 0x7fc2fc06a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc318142720 tx=0x7fc314009040 comp rx=0 tx=0).stop 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc31806dee0 msgr2=0x7fc3180ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc31806dee0 0x7fc3180ae060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c4d0 tx=0x7fc30800c890 comp rx=0 tx=0).stop 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 shutdown_connections 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc2fc0682f0 0x7fc2fc06a7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc31806dee0 0x7fc3180ae060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3181658c0 0x7fc318142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.341 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 --2- 172.21.15.153:0/3833050168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318071040 0x7fc3180b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 >> 172.21.15.153:0/3833050168 conn(0x7fc3180d0000 msgr2=0x7fc31806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:37.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 shutdown_connections 2023-11-28T13:00:37.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:37.338+0000 7fc31f87f700 1 -- 172.21.15.153:0/3833050168 wait complete. 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:37.352 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T13:00:37.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.353 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-11-28T13:00:37.372 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:37.372 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:37.372 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:37.372 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:37.372 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:00:37.373 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42332sec preferred_lft 42332sec 2023-11-28T13:00:37.374 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::70d6:b0a0:24e:88d1/64 scope link noprefixroute 2023-11-28T13:00:37.374 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:37.374 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T13:00:37.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.374 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:37.374 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:37.374 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add name ceph-brx type bridge 2023-11-28T13:00:37.374 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr flush dev ceph-brx 2023-11-28T13:00:37.375 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set ceph-brx up 2023-11-28T13:00:37.375 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T13:00:37.375 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:37.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.545 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:37.571 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-11-28T13:00:37.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.573 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-11-28T13:00:37.647 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-11-28T13:00:37.647 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:37.647 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:00:37.647 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:00:37.648 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:37.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.648 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:37.648 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:37.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:00:37.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:00:37.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:00:37.649 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:37.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.889 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T13:00:37.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.907 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-11-28T13:00:37.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:37.963 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:37.963 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:37.964 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:00:37.964 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T13:00:37.964 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:38.057 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T13:00:38.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.058 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:38.058 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:38.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T13:00:38.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:38.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T13:00:38.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T13:00:38.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T13:00:38.059 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:38.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.262 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:38.262 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:38.263 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.0 up 2023-11-28T13:00:38.263 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T13:00:38.263 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:38.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:38.363 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:38.388 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T13:00:38.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:38.388 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:38.444 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:00:38.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.445 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:00:38.541 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:00:38.541 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T13:00:38.541 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T13:00:38.634 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:00:38.634 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T13:00:38.634 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T13:00:38.634 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:38.634 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T13:00:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=47bfaa00-a151-4537-aa4c-32b0866eb248" will pass to kernel 2023-11-28T13:00:38.682 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:00:38.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.683 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:38.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.712 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:38.784 INFO:teuthology.orchestra.run.smithi177.stdout:client4564 2023-11-28T13:00:38.784 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:38.785 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:38.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.842 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:38.913 INFO:teuthology.orchestra.run.smithi177.stdout:client4564 2023-11-28T13:00:38.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.913 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:38.984 INFO:teuthology.orchestra.run.smithi177.stdout:client4564 2023-11-28T13:00:38.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:38.985 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/47bfaa00-a151-4537-aa4c-32b0866eb248.client4564/status') 2023-11-28T13:00:39.063 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4564 (3)192.168.0.1:0/2857476000 2023-11-28T13:00:39.064 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2023-11-28T13:00:39.064 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2023-11-28T13:00:39.064 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2023-11-28T13:00:39.064 INFO:teuthology.orchestra.run.smithi177.stderr:69 bytes copied, 5.4955e-05 s, 1.3 MB/s 2023-11-28T13:00:39.065 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-28T13:00:39.065 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:39.065 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:39.065 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:39.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.198+0000 7f597f7e8700 1 Processor -- start 2023-11-28T13:00:39.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.198+0000 7f597f7e8700 1 -- start start 2023-11-28T13:00:39.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.199+0000 7f597f7e8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5978140950 0x7f59780b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.199+0000 7f597f7e8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597813ff00 con 0x7f59780b2e70 2023-11-28T13:00:39.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.199+0000 7f597f7e8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59781404c0 con 0x7f59780cd8c0 2023-11-28T13:00:39.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.199+0000 7f597f7e8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5978140dc0 con 0x7f5978140950 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.199+0000 7f597dd85700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f59780b2e70 0x7f597806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57092/0 (socket says 172.21.15.153:57092) 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.199+0000 7f597dd85700 1 -- 172.21.15.153:0/77110403 learned_addr learned my addr 172.21.15.153:0/77110403 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597cd83700 1 --2- 172.21.15.153:0/77110403 >> v2:172.21.15.177:3300/0 conn(0x7f5978140950 0x7f59780b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302574983 0 0) 0x7f597813ff00 con 0x7f59780b2e70 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59640035b0 con 0x7f59780b2e70 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3865406415 0 0) 0x7f59640035b0 con 0x7f59780b2e70 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 >> v2:172.21.15.177:3300/0 conn(0x7f5978140950 msgr2=0x7f59780b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 --2- 172.21.15.153:0/77110403 >> v2:172.21.15.177:3300/0 conn(0x7f5978140950 0x7f59780b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 >> v1:172.21.15.177:6789/0 conn(0x7f59780cd8c0 legacy=0x7f59780cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597815aa80 con 0x7f59780b2e70 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f59700019f0 con 0x7f59780b2e70 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 >> v1:172.21.15.153:6789/0 conn(0x7f59780b2e70 legacy=0x7f597806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5964003a00 0x7f5964003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5964004850 0x7f5964004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964005200 0x7f596400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978140dc0 con 0x7f5964005200 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596400c110 con 0x7f5964003a00 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596400c2d0 con 0x7f5964004850 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597d584700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5964003a00 0x7f5964003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597cd83700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5964004850 0x7f5964004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597dd85700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964005200 0x7f596400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597d584700 1 -- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5964004850 msgr2=0x7f5964004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597d584700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5964004850 0x7f5964004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597d584700 1 -- 172.21.15.153:0/77110403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964005200 msgr2=0x7f596400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597d584700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964005200 0x7f596400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597d584700 1 -- 172.21.15.153:0/77110403 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640035b0 con 0x7f5964003a00 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597cd83700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5964004850 0x7f5964004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:39.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597dd85700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964005200 0x7f596400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.200+0000 7f597dd85700 1 -- 172.21.15.153:0/77110403 reap_dead start 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597d584700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5964003a00 0x7f5964003e10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5968001820 tx=0x7f5964005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2a77343f4972f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680012e0 con 0x7f5964003a00 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59680052d0 con 0x7f5964003a00 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f596ffff700 1 -- 172.21.15.153:0/77110403 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680036d0 con 0x7f5964003a00 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597f7e8700 1 -- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5964003a00 msgr2=0x7f5964003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597f7e8700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5964003a00 0x7f5964003e10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5968001820 tx=0x7f5964005880 comp rx=0 tx=0).stop 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597f7e8700 1 -- 172.21.15.153:0/77110403 shutdown_connections 2023-11-28T13:00:39.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597f7e8700 1 --2- 172.21.15.153:0/77110403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5964003a00 0x7f5964003e10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597f7e8700 1 -- 172.21.15.153:0/77110403 >> 172.21.15.153:0/77110403 conn(0x7f597806aed0 msgr2=0x7f597806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:39.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597f7e8700 1 -- 172.21.15.153:0/77110403 shutdown_connections 2023-11-28T13:00:39.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.201+0000 7f597f7e8700 1 -- 172.21.15.153:0/77110403 wait complete. 2023-11-28T13:00:39.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 Processor -- start 2023-11-28T13:00:39.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 -- start start 2023-11-28T13:00:39.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964003a00 0x7f59781efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59780c7300 0x7f59780ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59780c7f80 0x7f59780d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780c46f0 con 0x7f5964003a00 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780c5b60 con 0x7f59780c7300 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597f7e8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59780ca650 con 0x7f59780c7f80 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597d584700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964003a00 0x7f59781efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597d584700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964003a00 0x7f59781efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38936/0 (socket says 172.21.15.153:38936) 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.203+0000 7f597d584700 1 -- 172.21.15.153:0/2989712161 learned_addr learned my addr 172.21.15.153:0/2989712161 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597cd83700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59780c7300 0x7f59780ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597dd85700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59780c7f80 0x7f59780d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597d584700 1 -- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59780c7f80 msgr2=0x7f59780d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597d584700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59780c7f80 0x7f59780d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597d584700 1 -- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59780c7300 msgr2=0x7f59780ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597d584700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59780c7300 0x7f59780ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597d584700 1 -- 172.21.15.153:0/2989712161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780b35f0 con 0x7f5964003a00 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597dd85700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59780c7f80 0x7f59780d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597d584700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964003a00 0x7f59781efd40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f5968001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f596dffb700 1 -- 172.21.15.153:0/2989712161 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968001280 con 0x7f5964003a00 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f596dffb700 1 -- 172.21.15.153:0/2989712161 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59680050d0 con 0x7f5964003a00 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59780b3310 con 0x7f5964003a00 2023-11-28T13:00:39.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f596dffb700 1 -- 172.21.15.153:0/2989712161 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596801d420 con 0x7f5964003a00 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.204+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978140150 con 0x7f5964003a00 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.205+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59780c5e00 con 0x7f5964003a00 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.206+0000 7f596dffb700 1 -- 172.21.15.153:0/2989712161 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f596802d070 con 0x7f5964003a00 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.206+0000 7f596dffb700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f595c0682f0 0x7f595c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.206+0000 7f596dffb700 1 -- 172.21.15.153:0/2989712161 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5968086550 con 0x7f5964003a00 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.206+0000 7f597cd83700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f595c0682f0 0x7f595c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.206+0000 7f597cd83700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f595c0682f0 0x7f595c06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f59780cdd90 tx=0x7f5974008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.210+0000 7f596dffb700 1 -- 172.21.15.153:0/2989712161 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5968052db0 con 0x7f5964003a00 2023-11-28T13:00:39.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.436+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5978068690 con 0x7f5964003a00 2023-11-28T13:00:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.436+0000 7f596dffb700 1 -- 172.21.15.153:0/2989712161 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59680563d0 con 0x7f5964003a00 2023-11-28T13:00:39.438 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:00:39.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f595c0682f0 msgr2=0x7f595c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f595c0682f0 0x7f595c06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f59780cdd90 tx=0x7f5974008040 comp rx=0 tx=0).stop 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964003a00 msgr2=0x7f59781efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964003a00 0x7f59781efd40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f5968001170 comp rx=0 tx=0).stop 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 shutdown_connections 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f595c0682f0 0x7f595c06a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5964003a00 0x7f59781efd40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59780c7f80 0x7f59780d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 --2- 172.21.15.153:0/2989712161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59780c7300 0x7f59780ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 >> 172.21.15.153:0/2989712161 conn(0x7f597806aed0 msgr2=0x7f59780cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:39.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 shutdown_connections 2023-11-28T13:00:39.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:39.438+0000 7f597f7e8700 1 -- 172.21.15.153:0/2989712161 wait complete. 2023-11-28T13:00:39.452 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:39.452 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:39.452 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-11-28T13:00:39.452 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T13:00:39.452 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:39.452 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:39.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:39.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:39.453 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T13:00:39.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.453 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:39.470 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42330sec preferred_lft 42330sec 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::70d6:b0a0:24e:88d1/64 scope link noprefixroute 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:39.471 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 5a:f6:f7:af:8e:69 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::e0a1:2bff:feec:bd15/64 scope link 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 5a:f6:f7:af:8e:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::58f6:f7ff:feaf:8e69/64 scope link 2023-11-28T13:00:39.472 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.473 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:39.535 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-11-28T13:00:39.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.537 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-11-28T13:00:39.596 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-11-28T13:00:39.596 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:39.597 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:00:39.597 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:00:39.597 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:39.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.597 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:39.597 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:39.597 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:00:39.598 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:00:39.598 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:00:39.598 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:39.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.701 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T13:00:39.717 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T13:00:39.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.718 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-11-28T13:00:39.774 INFO:teuthology.orchestra.run.smithi177.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T13:00:39.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.774 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:39.774 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:39.774 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T13:00:39.774 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T13:00:39.775 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:39.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.867 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T13:00:39.903 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether b2:2e:d8:d6:b1:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::b02e:d8ff:fed6:b1fe/64 scope link tentative 2023-11-28T13:00:39.904 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:39.920 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T13:00:39.993 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether b2:2e:d8:d6:b1:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::b02e:d8ff:fed6:b1fe/64 scope link tentative 2023-11-28T13:00:39.994 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:40.002 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T13:00:40.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T13:00:40.003 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:40.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.276 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T13:00:40.276 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T13:00:40.276 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.1 up 2023-11-28T13:00:40.276 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T13:00:40.277 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T13:00:40.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:40.384 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:40.404 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T13:00:40.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:40.404 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:40.460 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:00:40.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.461 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:00:40.546 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:00:40.546 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T13:00:40.546 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T13:00:40.555 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:00:40.555 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T13:00:40.555 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T13:00:40.555 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:40.555 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:40.581 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T13:00:40.581 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T13:00:40.581 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T13:00:40.581 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:00:40.581 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:00:40.582 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T13:00:40.582 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T13:00:40.582 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=47bfaa00-a151-4537-aa4c-32b0866eb248" will pass to kernel 2023-11-28T13:00:40.585 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:00:40.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.585 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:40.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.614 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:40.686 INFO:teuthology.orchestra.run.smithi177.stdout:client4466 2023-11-28T13:00:40.686 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:40.686 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:40.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.744 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:40.815 INFO:teuthology.orchestra.run.smithi177.stdout:client4466 2023-11-28T13:00:40.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.815 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:00:40.887 INFO:teuthology.orchestra.run.smithi177.stdout:client4466 2023-11-28T13:00:40.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:40.887 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/47bfaa00-a151-4537-aa4c-32b0866eb248.client4466/status') 2023-11-28T13:00:40.967 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4466 (3)192.168.0.2:0/1398615396 2023-11-28T13:00:40.967 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2023-11-28T13:00:40.967 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2023-11-28T13:00:40.967 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2023-11-28T13:00:40.967 INFO:teuthology.orchestra.run.smithi177.stderr:69 bytes copied, 2.0953e-05 s, 3.3 MB/s 2023-11-28T13:00:40.968 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T13:00:41.010 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:41.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.144+0000 7fbcb2998700 1 Processor -- start 2023-11-28T13:00:41.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.144+0000 7fbcb2998700 1 -- start start 2023-11-28T13:00:41.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcb2998700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbcac0cf690 0x7fbcac06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcb2998700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac06a7b0 con 0x7fbcac15c810 2023-11-28T13:00:41.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcb2998700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac06aa10 con 0x7fbcac0af480 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcb2998700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac06acf0 con 0x7fbcac0cf690 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcabfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbcac15c810 0x7fbcac0cef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57108/0 (socket says 172.21.15.153:57108) 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcabfff700 1 -- 172.21.15.153:0/4289639881 learned_addr learned my addr 172.21.15.153:0/4289639881 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcab7fe700 1 --2- 172.21.15.153:0/4289639881 >> v2:172.21.15.177:3300/0 conn(0x7fbcac0cf690 0x7fbcac06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133507434 0 0) 0x7fbcac06a7b0 con 0x7fbcac15c810 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc940035b0 con 0x7fbcac15c810 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2249652074 0 0) 0x7fbc940035b0 con 0x7fbcac15c810 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 >> v2:172.21.15.177:3300/0 conn(0x7fbcac0cf690 msgr2=0x7fbcac06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcaaffd700 1 --2- 172.21.15.153:0/4289639881 >> v2:172.21.15.177:3300/0 conn(0x7fbcac0cf690 0x7fbcac06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 >> v1:172.21.15.177:6789/0 conn(0x7fbcac0af480 legacy=0x7fbcac0c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.145+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac06def0 con 0x7fbcac15c810 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fbca0001670 con 0x7fbcac15c810 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 >> v1:172.21.15.153:6789/0 conn(0x7fbcac15c810 legacy=0x7fbcac0cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc94003a00 0x7fbc94003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94004850 0x7fbc94004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc94005200 0x7fbc9400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac06acf0 con 0x7fbc94004850 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9400c080 con 0x7fbc94003a00 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9400c240 con 0x7fbc94005200 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcab7fe700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94004850 0x7fbc94004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcabfff700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc94003a00 0x7fbc94003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcb0f35700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc94005200 0x7fbc9400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcabfff700 1 -- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc94005200 msgr2=0x7fbc9400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcabfff700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc94005200 0x7fbc9400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcabfff700 1 -- 172.21.15.153:0/4289639881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94004850 msgr2=0x7fbc94004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcabfff700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94004850 0x7fbc94004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcabfff700 1 -- 172.21.15.153:0/4289639881 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc940035b0 con 0x7fbc94003a00 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcb0f35700 1 -- 172.21.15.153:0/4289639881 reap_dead start 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcabfff700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc94003a00 0x7fbc94003e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbca000c240 tx=0x7fbc94005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d32bacb66428e1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0007c10 con 0x7fbc94003a00 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca000cde0 con 0x7fbc94003a00 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.146+0000 7fbcaaffd700 1 -- 172.21.15.153:0/4289639881 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0005ba0 con 0x7fbc94003a00 2023-11-28T13:00:41.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.147+0000 7fbcb2998700 1 -- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc94003a00 msgr2=0x7fbc94003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.147+0000 7fbcb2998700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc94003a00 0x7fbc94003e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbca000c240 tx=0x7fbc94005880 comp rx=0 tx=0).stop 2023-11-28T13:00:41.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.147+0000 7fbcb2998700 1 -- 172.21.15.153:0/4289639881 shutdown_connections 2023-11-28T13:00:41.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.147+0000 7fbcb2998700 1 --2- 172.21.15.153:0/4289639881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc94003a00 0x7fbc94003e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.147+0000 7fbcb2998700 1 -- 172.21.15.153:0/4289639881 >> 172.21.15.153:0/4289639881 conn(0x7fbcac0b19f0 msgr2=0x7fbcac0b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.147+0000 7fbcb2998700 1 -- 172.21.15.153:0/4289639881 shutdown_connections 2023-11-28T13:00:41.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.147+0000 7fbcb2998700 1 -- 172.21.15.153:0/4289639881 wait complete. 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.148+0000 7fbcb2998700 1 Processor -- start 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.148+0000 7fbcb2998700 1 -- start start 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcb2998700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94003a00 0x7fbcac151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcb2998700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac0cdc70 0x7fbcac0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcb2998700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcac0b5280 0x7fbcac0b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcb2998700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0ce620 con 0x7fbc94003a00 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcb2998700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0b5c30 con 0x7fbcac0cdc70 2023-11-28T13:00:41.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcb2998700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac159bf0 con 0x7fbcac0b5280 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94003a00 0x7fbcac151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94003a00 0x7fbcac151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38956/0 (socket says 172.21.15.153:38956) 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcabfff700 1 -- 172.21.15.153:0/3777134015 learned_addr learned my addr 172.21.15.153:0/3777134015 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcab7fe700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac0cdc70 0x7fbcac0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcb0f35700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcac0b5280 0x7fbcac0b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcabfff700 1 -- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcac0b5280 msgr2=0x7fbcac0b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcabfff700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcac0b5280 0x7fbcac0b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcabfff700 1 -- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac0cdc70 msgr2=0x7fbcac0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.149+0000 7fbcabfff700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac0cdc70 0x7fbcac0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.150+0000 7fbcabfff700 1 -- 172.21.15.153:0/3777134015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac06e4b0 con 0x7fbc94003a00 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.150+0000 7fbcabfff700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94003a00 0x7fbcac151b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c210 tx=0x7fbca0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.150+0000 7fbca8ff9700 1 -- 172.21.15.153:0/3777134015 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0005ea0 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.150+0000 7fbca8ff9700 1 -- 172.21.15.153:0/3777134015 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca00070d0 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.150+0000 7fbca8ff9700 1 -- 172.21.15.153:0/3777134015 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0007460 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.150+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac06e1d0 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.150+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac06e9d0 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.151+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac068690 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.151+0000 7fbca8ff9700 1 -- 172.21.15.153:0/3777134015 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbca0024070 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.151+0000 7fbca8ff9700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc980682f0 0x7fbc9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.152+0000 7fbca8ff9700 1 -- 172.21.15.153:0/3777134015 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbca0086d20 con 0x7fbc94003a00 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.152+0000 7fbcab7fe700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc980682f0 0x7fbc9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.152+0000 7fbcab7fe700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc980682f0 0x7fbc9806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0b0920 tx=0x7fbc9c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.156+0000 7fbca8ff9700 1 -- 172.21.15.153:0/3777134015 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbca0053580 con 0x7fbc94003a00 2023-11-28T13:00:41.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.346+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcac070f00 con 0x7fbc94003a00 2023-11-28T13:00:41.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.347+0000 7fbca8ff9700 1 -- 172.21.15.153:0/3777134015 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fbca0019310 con 0x7fbc94003a00 2023-11-28T13:00:41.351 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:41.351 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.349+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc980682f0 msgr2=0x7fbc9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.349+0000 7fbcb2998700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc980682f0 0x7fbc9806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0b0920 tx=0x7fbc9c009210 comp rx=0 tx=0).stop 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.349+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94003a00 msgr2=0x7fbcac151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.349+0000 7fbcb2998700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94003a00 0x7fbcac151b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c210 tx=0x7fbca0001a00 comp rx=0 tx=0).stop 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 shutdown_connections 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc980682f0 0x7fbc9806a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc94003a00 0x7fbcac151b20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcac0b5280 0x7fbcac0b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 --2- 172.21.15.153:0/3777134015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac0cdc70 0x7fbcac0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.353 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 >> 172.21.15.153:0/3777134015 conn(0x7fbcac0b19f0 msgr2=0x7fbcac0af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.353 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 shutdown_connections 2023-11-28T13:00:41.353 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.350+0000 7fbcb2998700 1 -- 172.21.15.153:0/3777134015 wait complete. 2023-11-28T13:00:41.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:41.363 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:41.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.487+0000 7f19b069b700 1 Processor -- start 2023-11-28T13:00:41.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.487+0000 7f19b069b700 1 -- start start 2023-11-28T13:00:41.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19b069b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19a8154e70 0x7f19a80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19b069b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8071750 0x7f19a815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19b069b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a8155240 con 0x7f19a80cf360 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19b069b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a815b100 con 0x7f19a8154e70 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19b069b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a815b3e0 con 0x7f19a8071750 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ae437700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19a8154e70 0x7f19a80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19adc36700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f19a80cf360 0x7f19a8071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57134/0 (socket says 172.21.15.153:57134) 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19adc36700 1 -- 172.21.15.153:0/2349033761 learned_addr learned my addr 172.21.15.153:0/2349033761 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19aec38700 1 --2- 172.21.15.153:0/2349033761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8071750 0x7f19a815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ae437700 1 -- 172.21.15.153:0/2349033761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8071750 msgr2=0x7f19a815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ae437700 1 --2- 172.21.15.153:0/2349033761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8071750 0x7f19a815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ae437700 1 -- 172.21.15.153:0/2349033761 >> v1:172.21.15.153:6789/0 conn(0x7f19a80cf360 legacy=0x7f19a8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ae437700 1 -- 172.21.15.153:0/2349033761 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a815b6c0 con 0x7f19a8154e70 2023-11-28T13:00:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ad435700 1 -- 172.21.15.153:0/2349033761 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057449299 0 0) 0x7f19a8155240 con 0x7f19a80cf360 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ae437700 1 --2- 172.21.15.153:0/2349033761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a8154e70 0x7f19a80cdbd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009840 tx=0x7f19a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c2bfc2dc6da75f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ad435700 1 -- 172.21.15.153:0/2349033761 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4011830 con 0x7f19a8154e70 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ad435700 1 -- 172.21.15.153:0/2349033761 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4011990 con 0x7f19a8154e70 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.488+0000 7f19ad435700 1 -- 172.21.15.153:0/2349033761 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4011d20 con 0x7f19a8154e70 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 -- 172.21.15.153:0/2349033761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a8154e70 msgr2=0x7f19a80cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 --2- 172.21.15.153:0/2349033761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a8154e70 0x7f19a80cdbd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009840 tx=0x7f19a4009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 -- 172.21.15.153:0/2349033761 shutdown_connections 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 --2- 172.21.15.153:0/2349033761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8071750 0x7f19a815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 --2- 172.21.15.153:0/2349033761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a8154e70 0x7f19a80cdbd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 -- 172.21.15.153:0/2349033761 >> 172.21.15.153:0/2349033761 conn(0x7f19a80d0000 msgr2=0x7f19a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 -- 172.21.15.153:0/2349033761 shutdown_connections 2023-11-28T13:00:41.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.489+0000 7f19b069b700 1 -- 172.21.15.153:0/2349033761 wait complete. 2023-11-28T13:00:41.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.491+0000 7f19b069b700 1 Processor -- start 2023-11-28T13:00:41.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.491+0000 7f19b069b700 1 -- start start 2023-11-28T13:00:41.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19b069b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19a8071750 0x7f19a814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19b069b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a80cf360 0x7f19a81e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19b069b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8154e70 0x7f19a815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19b069b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a815a930 con 0x7f19a80cf360 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19b069b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a814d280 con 0x7f19a8071750 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19b069b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a81e4b40 con 0x7f19a8154e70 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a80cf360 0x7f19a81e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a80cf360 0x7f19a81e3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38976/0 (socket says 172.21.15.153:38976) 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 -- 172.21.15.153:0/2098406023 learned_addr learned my addr 172.21.15.153:0/2098406023 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19ae437700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19a8071750 0x7f19a814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19aec38700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8154e70 0x7f19a815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 -- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8154e70 msgr2=0x7f19a815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8154e70 0x7f19a815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 -- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19a8071750 msgr2=0x7f19a814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19a8071750 0x7f19a814cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19adc36700 1 -- 172.21.15.153:0/2098406023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a40094a0 con 0x7f19a80cf360 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.492+0000 7f19ae437700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19a8071750 0x7f19a814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.493+0000 7f19adc36700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a80cf360 0x7f19a81e3d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19980011e0 tx=0x7f199800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.493+0000 7f199effd700 1 -- 172.21.15.153:0/2098406023 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998004c80 con 0x7f19a80cf360 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.493+0000 7f199effd700 1 -- 172.21.15.153:0/2098406023 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1998004e00 con 0x7f19a80cf360 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.493+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a80af520 con 0x7f19a80cf360 2023-11-28T13:00:41.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.493+0000 7f199effd700 1 -- 172.21.15.153:0/2098406023 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998007310 con 0x7f19a80cf360 2023-11-28T13:00:41.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.493+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a815b9a0 con 0x7f19a80cf360 2023-11-28T13:00:41.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.494+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a815adb0 con 0x7f19a80cf360 2023-11-28T13:00:41.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.494+0000 7f199effd700 1 -- 172.21.15.153:0/2098406023 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f199801a020 con 0x7f19a80cf360 2023-11-28T13:00:41.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.494+0000 7f199effd700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f198c0682f0 0x7f198c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.494+0000 7f199effd700 1 -- 172.21.15.153:0/2098406023 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1998085a00 con 0x7f19a80cf360 2023-11-28T13:00:41.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.494+0000 7f19ae437700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f198c0682f0 0x7f198c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.495+0000 7f19ae437700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f198c0682f0 0x7f198c06a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19a40090b0 tx=0x7f19a401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.499+0000 7f199effd700 1 -- 172.21.15.153:0/2098406023 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1998052170 con 0x7f19a80cf360 2023-11-28T13:00:41.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.684+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19a815aa70 con 0x7f19a80cf360 2023-11-28T13:00:41.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.685+0000 7f199effd700 1 -- 172.21.15.153:0/2098406023 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19a815aa70 con 0x7f19a80cf360 2023-11-28T13:00:41.687 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:41.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f198c0682f0 msgr2=0x7f198c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f198c0682f0 0x7f198c06a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19a40090b0 tx=0x7f19a401c040 comp rx=0 tx=0).stop 2023-11-28T13:00:41.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a80cf360 msgr2=0x7f19a81e3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a80cf360 0x7f19a81e3d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19980011e0 tx=0x7f199800cae0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 shutdown_connections 2023-11-28T13:00:41.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19a8154e70 0x7f19a815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19a8071750 0x7f19a814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f198c0682f0 0x7f198c06a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 --2- 172.21.15.153:0/2098406023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19a80cf360 0x7f19a81e3d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 >> 172.21.15.153:0/2098406023 conn(0x7f19a80d0000 msgr2=0x7f19a80ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 shutdown_connections 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.687+0000 7f19b069b700 1 -- 172.21.15.153:0/2098406023 wait complete. 2023-11-28T13:00:41.700 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:41.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.822+0000 7fc4682e4700 1 Processor -- start 2023-11-28T13:00:41.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.822+0000 7fc4682e4700 1 -- start start 2023-11-28T13:00:41.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc4682e4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc46015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc4682e4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc46006a5e0 0x7fc46006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc4682e4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4600aee40 con 0x7fc46006e500 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc4682e4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46015bf10 con 0x7fc46006a5e0 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc4682e4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46006c930 con 0x7fc4600ae9d0 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc466881700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc46006a5e0 0x7fc46006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc46587f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc46015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc466080700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc46006e500 0x7fc46006e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:33764/0 (socket says 172.21.15.153:33764) 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.823+0000 7fc466080700 1 -- 172.21.15.153:0/2945128983 learned_addr learned my addr 172.21.15.153:0/2945128983 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:41.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc466881700 1 -- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 msgr2=0x7fc46015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc466881700 1 --2- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc46015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc466881700 1 -- 172.21.15.153:0/2945128983 >> v1:172.21.15.177:6789/0 conn(0x7fc46006e500 legacy=0x7fc46006e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc466881700 1 -- 172.21.15.153:0/2945128983 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46006cc10 con 0x7fc46006a5e0 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc46507e700 1 -- 172.21.15.153:0/2945128983 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852643195 0 0) 0x7fc4600aee40 con 0x7fc46006e500 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc46587f700 1 --2- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc46015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc466881700 1 --2- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46006aa00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc458009d90 tx=0x7fc45800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=780b8378179c2c11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc46507e700 1 -- 172.21.15.153:0/2945128983 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800e070 con 0x7fc46006a5e0 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc46507e700 1 -- 172.21.15.153:0/2945128983 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45800bb20 con 0x7fc46006a5e0 2023-11-28T13:00:41.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc46507e700 1 -- 172.21.15.153:0/2945128983 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458012430 con 0x7fc46006a5e0 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc4682e4700 1 -- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 msgr2=0x7fc46006aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc4682e4700 1 --2- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46006aa00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc458009d90 tx=0x7fc45800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc4682e4700 1 -- 172.21.15.153:0/2945128983 shutdown_connections 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc4682e4700 1 --2- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46006aa00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc4682e4700 1 --2- 172.21.15.153:0/2945128983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc46015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.824+0000 7fc4682e4700 1 -- 172.21.15.153:0/2945128983 >> 172.21.15.153:0/2945128983 conn(0x7fc4600b12b0 msgr2=0x7fc4600b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.825+0000 7fc4682e4700 1 -- 172.21.15.153:0/2945128983 shutdown_connections 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.825+0000 7fc4682e4700 1 -- 172.21.15.153:0/2945128983 wait complete. 2023-11-28T13:00:41.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 Processor -- start 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 -- start start 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46015ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46006e500 0x7fc4600ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc4600ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600ce190 con 0x7fc46006e500 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600ce5b0 con 0x7fc46006a5e0 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.826+0000 7fc4682e4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4600cd510 con 0x7fc4600ae9d0 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc466080700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46015ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46006e500 0x7fc4600ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46006e500 0x7fc4600ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38994/0 (socket says 172.21.15.153:38994) 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc466080700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46015ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:48458/0 (socket says 172.21.15.153:48458) 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 -- 172.21.15.153:0/3215040244 learned_addr learned my addr 172.21.15.153:0/3215040244 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc466881700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc4600ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 -- 172.21.15.153:0/3215040244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 msgr2=0x7fc4600ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc4600ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 -- 172.21.15.153:0/3215040244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 msgr2=0x7fc46015ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46015ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 -- 172.21.15.153:0/3215040244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c0094a0 con 0x7fc46006e500 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc46587f700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46006e500 0x7fc4600ce840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00ebe0 tx=0x7fc45c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc456ffd700 1 -- 172.21.15.153:0/3215040244 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00c850 con 0x7fc46006e500 2023-11-28T13:00:41.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc456ffd700 1 -- 172.21.15.153:0/3215040244 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c00c9b0 con 0x7fc46006e500 2023-11-28T13:00:41.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc456ffd700 1 -- 172.21.15.153:0/3215040244 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00cd40 con 0x7fc46006e500 2023-11-28T13:00:41.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc458009a70 con 0x7fc46006e500 2023-11-28T13:00:41.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.827+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46006cef0 con 0x7fc46006e500 2023-11-28T13:00:41.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.828+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46015ac80 con 0x7fc46006e500 2023-11-28T13:00:41.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.829+0000 7fc456ffd700 1 -- 172.21.15.153:0/3215040244 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc45c019030 con 0x7fc46006e500 2023-11-28T13:00:41.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.829+0000 7fc456ffd700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4440682f0 0x7fc44406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.829+0000 7fc456ffd700 1 -- 172.21.15.153:0/3215040244 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc45c085bc0 con 0x7fc46006e500 2023-11-28T13:00:41.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.829+0000 7fc466080700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4440682f0 0x7fc44406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.829+0000 7fc466080700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4440682f0 0x7fc44406a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc450007fd0 tx=0x7fc450007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:41.833+0000 7fc456ffd700 1 -- 172.21.15.153:0/3215040244 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc45c052420 con 0x7fc46006e500 2023-11-28T13:00:42.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.013+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4600ce2d0 con 0x7fc46006e500 2023-11-28T13:00:42.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.013+0000 7fc456ffd700 1 -- 172.21.15.153:0/3215040244 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc4600ce2d0 con 0x7fc46006e500 2023-11-28T13:00:42.014 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:42.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4440682f0 msgr2=0x7fc44406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4440682f0 0x7fc44406a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc450007fd0 tx=0x7fc450007de0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46006e500 msgr2=0x7fc4600ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46006e500 0x7fc4600ce840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00ebe0 tx=0x7fc45c00eef0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 shutdown_connections 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4600ae9d0 0x7fc4600ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc46006a5e0 0x7fc46015ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4440682f0 0x7fc44406a7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 --2- 172.21.15.153:0/3215040244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc46006e500 0x7fc4600ce840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.015+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 >> 172.21.15.153:0/3215040244 conn(0x7fc4600b12b0 msgr2=0x7fc4600b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.016+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 shutdown_connections 2023-11-28T13:00:42.018 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.016+0000 7fc4682e4700 1 -- 172.21.15.153:0/3215040244 wait complete. 2023-11-28T13:00:42.028 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:42.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.150+0000 7f2a15235700 1 Processor -- start 2023-11-28T13:00:42.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.150+0000 7f2a15235700 1 -- start start 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a15235700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2a100cd8c0 0x7f2a10140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a15235700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cdc90 0x7f2a100b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a15235700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1013ff00 con 0x7f2a100b2e70 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a15235700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a101404c0 con 0x7f2a100cd8c0 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a15235700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1015aa80 con 0x7f2a100cdc90 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a0ed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2a100cd8c0 0x7f2a10140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a0f59e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2a100b2e70 0x7f2a1006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57168/0 (socket says 172.21.15.153:57168) 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.151+0000 7f2a0f59e700 1 -- 172.21.15.153:0/333648994 learned_addr learned my addr 172.21.15.153:0/333648994 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:42.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0e59c700 1 --2- 172.21.15.153:0/333648994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cdc90 0x7f2a100b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0ed9d700 1 -- 172.21.15.153:0/333648994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cdc90 msgr2=0x7f2a100b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0ed9d700 1 --2- 172.21.15.153:0/333648994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cdc90 0x7f2a100b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0ed9d700 1 -- 172.21.15.153:0/333648994 >> v1:172.21.15.153:6789/0 conn(0x7f2a100b2e70 legacy=0x7f2a1006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0ed9d700 1 -- 172.21.15.153:0/333648994 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100b3310 con 0x7f2a100cd8c0 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0dd9b700 1 -- 172.21.15.153:0/333648994 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581374709 0 0) 0x7f2a1013ff00 con 0x7f2a100b2e70 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0e59c700 1 --2- 172.21.15.153:0/333648994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cdc90 0x7f2a100b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0ed9d700 1 --2- 172.21.15.153:0/333648994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100cd8c0 0x7f2a10140950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009820 tx=0x7f2a00009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278d0ebf9d52edb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0dd9b700 1 -- 172.21.15.153:0/333648994 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00005e30 con 0x7f2a100cd8c0 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0dd9b700 1 -- 172.21.15.153:0/333648994 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a00006030 con 0x7f2a100cd8c0 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a0dd9b700 1 -- 172.21.15.153:0/333648994 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a000063e0 con 0x7f2a100cd8c0 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a15235700 1 -- 172.21.15.153:0/333648994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100cd8c0 msgr2=0x7f2a10140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a15235700 1 --2- 172.21.15.153:0/333648994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100cd8c0 0x7f2a10140950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009820 tx=0x7f2a00009b30 comp rx=0 tx=0).stop 2023-11-28T13:00:42.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a15235700 1 -- 172.21.15.153:0/333648994 shutdown_connections 2023-11-28T13:00:42.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.152+0000 7f2a15235700 1 --2- 172.21.15.153:0/333648994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100cd8c0 0x7f2a10140950 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.153+0000 7f2a15235700 1 --2- 172.21.15.153:0/333648994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cdc90 0x7f2a100b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.153+0000 7f2a15235700 1 -- 172.21.15.153:0/333648994 >> 172.21.15.153:0/333648994 conn(0x7f2a1006aed0 msgr2=0x7f2a1006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.153+0000 7f2a15235700 1 -- 172.21.15.153:0/333648994 shutdown_connections 2023-11-28T13:00:42.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.153+0000 7f2a15235700 1 -- 172.21.15.153:0/333648994 wait complete. 2023-11-28T13:00:42.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.154+0000 7f2a15235700 1 Processor -- start 2023-11-28T13:00:42.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.154+0000 7f2a15235700 1 -- start start 2023-11-28T13:00:42.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a15235700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100b2e70 0x7f2a10155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a15235700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cd8c0 0x7f2a100b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a15235700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a100cdc90 0x7f2a10156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a15235700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a100b1d20 con 0x7f2a100b2e70 2023-11-28T13:00:42.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a15235700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a10155740 con 0x7f2a100cdc90 2023-11-28T13:00:42.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a15235700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a100b13c0 con 0x7f2a100cd8c0 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100b2e70 0x7f2a10155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100b2e70 0x7f2a10155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39006/0 (socket says 172.21.15.153:39006) 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 -- 172.21.15.153:0/571149451 learned_addr learned my addr 172.21.15.153:0/571149451 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0f59e700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a100cdc90 0x7f2a10156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0e59c700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cd8c0 0x7f2a100b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 -- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cd8c0 msgr2=0x7f2a100b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cd8c0 0x7f2a100b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 -- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a100cdc90 msgr2=0x7f2a10156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a100cdc90 0x7f2a10156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 -- 172.21.15.153:0/571149451 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00009430 con 0x7f2a100b2e70 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.155+0000 7f2a0ed9d700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100b2e70 0x7f2a10155200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a100b3aa0 tx=0x7f2a00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.156+0000 7f29f77fe700 1 -- 172.21.15.153:0/571149451 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00006950 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.156+0000 7f29f77fe700 1 -- 172.21.15.153:0/571149451 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a00020070 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.156+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a10140150 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.156+0000 7f29f77fe700 1 -- 172.21.15.153:0/571149451 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00025670 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.156+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a100b35f0 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.157+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a100ce100 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.157+0000 7f29f77fe700 1 -- 172.21.15.153:0/571149451 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a0002b020 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.157+0000 7f29f77fe700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29fc0682f0 0x7f29fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.157+0000 7f29f77fe700 1 -- 172.21.15.153:0/571149451 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a000875f0 con 0x7f2a100b2e70 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.157+0000 7f2a0e59c700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29fc0682f0 0x7f29fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.158+0000 7f2a0e59c700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29fc0682f0 0x7f29fc06a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a100c3520 tx=0x7f2a04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.162+0000 7f29f77fe700 1 -- 172.21.15.153:0/571149451 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2a00053d60 con 0x7f2a100b2e70 2023-11-28T13:00:42.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.344+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a100b1f90 con 0x7f2a100b2e70 2023-11-28T13:00:42.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.344+0000 7f29f77fe700 1 -- 172.21.15.153:0/571149451 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a00057380 con 0x7f2a100b2e70 2023-11-28T13:00:42.345 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:42.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29fc0682f0 msgr2=0x7f29fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29fc0682f0 0x7f29fc06a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a100c3520 tx=0x7f2a04008040 comp rx=0 tx=0).stop 2023-11-28T13:00:42.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100b2e70 msgr2=0x7f2a10155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100b2e70 0x7f2a10155200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2a100b3aa0 tx=0x7f2a00004750 comp rx=0 tx=0).stop 2023-11-28T13:00:42.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 shutdown_connections 2023-11-28T13:00:42.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29fc0682f0 0x7f29fc06a7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a100b2e70 0x7f2a10155200 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a100cdc90 0x7f2a10156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 --2- 172.21.15.153:0/571149451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a100cd8c0 0x7f2a100b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 >> 172.21.15.153:0/571149451 conn(0x7f2a1006aed0 msgr2=0x7f2a10159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 shutdown_connections 2023-11-28T13:00:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.346+0000 7f2a15235700 1 -- 172.21.15.153:0/571149451 wait complete. 2023-11-28T13:00:42.360 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:42.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.483+0000 7fba5b382700 1 Processor -- start 2023-11-28T13:00:42.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.484+0000 7fba5b382700 1 -- start start 2023-11-28T13:00:42.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.484+0000 7fba5b382700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540d01b0 0x7fba540d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.484+0000 7fba5b382700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba540b20b0 0x7fba540d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.484+0000 7fba5b382700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba540b0230 con 0x7fba540b29b0 2023-11-28T13:00:42.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.484+0000 7fba5b382700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5415cec0 con 0x7fba540b20b0 2023-11-28T13:00:42.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5b382700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b4570 con 0x7fba540d01b0 2023-11-28T13:00:42.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba540b20b0 0x7fba540d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba540b20b0 0x7fba540d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39016/0 (socket says 172.21.15.153:39016) 2023-11-28T13:00:42.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 -- 172.21.15.153:0/1341403768 learned_addr learned my addr 172.21.15.153:0/1341403768 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:42.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5911e700 1 --2- 172.21.15.153:0/1341403768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540d01b0 0x7fba540d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 -- 172.21.15.153:0/1341403768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540d01b0 msgr2=0x7fba540d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:42.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 --2- 172.21.15.153:0/1341403768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540d01b0 0x7fba540d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 -- 172.21.15.153:0/1341403768 >> v1:172.21.15.177:6789/0 conn(0x7fba540b29b0 legacy=0x7fba540b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 -- 172.21.15.153:0/1341403768 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540b4850 con 0x7fba540b20b0 2023-11-28T13:00:42.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba5891d700 1 --2- 172.21.15.153:0/1341403768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba540d1160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba440098f0 tx=0x7fba44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d91f76a1df1ffa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba4bfff700 1 -- 172.21.15.153:0/1341403768 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44011830 con 0x7fba540b20b0 2023-11-28T13:00:42.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba4bfff700 1 -- 172.21.15.153:0/1341403768 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba44011990 con 0x7fba540b20b0 2023-11-28T13:00:42.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.485+0000 7fba4bfff700 1 -- 172.21.15.153:0/1341403768 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44011d20 con 0x7fba540b20b0 2023-11-28T13:00:42.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 -- 172.21.15.153:0/1341403768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 msgr2=0x7fba540d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 --2- 172.21.15.153:0/1341403768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba540d1160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba440098f0 tx=0x7fba44009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 -- 172.21.15.153:0/1341403768 shutdown_connections 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 --2- 172.21.15.153:0/1341403768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540d01b0 0x7fba540d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 --2- 172.21.15.153:0/1341403768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba540d1160 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 -- 172.21.15.153:0/1341403768 >> 172.21.15.153:0/1341403768 conn(0x7fba5409c3f0 msgr2=0x7fba540d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 -- 172.21.15.153:0/1341403768 shutdown_connections 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.486+0000 7fba5b382700 1 -- 172.21.15.153:0/1341403768 wait complete. 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.487+0000 7fba5b382700 1 Processor -- start 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.487+0000 7fba5b382700 1 -- start start 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5b382700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba5415fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5b382700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540b29b0 0x7fba540cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5b382700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba540d01b0 0x7fba540cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5b382700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b7870 con 0x7fba540b20b0 2023-11-28T13:00:42.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5b382700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54160010 con 0x7fba540d01b0 2023-11-28T13:00:42.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5b382700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540ccea0 con 0x7fba540b29b0 2023-11-28T13:00:42.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba5415fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba5415fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39020/0 (socket says 172.21.15.153:39020) 2023-11-28T13:00:42.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 -- 172.21.15.153:0/3577922269 learned_addr learned my addr 172.21.15.153:0/3577922269 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:42.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5891d700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540b29b0 0x7fba540cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5991f700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba540d01b0 0x7fba540cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 -- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540b29b0 msgr2=0x7fba540cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540b29b0 0x7fba540cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 -- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba540d01b0 msgr2=0x7fba540cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba540d01b0 0x7fba540cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 -- 172.21.15.153:0/3577922269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009550 con 0x7fba540b20b0 2023-11-28T13:00:42.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5991f700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba540d01b0 0x7fba540cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:42.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.488+0000 7fba5911e700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba5415fad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba5000e870 tx=0x7fba5000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.490+0000 7fba49ffb700 1 -- 172.21.15.153:0/3577922269 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50009590 con 0x7fba540b20b0 2023-11-28T13:00:42.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.490+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba540afec0 con 0x7fba540b20b0 2023-11-28T13:00:42.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.490+0000 7fba49ffb700 1 -- 172.21.15.153:0/3577922269 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba50013680 con 0x7fba540b20b0 2023-11-28T13:00:42.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.490+0000 7fba49ffb700 1 -- 172.21.15.153:0/3577922269 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500139f0 con 0x7fba540b20b0 2023-11-28T13:00:42.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.491+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba540afbe0 con 0x7fba540b20b0 2023-11-28T13:00:42.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.492+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540b2600 con 0x7fba540b20b0 2023-11-28T13:00:42.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.492+0000 7fba49ffb700 1 -- 172.21.15.153:0/3577922269 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba50019030 con 0x7fba540b20b0 2023-11-28T13:00:42.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.492+0000 7fba49ffb700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba380682f0 0x7fba3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.492+0000 7fba49ffb700 1 -- 172.21.15.153:0/3577922269 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba50085ab0 con 0x7fba540b20b0 2023-11-28T13:00:42.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.492+0000 7fba5891d700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba380682f0 0x7fba3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.493+0000 7fba5891d700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba380682f0 0x7fba3806a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba44009c40 tx=0x7fba44008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.497+0000 7fba49ffb700 1 -- 172.21.15.153:0/3577922269 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba50052290 con 0x7fba540b20b0 2023-11-28T13:00:42.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.680+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba540cccf0 con 0x7fba540b20b0 2023-11-28T13:00:42.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.681+0000 7fba49ffb700 1 -- 172.21.15.153:0/3577922269 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba50017020 con 0x7fba540b20b0 2023-11-28T13:00:42.682 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba380682f0 msgr2=0x7fba3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba380682f0 0x7fba3806a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba44009c40 tx=0x7fba44008030 comp rx=0 tx=0).stop 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 msgr2=0x7fba5415fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba5415fad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba5000e870 tx=0x7fba5000eb80 comp rx=0 tx=0).stop 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 shutdown_connections 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba380682f0 0x7fba3806a7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba540b20b0 0x7fba5415fad0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba540d01b0 0x7fba540cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 --2- 172.21.15.153:0/3577922269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba540b29b0 0x7fba540cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 >> 172.21.15.153:0/3577922269 conn(0x7fba5409c3f0 msgr2=0x7fba5406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 shutdown_connections 2023-11-28T13:00:42.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.683+0000 7fba5b382700 1 -- 172.21.15.153:0/3577922269 wait complete. 2023-11-28T13:00:42.696 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:42.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.815+0000 7fe70343f700 1 Processor -- start 2023-11-28T13:00:42.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.815+0000 7fe70343f700 1 -- start start 2023-11-28T13:00:42.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe70343f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe6fc06dee0 0x7fe6fc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe70343f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc15d060 con 0x7fe6fc071040 2023-11-28T13:00:42.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe70343f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc15d340 con 0x7fe6fc160540 2023-11-28T13:00:42.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe70343f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc15d620 con 0x7fe6fc06dee0 2023-11-28T13:00:42.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe7011db700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe6fc071040 0x7fe6fc071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57192/0 (socket says 172.21.15.153:57192) 2023-11-28T13:00:42.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe7011db700 1 -- 172.21.15.153:0/838372001 learned_addr learned my addr 172.21.15.153:0/838372001 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:42.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312843326 0 0) 0x7fe6fc15d060 con 0x7fe6fc071040 2023-11-28T13:00:42.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe7009da700 1 --2- 172.21.15.153:0/838372001 >> v2:172.21.15.177:3300/0 conn(0x7fe6fc06dee0 0x7fe6fc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e80035b0 con 0x7fe6fc071040 2023-11-28T13:00:42.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1328633884 0 0) 0x7fe6e80035b0 con 0x7fe6fc071040 2023-11-28T13:00:42.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 >> v2:172.21.15.177:3300/0 conn(0x7fe6fc06dee0 msgr2=0x7fe6fc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:42.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 --2- 172.21.15.153:0/838372001 >> v2:172.21.15.177:3300/0 conn(0x7fe6fc06dee0 0x7fe6fc06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 >> v1:172.21.15.177:6789/0 conn(0x7fe6fc160540 legacy=0x7fe6fc160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc15d900 con 0x7fe6fc071040 2023-11-28T13:00:42.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fe6f8001670 con 0x7fe6fc071040 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 >> v1:172.21.15.153:6789/0 conn(0x7fe6fc071040 legacy=0x7fe6fc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.816+0000 7fe6f3fff700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e8003a00 0x7fe6e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6e8004850 0x7fe6e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8005200 0x7fe6e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc15d620 con 0x7fe6e8003a00 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e800c190 con 0x7fe6e8005200 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e800c350 con 0x7fe6e8004850 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7011db700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e8003a00 0x7fe6e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7019dc700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8005200 0x7fe6e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7009da700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6e8004850 0x7fe6e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7011db700 1 -- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6e8004850 msgr2=0x7fe6e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7011db700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6e8004850 0x7fe6e8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7011db700 1 -- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8005200 msgr2=0x7fe6e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7011db700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8005200 0x7fe6e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7019dc700 1 -- 172.21.15.153:0/838372001 reap_dead start 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7011db700 1 -- 172.21.15.153:0/838372001 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80035b0 con 0x7fe6e8003a00 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7009da700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6e8004850 0x7fe6e8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe7011db700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e8003a00 0x7fe6e8003dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8001eb0 tx=0x7fe6e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8d6685885963c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800ce00 con 0x7fe6e8003a00 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8005960 con 0x7fe6e8003a00 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.817+0000 7fe6f3fff700 1 -- 172.21.15.153:0/838372001 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8005cf0 con 0x7fe6e8003a00 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.818+0000 7fe70343f700 1 -- 172.21.15.153:0/838372001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e8003a00 msgr2=0x7fe6e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.818+0000 7fe70343f700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e8003a00 0x7fe6e8003dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8001eb0 tx=0x7fe6e8005880 comp rx=0 tx=0).stop 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.818+0000 7fe70343f700 1 -- 172.21.15.153:0/838372001 shutdown_connections 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.818+0000 7fe70343f700 1 --2- 172.21.15.153:0/838372001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e8003a00 0x7fe6e8003dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.818+0000 7fe70343f700 1 -- 172.21.15.153:0/838372001 >> 172.21.15.153:0/838372001 conn(0x7fe6fc0d0000 msgr2=0x7fe6fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.818+0000 7fe70343f700 1 -- 172.21.15.153:0/838372001 shutdown_connections 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.818+0000 7fe70343f700 1 -- 172.21.15.153:0/838372001 wait complete. 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.819+0000 7fe70343f700 1 Processor -- start 2023-11-28T13:00:42.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.819+0000 7fe70343f700 1 -- start start 2023-11-28T13:00:42.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe70343f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8003a00 0x7fe6fc0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe70343f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6fc0b3e50 0x7fe6fc1567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe70343f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc1560b0 0x7fe6fc15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe70343f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0c67d0 con 0x7fe6fc1560b0 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe70343f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc156cf0 con 0x7fe6e8003a00 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe70343f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc156f20 con 0x7fe6fc0b3e50 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc1560b0 0x7fe6fc15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc1560b0 0x7fe6fc15e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39044/0 (socket says 172.21.15.153:39044) 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 -- 172.21.15.153:0/813566666 learned_addr learned my addr 172.21.15.153:0/813566666 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7009da700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6fc0b3e50 0x7fe6fc1567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7011db700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8003a00 0x7fe6fc0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 -- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6fc0b3e50 msgr2=0x7fe6fc1567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6fc0b3e50 0x7fe6fc1567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 -- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8003a00 msgr2=0x7fe6fc0c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8003a00 0x7fe6fc0c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 -- 172.21.15.153:0/813566666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc06a270 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7011db700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8003a00 0x7fe6fc0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.820+0000 7fe7019dc700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc1560b0 0x7fe6fc15e0f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4001520 tx=0x7fe6f400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.821+0000 7fe6f1ffb700 1 -- 172.21.15.153:0/813566666 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4003c10 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.821+0000 7fe6f1ffb700 1 -- 172.21.15.153:0/813566666 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4003d70 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.821+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc15ccd0 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.821+0000 7fe6f1ffb700 1 -- 172.21.15.153:0/813566666 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400dbb0 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.821+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc1574e0 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.822+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc06a400 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.822+0000 7fe6f1ffb700 1 -- 172.21.15.153:0/813566666 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe6f400dd10 con 0x7fe6fc1560b0 2023-11-28T13:00:42.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.822+0000 7fe6f1ffb700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6e00682f0 0x7fe6e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.822+0000 7fe6f1ffb700 1 -- 172.21.15.153:0/813566666 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6f40860a0 con 0x7fe6fc1560b0 2023-11-28T13:00:42.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.823+0000 7fe7011db700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6e00682f0 0x7fe6e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.823+0000 7fe7011db700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6e00682f0 0x7fe6e006a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:42.827+0000 7fe6f1ffb700 1 -- 172.21.15.153:0/813566666 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe6fc06a400 con 0x7fe6fc1560b0 2023-11-28T13:00:43.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.011+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6fc06a400 con 0x7fe6fc1560b0 2023-11-28T13:00:43.013 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.012+0000 7fe6f1ffb700 1 -- 172.21.15.153:0/813566666 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fe6fc06a400 con 0x7fe6fc1560b0 2023-11-28T13:00:43.013 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:43.014 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:43.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6e00682f0 msgr2=0x7fe6e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6e00682f0 0x7fe6e006a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f8007000 comp rx=0 tx=0).stop 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc1560b0 msgr2=0x7fe6fc15e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc1560b0 0x7fe6fc15e0f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4001520 tx=0x7fe6f400d430 comp rx=0 tx=0).stop 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 shutdown_connections 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6e00682f0 0x7fe6e006a7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6fc1560b0 0x7fe6fc15e0f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6e8003a00 0x7fe6fc0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 --2- 172.21.15.153:0/813566666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe6fc0b3e50 0x7fe6fc1567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.014+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 >> 172.21.15.153:0/813566666 conn(0x7fe6fc0d0000 msgr2=0x7fe6fc157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.015+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 shutdown_connections 2023-11-28T13:00:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.015+0000 7fe70343f700 1 -- 172.21.15.153:0/813566666 wait complete. 2023-11-28T13:00:43.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:43.029 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:43.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.150+0000 7f869eb40700 1 Processor -- start 2023-11-28T13:00:43.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.151+0000 7f869eb40700 1 -- start start 2023-11-28T13:00:43.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.151+0000 7f869eb40700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f86980cd8c0 0x7f8698140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.151+0000 7f869eb40700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f86980cdc90 0x7f86980b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.151+0000 7f869eb40700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980b2e70 0x7f869806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.151+0000 7f869eb40700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869813ff00 con 0x7f86980cdc90 2023-11-28T13:00:43.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.151+0000 7f869eb40700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86981404c0 con 0x7f86980cd8c0 2023-11-28T13:00:43.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.151+0000 7f869eb40700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869815aa80 con 0x7f86980b2e70 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f86980cdc90 0x7f86980b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f86980cdc90 0x7f86980b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39050/0 (socket says 172.21.15.153:39050) 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 -- 172.21.15.153:0/514301297 learned_addr learned my addr 172.21.15.153:0/514301297 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f869c8dc700 1 --2- 172.21.15.153:0/514301297 >> v2:172.21.15.177:3300/0 conn(0x7f86980cd8c0 0x7f8698140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f869d0dd700 1 --2- 172.21.15.153:0/514301297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980b2e70 0x7f869806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 -- 172.21.15.153:0/514301297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980b2e70 msgr2=0x7f869806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 --2- 172.21.15.153:0/514301297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980b2e70 0x7f869806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 -- 172.21.15.153:0/514301297 >> v2:172.21.15.177:3300/0 conn(0x7f86980cd8c0 msgr2=0x7f8698140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 --2- 172.21.15.153:0/514301297 >> v2:172.21.15.177:3300/0 conn(0x7f86980cd8c0 0x7f8698140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 -- 172.21.15.153:0/514301297 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86980b3310 con 0x7f86980cdc90 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f869d0dd700 1 --2- 172.21.15.153:0/514301297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980b2e70 0x7f869806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f8697fff700 1 --2- 172.21.15.153:0/514301297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cdc90 0x7f86980b2930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8698070a20 tx=0x7f8684009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3b952a72913db7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f86977fe700 1 -- 172.21.15.153:0/514301297 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400b870 con 0x7f86980cdc90 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f86977fe700 1 -- 172.21.15.153:0/514301297 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868400b9d0 con 0x7f86980cdc90 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.152+0000 7f86977fe700 1 -- 172.21.15.153:0/514301297 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400bd60 con 0x7f86980cdc90 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 -- 172.21.15.153:0/514301297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cdc90 msgr2=0x7f86980b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 --2- 172.21.15.153:0/514301297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cdc90 0x7f86980b2930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8698070a20 tx=0x7f8684009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 -- 172.21.15.153:0/514301297 shutdown_connections 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 --2- 172.21.15.153:0/514301297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980b2e70 0x7f869806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 --2- 172.21.15.153:0/514301297 >> v2:172.21.15.177:3300/0 conn(0x7f86980cd8c0 0x7f8698140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 --2- 172.21.15.153:0/514301297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cdc90 0x7f86980b2930 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 -- 172.21.15.153:0/514301297 >> 172.21.15.153:0/514301297 conn(0x7f869806aed0 msgr2=0x7f869806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 -- 172.21.15.153:0/514301297 shutdown_connections 2023-11-28T13:00:43.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.153+0000 7f869eb40700 1 -- 172.21.15.153:0/514301297 wait complete. 2023-11-28T13:00:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.154+0000 7f869eb40700 1 Processor -- start 2023-11-28T13:00:43.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.154+0000 7f869eb40700 1 -- start start 2023-11-28T13:00:43.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.155+0000 7f869eb40700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86980b2e70 0x7f8698154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.155+0000 7f869eb40700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cd8c0 0x7f86980b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869eb40700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980cdc90 0x7f86980b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869eb40700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c9b80 con 0x7f86980cd8c0 2023-11-28T13:00:43.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869eb40700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c8f00 con 0x7f86980b2e70 2023-11-28T13:00:43.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869eb40700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c91e0 con 0x7f86980cdc90 2023-11-28T13:00:43.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cd8c0 0x7f86980b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cd8c0 0x7f86980b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39056/0 (socket says 172.21.15.153:39056) 2023-11-28T13:00:43.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 -- 172.21.15.153:0/959886247 learned_addr learned my addr 172.21.15.153:0/959886247 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:43.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869c8dc700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86980b2e70 0x7f8698154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869d0dd700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980cdc90 0x7f86980b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 -- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980cdc90 msgr2=0x7f86980b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980cdc90 0x7f86980b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 -- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86980b2e70 msgr2=0x7f8698154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86980b2e70 0x7f8698154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 -- 172.21.15.153:0/959886247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684009550 con 0x7f86980cd8c0 2023-11-28T13:00:43.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f8697fff700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cd8c0 0x7f86980b2620 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f8684004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f86957fa700 1 -- 172.21.15.153:0/959886247 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684004480 con 0x7f86980cd8c0 2023-11-28T13:00:43.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f86957fa700 1 -- 172.21.15.153:0/959886247 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86840045e0 con 0x7f86980cd8c0 2023-11-28T13:00:43.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8698140150 con 0x7f86980cd8c0 2023-11-28T13:00:43.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f86957fa700 1 -- 172.21.15.153:0/959886247 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840038e0 con 0x7f86980cd8c0 2023-11-28T13:00:43.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.156+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86980b35f0 con 0x7f86980cd8c0 2023-11-28T13:00:43.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.157+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86980ce100 con 0x7f86980cd8c0 2023-11-28T13:00:43.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.158+0000 7f86957fa700 1 -- 172.21.15.153:0/959886247 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f868400f050 con 0x7f86980cd8c0 2023-11-28T13:00:43.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.158+0000 7f86957fa700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f867c0682f0 0x7f867c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.158+0000 7f86957fa700 1 -- 172.21.15.153:0/959886247 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8684087430 con 0x7f86980cd8c0 2023-11-28T13:00:43.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.158+0000 7f869c8dc700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f867c0682f0 0x7f867c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.158+0000 7f869c8dc700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f867c0682f0 0x7f867c06a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f868c006fd0 tx=0x7f868c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.162+0000 7f86957fa700 1 -- 172.21.15.153:0/959886247 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8684053c10 con 0x7f86980cd8c0 2023-11-28T13:00:43.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.343+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8698068690 con 0x7f86980cd8c0 2023-11-28T13:00:43.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.343+0000 7f86957fa700 1 -- 172.21.15.153:0/959886247 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8684057230 con 0x7f86980cd8c0 2023-11-28T13:00:43.345 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:43.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.345+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f867c0682f0 msgr2=0x7f867c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.345+0000 7f869eb40700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f867c0682f0 0x7f867c06a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f868c006fd0 tx=0x7f868c008040 comp rx=0 tx=0).stop 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.345+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cd8c0 msgr2=0x7f86980b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.345+0000 7f869eb40700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cd8c0 0x7f86980b2620 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f8684004750 comp rx=0 tx=0).stop 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.345+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 shutdown_connections 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.345+0000 7f869eb40700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86980cdc90 0x7f86980b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.346+0000 7f869eb40700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86980b2e70 0x7f8698154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.346+0000 7f869eb40700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f867c0682f0 0x7f867c06a7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.346+0000 7f869eb40700 1 --2- 172.21.15.153:0/959886247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86980cd8c0 0x7f86980b2620 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.346+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 >> 172.21.15.153:0/959886247 conn(0x7f869806aed0 msgr2=0x7f869815a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.346+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 shutdown_connections 2023-11-28T13:00:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.346+0000 7f869eb40700 1 -- 172.21.15.153:0/959886247 wait complete. 2023-11-28T13:00:43.358 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.483+0000 7f5cfab4d700 1 Processor -- start 2023-11-28T13:00:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.483+0000 7f5cfab4d700 1 -- start start 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cfab4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 0x7f5cf415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cfab4d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5cf406a5e0 0x7f5cf406aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cfab4d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf40aee40 con 0x7f5cf40ae9d0 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cfab4d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf415bf10 con 0x7f5cf406a5e0 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cfab4d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf406c930 con 0x7f5cf406e500 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf3fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5cf40ae9d0 0x7f5cf406e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57228/0 (socket says 172.21.15.153:57228) 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf3fff700 1 -- 172.21.15.153:0/381638089 learned_addr learned my addr 172.21.15.153:0/381638089 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf88e9700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 0x7f5cf415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf90ea700 1 --2- 172.21.15.153:0/381638089 >> v2:172.21.15.177:3300/0 conn(0x7f5cf406a5e0 0x7f5cf406aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926868246 0 0) 0x7f5cf40aee40 con 0x7f5cf40ae9d0 2023-11-28T13:00:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce40035b0 con 0x7f5cf40ae9d0 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 648927207 0 0) 0x7f5ce40035b0 con 0x7f5cf40ae9d0 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 msgr2=0x7f5cf415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 0x7f5cf415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 >> v2:172.21.15.177:3300/0 conn(0x7f5cf406a5e0 msgr2=0x7f5cf406aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 --2- 172.21.15.153:0/381638089 >> v2:172.21.15.177:3300/0 conn(0x7f5cf406a5e0 0x7f5cf406aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf406cc10 con 0x7f5cf40ae9d0 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.484+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f5ce0001670 con 0x7f5cf40ae9d0 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 >> v1:172.21.15.153:6789/0 conn(0x7f5cf40ae9d0 legacy=0x7f5cf406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce4003a00 0x7f5ce4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5ce4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 0x7f5ce4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf415bf10 con 0x7f5ce4004850 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf406c930 con 0x7f5ce4003a00 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce400ba90 con 0x7f5cf406e500 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf3fff700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5ce4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf88e9700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce4003a00 0x7f5ce4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf90ea700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 0x7f5ce4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf3fff700 1 -- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 msgr2=0x7f5ce4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf3fff700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf406e500 0x7f5ce4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf3fff700 1 -- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce4003a00 msgr2=0x7f5ce4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf3fff700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce4003a00 0x7f5ce4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf3fff700 1 -- 172.21.15.153:0/381638089 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40035b0 con 0x7f5ce4004850 2023-11-28T13:00:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf3fff700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5ce4004cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c370 tx=0x7f5ce4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d455586f0cefc760 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0007c60 con 0x7f5ce4004850 2023-11-28T13:00:43.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce0005960 con 0x7f5ce4004850 2023-11-28T13:00:43.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.485+0000 7f5cf37fe700 1 -- 172.21.15.153:0/381638089 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0005d10 con 0x7f5ce4004850 2023-11-28T13:00:43.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.486+0000 7f5cfab4d700 1 -- 172.21.15.153:0/381638089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 msgr2=0x7f5ce4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.486+0000 7f5cfab4d700 1 --2- 172.21.15.153:0/381638089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5ce4004cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c370 tx=0x7f5ce4005880 comp rx=0 tx=0).stop 2023-11-28T13:00:43.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.486+0000 7f5cf90ea700 1 -- 172.21.15.153:0/381638089 reap_dead start 2023-11-28T13:00:43.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.486+0000 7f5cfab4d700 1 -- 172.21.15.153:0/381638089 shutdown_connections 2023-11-28T13:00:43.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.486+0000 7f5cfab4d700 1 -- 172.21.15.153:0/381638089 >> 172.21.15.153:0/381638089 conn(0x7f5cf40b12b0 msgr2=0x7f5cf40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.486+0000 7f5cfab4d700 1 -- 172.21.15.153:0/381638089 shutdown_connections 2023-11-28T13:00:43.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.486+0000 7f5cfab4d700 1 -- 172.21.15.153:0/381638089 wait complete. 2023-11-28T13:00:43.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.487+0000 7f5cfab4d700 1 Processor -- start 2023-11-28T13:00:43.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.487+0000 7f5cfab4d700 1 -- start start 2023-11-28T13:00:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cfab4d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5cf4144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cfab4d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf415b190 0x7f5cf41f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cfab4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf415b600 0x7f5cf415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cfab4d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40c71c0 con 0x7f5ce4004850 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cfab4d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41f4980 con 0x7f5cf415b190 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cfab4d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41f43c0 con 0x7f5cf415b600 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5cf4144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5cf4144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39074/0 (socket says 172.21.15.153:39074) 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 -- 172.21.15.153:0/3531941768 learned_addr learned my addr 172.21.15.153:0/3531941768 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf90ea700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf415b600 0x7f5cf415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf3fff700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf415b190 0x7f5cf41f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 -- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf415b600 msgr2=0x7f5cf415e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf415b600 0x7f5cf415e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 -- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf415b190 msgr2=0x7f5cf41f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf415b190 0x7f5cf41f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf88e9700 1 -- 172.21.15.153:0/3531941768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf406cef0 con 0x7f5ce4004850 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.488+0000 7f5cf90ea700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf415b600 0x7f5cf415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.489+0000 7f5cf88e9700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5cf4144460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80119d0 tx=0x7f5ce8011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.489+0000 7f5cf17fa700 1 -- 172.21.15.153:0/3531941768 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800f940 con 0x7f5ce4004850 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.489+0000 7f5cf17fa700 1 -- 172.21.15.153:0/3531941768 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce800faa0 con 0x7f5ce4004850 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.489+0000 7f5cf17fa700 1 -- 172.21.15.153:0/3531941768 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800fe30 con 0x7f5ce4004850 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.489+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf40664c0 con 0x7f5ce4004850 2023-11-28T13:00:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.489+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf415ac80 con 0x7f5ce4004850 2023-11-28T13:00:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.490+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf4065fc0 con 0x7f5ce4004850 2023-11-28T13:00:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.490+0000 7f5cf17fa700 1 -- 172.21.15.153:0/3531941768 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ce8014440 con 0x7f5ce4004850 2023-11-28T13:00:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.491+0000 7f5cf17fa700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cd80682f0 0x7f5cd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.491+0000 7f5cf17fa700 1 -- 172.21.15.153:0/3531941768 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ce8086420 con 0x7f5ce4004850 2023-11-28T13:00:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.491+0000 7f5cf3fff700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cd80682f0 0x7f5cd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.491+0000 7f5cf3fff700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cd80682f0 0x7f5cd806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5cf41f39b0 tx=0x7f5ce000a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.496+0000 7f5cf17fa700 1 -- 172.21.15.153:0/3531941768 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ce8052cd0 con 0x7f5ce4004850 2023-11-28T13:00:43.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.682+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5cf41442b0 con 0x7f5ce4004850 2023-11-28T13:00:43.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.682+0000 7f5cf17fa700 1 -- 172.21.15.153:0/3531941768 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5ce8022020 con 0x7f5ce4004850 2023-11-28T13:00:43.683 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cd80682f0 msgr2=0x7f5cd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cd80682f0 0x7f5cd806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5cf41f39b0 tx=0x7f5ce000a000 comp rx=0 tx=0).stop 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 msgr2=0x7f5cf4144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5cf4144460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80119d0 tx=0x7f5ce8011d90 comp rx=0 tx=0).stop 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 shutdown_connections 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cd80682f0 0x7f5cd806a7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ce4004850 0x7f5cf4144460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf415b600 0x7f5cf415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 --2- 172.21.15.153:0/3531941768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf415b190 0x7f5cf41f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 >> 172.21.15.153:0/3531941768 conn(0x7f5cf40b12b0 msgr2=0x7f5cf406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 shutdown_connections 2023-11-28T13:00:43.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.684+0000 7f5cfab4d700 1 -- 172.21.15.153:0/3531941768 wait complete. 2023-11-28T13:00:43.697 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:43.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.820+0000 7ffb660f3700 1 Processor -- start 2023-11-28T13:00:43.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.820+0000 7ffb660f3700 1 -- start start 2023-11-28T13:00:43.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb660f3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600b0670 0x7ffb6015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb660f3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb600cc730 0x7ffb600b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb660f3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb600ce100 con 0x7ffb600ad0a0 2023-11-28T13:00:43.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb660f3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb600acc10 con 0x7ffb600cc730 2023-11-28T13:00:43.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb660f3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015dd30 con 0x7ffb600b0670 2023-11-28T13:00:43.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb600cc730 0x7ffb600b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb600cc730 0x7ffb600b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39084/0 (socket says 172.21.15.153:39084) 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ffb600ad0a0 0x7ffb600c91b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57250/0 (socket says 172.21.15.153:57250) 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5effd700 1 -- 172.21.15.153:0/1915480356 learned_addr learned my addr 172.21.15.153:0/1915480356 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5f7fe700 1 --2- 172.21.15.153:0/1915480356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600b0670 0x7ffb6015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5effd700 1 -- 172.21.15.153:0/1915480356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600b0670 msgr2=0x7ffb6015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5effd700 1 --2- 172.21.15.153:0/1915480356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600b0670 0x7ffb6015d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5effd700 1 -- 172.21.15.153:0/1915480356 >> v1:172.21.15.153:6789/0 conn(0x7ffb600ad0a0 legacy=0x7ffb600c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5e7fc700 1 -- 172.21.15.153:0/1915480356 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882540767 0 0) 0x7ffb600ce100 con 0x7ffb600ad0a0 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.821+0000 7ffb5effd700 1 -- 172.21.15.153:0/1915480356 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600ac8c0 con 0x7ffb600cc730 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb5effd700 1 --2- 172.21.15.153:0/1915480356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cc730 0x7ffb600b0a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb500098f0 tx=0x7ffb50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe78a6db4c5b6c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb5e7fc700 1 -- 172.21.15.153:0/1915480356 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000b8a0 con 0x7ffb600cc730 2023-11-28T13:00:43.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb5e7fc700 1 -- 172.21.15.153:0/1915480356 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5000ba00 con 0x7ffb600cc730 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb5e7fc700 1 -- 172.21.15.153:0/1915480356 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000bd90 con 0x7ffb600cc730 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 -- 172.21.15.153:0/1915480356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cc730 msgr2=0x7ffb600b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 --2- 172.21.15.153:0/1915480356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cc730 0x7ffb600b0a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb500098f0 tx=0x7ffb50009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 -- 172.21.15.153:0/1915480356 shutdown_connections 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 --2- 172.21.15.153:0/1915480356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600b0670 0x7ffb6015d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 --2- 172.21.15.153:0/1915480356 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600cc730 0x7ffb600b0a40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 -- 172.21.15.153:0/1915480356 >> 172.21.15.153:0/1915480356 conn(0x7ffb600cb670 msgr2=0x7ffb600cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 -- 172.21.15.153:0/1915480356 shutdown_connections 2023-11-28T13:00:43.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.822+0000 7ffb660f3700 1 -- 172.21.15.153:0/1915480356 wait complete. 2023-11-28T13:00:43.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.823+0000 7ffb660f3700 1 Processor -- start 2023-11-28T13:00:43.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.824+0000 7ffb660f3700 1 -- start start 2023-11-28T13:00:43.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.824+0000 7ffb660f3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb600ad0a0 0x7ffb601501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.824+0000 7ffb660f3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600b0670 0x7ffb60071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.824+0000 7ffb660f3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600cc730 0x7ffb60071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.824+0000 7ffb660f3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600b3880 con 0x7ffb600b0670 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.824+0000 7ffb660f3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600b1ba0 con 0x7ffb600ad0a0 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.824+0000 7ffb660f3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600b2940 con 0x7ffb600cc730 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600b0670 0x7ffb60071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600b0670 0x7ffb60071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39090/0 (socket says 172.21.15.153:39090) 2023-11-28T13:00:43.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 -- 172.21.15.153:0/3700025870 learned_addr learned my addr 172.21.15.153:0/3700025870 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:43.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5ffff700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600cc730 0x7ffb60071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5f7fe700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb600ad0a0 0x7ffb601501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 -- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600cc730 msgr2=0x7ffb60071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600cc730 0x7ffb60071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 -- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb600ad0a0 msgr2=0x7ffb601501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb600ad0a0 0x7ffb601501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 -- 172.21.15.153:0/3700025870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50009550 con 0x7ffb600b0670 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5f7fe700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb600ad0a0 0x7ffb601501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5ffff700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600cc730 0x7ffb60071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb5effd700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600b0670 0x7ffb60071030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffb50014040 tx=0x7ffb50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb57fff700 1 -- 172.21.15.153:0/3700025870 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500036a0 con 0x7ffb600b0670 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb57fff700 1 -- 172.21.15.153:0/3700025870 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5001e070 con 0x7ffb600b0670 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb600ce330 con 0x7ffb600b0670 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb57fff700 1 -- 172.21.15.153:0/3700025870 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50003a20 con 0x7ffb600b0670 2023-11-28T13:00:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.825+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb600b0f80 con 0x7ffb600b0670 2023-11-28T13:00:43.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.826+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb600ce610 con 0x7ffb600b0670 2023-11-28T13:00:43.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.827+0000 7ffb57fff700 1 -- 172.21.15.153:0/3700025870 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb5000f050 con 0x7ffb600b0670 2023-11-28T13:00:43.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.827+0000 7ffb57fff700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb400682f0 0x7ffb4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.827+0000 7ffb57fff700 1 -- 172.21.15.153:0/3700025870 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb50020070 con 0x7ffb600b0670 2023-11-28T13:00:43.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.827+0000 7ffb5f7fe700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb400682f0 0x7ffb4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.827+0000 7ffb5f7fe700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb400682f0 0x7ffb4006a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb48007a50 tx=0x7ffb48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:43.831+0000 7ffb57fff700 1 -- 172.21.15.153:0/3700025870 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb50053a30 con 0x7ffb600b0670 2023-11-28T13:00:44.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.015+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb600745c0 con 0x7ffb600b0670 2023-11-28T13:00:44.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.015+0000 7ffb57fff700 1 -- 172.21.15.153:0/3700025870 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb50057050 con 0x7ffb600b0670 2023-11-28T13:00:44.016 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:44.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb400682f0 msgr2=0x7ffb4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb400682f0 0x7ffb4006a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb48007a50 tx=0x7ffb48008040 comp rx=0 tx=0).stop 2023-11-28T13:00:44.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600b0670 msgr2=0x7ffb60071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600b0670 0x7ffb60071030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffb50014040 tx=0x7ffb50004750 comp rx=0 tx=0).stop 2023-11-28T13:00:44.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 shutdown_connections 2023-11-28T13:00:44.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb600cc730 0x7ffb60071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb600ad0a0 0x7ffb601501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb400682f0 0x7ffb4006a7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 --2- 172.21.15.153:0/3700025870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb600b0670 0x7ffb60071030 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 >> 172.21.15.153:0/3700025870 conn(0x7ffb600cb670 msgr2=0x7ffb6006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 shutdown_connections 2023-11-28T13:00:44.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.017+0000 7ffb660f3700 1 -- 172.21.15.153:0/3700025870 wait complete. 2023-11-28T13:00:44.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:44.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.153+0000 7efde1b86700 1 Processor -- start 2023-11-28T13:00:44.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.153+0000 7efde1b86700 1 -- start start 2023-11-28T13:00:44.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efde1b86700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efddc160740 0x7efddc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efde1b86700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc06b060 0x7efddc06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efde1b86700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc0cc030 con 0x7efddc160b10 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efde1b86700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0ca720 con 0x7efddc160740 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efde1b86700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0c9e10 con 0x7efddc06b060 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efddc160740 0x7efddc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efddc160740 0x7efddc0cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39096/0 (socket says 172.21.15.153:39096) 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 -- 172.21.15.153:0/4193674575 learned_addr learned my addr 172.21.15.153:0/4193674575 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddbfff700 1 --2- 172.21.15.153:0/4193674575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc06b060 0x7efddc06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 -- 172.21.15.153:0/4193674575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc06b060 msgr2=0x7efddc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 --2- 172.21.15.153:0/4193674575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc06b060 0x7efddc06b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 -- 172.21.15.153:0/4193674575 >> v1:172.21.15.177:6789/0 conn(0x7efddc160b10 legacy=0x7efddc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 -- 172.21.15.153:0/4193674575 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc06efc0 con 0x7efddc160740 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efddb7fe700 1 --2- 172.21.15.153:0/4193674575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160740 0x7efddc0cbaf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0098f0 tx=0x7efdcc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285612e1b48b865b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.154+0000 7efdda7fc700 1 -- 172.21.15.153:0/4193674575 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc011830 con 0x7efddc160740 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efdda7fc700 1 -- 172.21.15.153:0/4193674575 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc011990 con 0x7efddc160740 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efdda7fc700 1 -- 172.21.15.153:0/4193674575 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc011d20 con 0x7efddc160740 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 -- 172.21.15.153:0/4193674575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160740 msgr2=0x7efddc0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 --2- 172.21.15.153:0/4193674575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160740 0x7efddc0cbaf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0098f0 tx=0x7efdcc009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 -- 172.21.15.153:0/4193674575 shutdown_connections 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 --2- 172.21.15.153:0/4193674575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc06b060 0x7efddc06b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 --2- 172.21.15.153:0/4193674575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160740 0x7efddc0cbaf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 -- 172.21.15.153:0/4193674575 >> 172.21.15.153:0/4193674575 conn(0x7efddc0b12b0 msgr2=0x7efddc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 -- 172.21.15.153:0/4193674575 shutdown_connections 2023-11-28T13:00:44.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.155+0000 7efde1b86700 1 -- 172.21.15.153:0/4193674575 wait complete. 2023-11-28T13:00:44.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.156+0000 7efde1b86700 1 Processor -- start 2023-11-28T13:00:44.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.156+0000 7efde1b86700 1 -- start start 2023-11-28T13:00:44.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efde1b86700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efddc06b060 0x7efddc1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efde1b86700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc160740 0x7efddc0adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efde1b86700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160b10 0x7efddc157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efde1b86700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc158f90 con 0x7efddc160b10 2023-11-28T13:00:44.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efde1b86700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc159270 con 0x7efddc06b060 2023-11-28T13:00:44.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efde1b86700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc159550 con 0x7efddc160740 2023-11-28T13:00:44.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efddbfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160b10 0x7efddc157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efddbfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160b10 0x7efddc157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39104/0 (socket says 172.21.15.153:39104) 2023-11-28T13:00:44.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.157+0000 7efddbfff700 1 -- 172.21.15.153:0/814221563 learned_addr learned my addr 172.21.15.153:0/814221563 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:44.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddb7fe700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efddc06b060 0x7efddc1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddaffd700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc160740 0x7efddc0adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddbfff700 1 -- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc160740 msgr2=0x7efddc0adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddbfff700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc160740 0x7efddc0adad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddbfff700 1 -- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efddc06b060 msgr2=0x7efddc1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddbfff700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efddc06b060 0x7efddc1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddbfff700 1 -- 172.21.15.153:0/814221563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc009550 con 0x7efddc160b10 2023-11-28T13:00:44.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efddbfff700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160b10 0x7efddc157780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efdd400efc0 tx=0x7efdd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efdd3fff700 1 -- 172.21.15.153:0/814221563 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd400cd40 con 0x7efddc160b10 2023-11-28T13:00:44.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efdd3fff700 1 -- 172.21.15.153:0/814221563 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd400f040 con 0x7efddc160b10 2023-11-28T13:00:44.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc0ca3b0 con 0x7efddc160b10 2023-11-28T13:00:44.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efdd3fff700 1 -- 172.21.15.153:0/814221563 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd40146c0 con 0x7efddc160b10 2023-11-28T13:00:44.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.158+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc0ca0d0 con 0x7efddc160b10 2023-11-28T13:00:44.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.159+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc160f60 con 0x7efddc160b10 2023-11-28T13:00:44.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.159+0000 7efdd3fff700 1 -- 172.21.15.153:0/814221563 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efdd4014820 con 0x7efddc160b10 2023-11-28T13:00:44.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.160+0000 7efdd3fff700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efdbc0682f0 0x7efdbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.160+0000 7efddb7fe700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efdbc0682f0 0x7efdbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.160+0000 7efdd3fff700 1 -- 172.21.15.153:0/814221563 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efdd4086a00 con 0x7efddc160b10 2023-11-28T13:00:44.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.160+0000 7efddb7fe700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efdbc0682f0 0x7efdbc06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ce610 tx=0x7efdcc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.164+0000 7efdd3fff700 1 -- 172.21.15.153:0/814221563 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efdd4053260 con 0x7efddc160b10 2023-11-28T13:00:44.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.346+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efddc0c65b0 con 0x7efddc160b10 2023-11-28T13:00:44.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.347+0000 7efdd3fff700 1 -- 172.21.15.153:0/814221563 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7efdd4018020 con 0x7efddc160b10 2023-11-28T13:00:44.348 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:44.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efdbc0682f0 msgr2=0x7efdbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efdbc0682f0 0x7efdbc06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ce610 tx=0x7efdcc008030 comp rx=0 tx=0).stop 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160b10 msgr2=0x7efddc157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160b10 0x7efddc157780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efdd400efc0 tx=0x7efdd400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 shutdown_connections 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efdbc0682f0 0x7efdbc06a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efddc160b10 0x7efddc157780 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efddc06b060 0x7efddc1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 --2- 172.21.15.153:0/814221563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efddc160740 0x7efddc0adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.350 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 >> 172.21.15.153:0/814221563 conn(0x7efddc0b12b0 msgr2=0x7efddc15b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.351 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 shutdown_connections 2023-11-28T13:00:44.351 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.349+0000 7efde1b86700 1 -- 172.21.15.153:0/814221563 wait complete. 2023-11-28T13:00:44.362 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:44.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.481+0000 7f00799f4700 1 Processor -- start 2023-11-28T13:00:44.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.481+0000 7f00799f4700 1 -- start start 2023-11-28T13:00:44.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00799f4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f00740cd7c0 0x7f00740cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00799f4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740d1950 con 0x7f007415c9b0 2023-11-28T13:00:44.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00799f4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740ce140 con 0x7f0074161340 2023-11-28T13:00:44.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00799f4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00740b5800 con 0x7f00740cd7c0 2023-11-28T13:00:44.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f0072ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f007415c9b0 0x7f00740b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57272/0 (socket says 172.21.15.153:57272) 2023-11-28T13:00:44.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f0072ffd700 1 -- 172.21.15.153:0/1257542630 learned_addr learned my addr 172.21.15.153:0/1257542630 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00737fe700 1 --2- 172.21.15.153:0/1257542630 >> v2:172.21.15.177:3300/0 conn(0x7f00740cd7c0 0x7f00740cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f0071ffb700 1 -- 172.21.15.153:0/1257542630 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843894164 0 0) 0x7f00740d1950 con 0x7f007415c9b0 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f0071ffb700 1 -- 172.21.15.153:0/1257542630 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f005c0035b0 con 0x7f007415c9b0 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f0071ffb700 1 -- 172.21.15.153:0/1257542630 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500334304 0 0) 0x7f00740ce140 con 0x7f0074161340 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f0071ffb700 1 -- 172.21.15.153:0/1257542630 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00740d1950 con 0x7f0074161340 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00737fe700 1 -- 172.21.15.153:0/1257542630 >> v1:172.21.15.177:6789/0 conn(0x7f0074161340 legacy=0x7f00740d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00737fe700 1 -- 172.21.15.153:0/1257542630 >> v1:172.21.15.153:6789/0 conn(0x7f007415c9b0 legacy=0x7f00740b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.482+0000 7f00737fe700 1 -- 172.21.15.153:0/1257542630 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740d24f0 con 0x7f00740cd7c0 2023-11-28T13:00:44.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00737fe700 1 --2- 172.21.15.153:0/1257542630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cd7c0 0x7f00740cdc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f006c009d90 tx=0x7f006c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6c875780a3b442 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f0071ffb700 1 -- 172.21.15.153:0/1257542630 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00e070 con 0x7f00740cd7c0 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f0071ffb700 1 -- 172.21.15.153:0/1257542630 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c00bb20 con 0x7f00740cd7c0 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f0071ffb700 1 -- 172.21.15.153:0/1257542630 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c012430 con 0x7f00740cd7c0 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00799f4700 1 -- 172.21.15.153:0/1257542630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cd7c0 msgr2=0x7f00740cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00799f4700 1 --2- 172.21.15.153:0/1257542630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cd7c0 0x7f00740cdc00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f006c009d90 tx=0x7f006c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00799f4700 1 -- 172.21.15.153:0/1257542630 shutdown_connections 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00799f4700 1 --2- 172.21.15.153:0/1257542630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00740cd7c0 0x7f00740cdc00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00799f4700 1 -- 172.21.15.153:0/1257542630 >> 172.21.15.153:0/1257542630 conn(0x7f0074072600 msgr2=0x7f00740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00799f4700 1 -- 172.21.15.153:0/1257542630 shutdown_connections 2023-11-28T13:00:44.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.483+0000 7f00799f4700 1 -- 172.21.15.153:0/1257542630 wait complete. 2023-11-28T13:00:44.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.485+0000 7f00799f4700 1 Processor -- start 2023-11-28T13:00:44.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.485+0000 7f00799f4700 1 -- start start 2023-11-28T13:00:44.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.485+0000 7f00799f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740cd7c0 0x7f00741e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.485+0000 7f00799f4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007415c9b0 0x7f0074158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00799f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074161340 0x7f00741e8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00799f4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00741e77e0 con 0x7f0074161340 2023-11-28T13:00:44.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00799f4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074159bd0 con 0x7f007415c9b0 2023-11-28T13:00:44.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00799f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00741e64e0 con 0x7f00740cd7c0 2023-11-28T13:00:44.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074161340 0x7f00741e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074161340 0x7f00741e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39118/0 (socket says 172.21.15.153:39118) 2023-11-28T13:00:44.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 -- 172.21.15.153:0/627645633 learned_addr learned my addr 172.21.15.153:0/627645633 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00727fc700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007415c9b0 0x7f0074158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f0072ffd700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740cd7c0 0x7f00741e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 -- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740cd7c0 msgr2=0x7f00741e72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740cd7c0 0x7f00741e72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 -- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007415c9b0 msgr2=0x7f0074158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007415c9b0 0x7f0074158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 -- 172.21.15.153:0/627645633 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c009a70 con 0x7f0074161340 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00737fe700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074161340 0x7f00741e8320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00740675d0 tx=0x7f006c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f005bfff700 1 -- 172.21.15.153:0/627645633 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c012430 con 0x7f0074161340 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f005bfff700 1 -- 172.21.15.153:0/627645633 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c020070 con 0x7f0074161340 2023-11-28T13:00:44.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00740d1300 con 0x7f0074161340 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f005bfff700 1 -- 172.21.15.153:0/627645633 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c01ba50 con 0x7f0074161340 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.486+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00740b5520 con 0x7f0074161340 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.487+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00740d15e0 con 0x7f0074161340 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.488+0000 7f005bfff700 1 -- 172.21.15.153:0/627645633 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f006c01bbb0 con 0x7f0074161340 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.488+0000 7f005bfff700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00600682f0 0x7f006006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.488+0000 7f0072ffd700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00600682f0 0x7f006006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.488+0000 7f005bfff700 1 -- 172.21.15.153:0/627645633 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f006c022070 con 0x7f0074161340 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.488+0000 7f0072ffd700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00600682f0 0x7f006006a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0068007fd0 tx=0x7f0068007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.492+0000 7f005bfff700 1 -- 172.21.15.153:0/627645633 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f006c0549f0 con 0x7f0074161340 2023-11-28T13:00:44.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.678+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00741e68e0 con 0x7f0074161340 2023-11-28T13:00:44.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.680+0000 7f005bfff700 1 -- 172.21.15.153:0/627645633 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f006c058010 con 0x7f0074161340 2023-11-28T13:00:44.680 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:44.681 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00600682f0 msgr2=0x7f006006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00600682f0 0x7f006006a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0068007fd0 tx=0x7f0068007e00 comp rx=0 tx=0).stop 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074161340 msgr2=0x7f00741e8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074161340 0x7f00741e8320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00740675d0 tx=0x7f006c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 shutdown_connections 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00600682f0 0x7f006006a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074161340 0x7f00741e8320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00740cd7c0 0x7f00741e72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 --2- 172.21.15.153:0/627645633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f007415c9b0 0x7f0074158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 >> 172.21.15.153:0/627645633 conn(0x7f0074072600 msgr2=0x7f00740b7500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 shutdown_connections 2023-11-28T13:00:44.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.682+0000 7f00799f4700 1 -- 172.21.15.153:0/627645633 wait complete. 2023-11-28T13:00:44.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:44.695 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:44.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.818+0000 7fc501be1700 1 Processor -- start 2023-11-28T13:00:44.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.818+0000 7fc501be1700 1 -- start start 2023-11-28T13:00:44.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.818+0000 7fc501be1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc4fc1658c0 0x7fc4fc167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.818+0000 7fc501be1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc15d060 con 0x7fc4fc071040 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.818+0000 7fc501be1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc15d340 con 0x7fc4fc06dee0 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.818+0000 7fc501be1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc161550 con 0x7fc4fc1658c0 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fbfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc4fc1658c0 0x7fc4fc167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fb7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc4fc071040 0x7fc4fc071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57286/0 (socket says 172.21.15.153:57286) 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fb7fe700 1 -- 172.21.15.153:0/3728368223 learned_addr learned my addr 172.21.15.153:0/3728368223 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fbfff700 1 -- 172.21.15.153:0/3728368223 >> v1:172.21.15.177:6789/0 conn(0x7fc4fc06dee0 legacy=0x7fc4fc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fbfff700 1 -- 172.21.15.153:0/3728368223 >> v1:172.21.15.153:6789/0 conn(0x7fc4fc071040 legacy=0x7fc4fc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fbfff700 1 -- 172.21.15.153:0/3728368223 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc15d620 con 0x7fc4fc1658c0 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fa7fc700 1 -- 172.21.15.153:0/3728368223 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595905263 0 0) 0x7fc4fc15d060 con 0x7fc4fc071040 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fbfff700 1 --2- 172.21.15.153:0/3728368223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc167ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009d90 tx=0x7fc4f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e6d2481454fc72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fa7fc700 1 -- 172.21.15.153:0/3728368223 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f400e070 con 0x7fc4fc1658c0 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fa7fc700 1 -- 172.21.15.153:0/3728368223 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f400bbd0 con 0x7fc4fc1658c0 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.819+0000 7fc4fa7fc700 1 -- 172.21.15.153:0/3728368223 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4003a20 con 0x7fc4fc1658c0 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.820+0000 7fc501be1700 1 -- 172.21.15.153:0/3728368223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 msgr2=0x7fc4fc167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.820+0000 7fc501be1700 1 --2- 172.21.15.153:0/3728368223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc167ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009d90 tx=0x7fc4f400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.820+0000 7fc501be1700 1 -- 172.21.15.153:0/3728368223 shutdown_connections 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.820+0000 7fc501be1700 1 --2- 172.21.15.153:0/3728368223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc167ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.820+0000 7fc501be1700 1 -- 172.21.15.153:0/3728368223 >> 172.21.15.153:0/3728368223 conn(0x7fc4fc0d0000 msgr2=0x7fc4fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.820+0000 7fc501be1700 1 -- 172.21.15.153:0/3728368223 shutdown_connections 2023-11-28T13:00:44.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.820+0000 7fc501be1700 1 -- 172.21.15.153:0/3728368223 wait complete. 2023-11-28T13:00:44.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.821+0000 7fc501be1700 1 Processor -- start 2023-11-28T13:00:44.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.821+0000 7fc501be1700 1 -- start start 2023-11-28T13:00:44.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc501be1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc06dee0 0x7fc4fc0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc501be1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4fc071040 0x7fc4fc151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc501be1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc501be1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0b1290 con 0x7fc4fc1658c0 2023-11-28T13:00:44.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc501be1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0b5df0 con 0x7fc4fc06dee0 2023-11-28T13:00:44.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc501be1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0aeb00 con 0x7fc4fc071040 2023-11-28T13:00:44.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39132/0 (socket says 172.21.15.153:39132) 2023-11-28T13:00:44.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 -- 172.21.15.153:0/2340714117 learned_addr learned my addr 172.21.15.153:0/2340714117 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:44.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fb7fe700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc06dee0 0x7fc4fc0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 -- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4fc071040 msgr2=0x7fc4fc151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4faffd700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4fc071040 0x7fc4fc151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4fc071040 0x7fc4fc151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 -- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc06dee0 msgr2=0x7fc4fc0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc06dee0 0x7fc4fc0b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fbfff700 1 -- 172.21.15.153:0/2340714117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f4009a70 con 0x7fc4fc1658c0 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4fb7fe700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc06dee0 0x7fc4fc0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.822+0000 7fc4faffd700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4fc071040 0x7fc4fc151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.823+0000 7fc4fbfff700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc15ebb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4004000 tx=0x7fc4f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.823+0000 7fc4e3fff700 1 -- 172.21.15.153:0/2340714117 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4012630 con 0x7fc4fc1658c0 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.823+0000 7fc4e3fff700 1 -- 172.21.15.153:0/2340714117 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f40127b0 con 0x7fc4fc1658c0 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.823+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc15ccd0 con 0x7fc4fc1658c0 2023-11-28T13:00:44.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.823+0000 7fc4e3fff700 1 -- 172.21.15.153:0/2340714117 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4012b60 con 0x7fc4fc1658c0 2023-11-28T13:00:44.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.823+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc15d900 con 0x7fc4fc1658c0 2023-11-28T13:00:44.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.824+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc06a270 con 0x7fc4fc1658c0 2023-11-28T13:00:44.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.824+0000 7fc4e3fff700 1 -- 172.21.15.153:0/2340714117 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc4f4012cc0 con 0x7fc4fc1658c0 2023-11-28T13:00:44.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.824+0000 7fc4e3fff700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4e80682f0 0x7fc4e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.824+0000 7fc4e3fff700 1 -- 172.21.15.153:0/2340714117 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4f4088430 con 0x7fc4fc1658c0 2023-11-28T13:00:44.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.824+0000 7fc4fb7fe700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4e80682f0 0x7fc4e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.825+0000 7fc4fb7fe700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4e80682f0 0x7fc4e806a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0000f60 tx=0x7fc4f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:44.829+0000 7fc4e3fff700 1 -- 172.21.15.153:0/2340714117 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc4f4054c90 con 0x7fc4fc1658c0 2023-11-28T13:00:45.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.008+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4fc0720a0 con 0x7fc4fc1658c0 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.009+0000 7fc4e3fff700 1 -- 172.21.15.153:0/2340714117 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4f40582b0 con 0x7fc4fc1658c0 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:45.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4e80682f0 msgr2=0x7fc4e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4e80682f0 0x7fc4e806a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0000f60 tx=0x7fc4f0009040 comp rx=0 tx=0).stop 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 msgr2=0x7fc4fc15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc15ebb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4004000 tx=0x7fc4f4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 shutdown_connections 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc4e80682f0 0x7fc4e806a7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4fc1658c0 0x7fc4fc15ebb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4fc06dee0 0x7fc4fc0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 --2- 172.21.15.153:0/2340714117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4fc071040 0x7fc4fc151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.013 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 >> 172.21.15.153:0/2340714117 conn(0x7fc4fc0d0000 msgr2=0x7fc4fc152c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.013 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 shutdown_connections 2023-11-28T13:00:45.013 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.011+0000 7fc501be1700 1 -- 172.21.15.153:0/2340714117 wait complete. 2023-11-28T13:00:45.024 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:45.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.147+0000 7f5d1f84c700 1 Processor -- start 2023-11-28T13:00:45.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.147+0000 7f5d1f84c700 1 -- start start 2023-11-28T13:00:45.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1f84c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5d1806c990 0x7f5d1806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1f84c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d180ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1f84c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1806ae60 con 0x7f5d1806e9f0 2023-11-28T13:00:45.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1f84c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1806d350 con 0x7f5d1806c990 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1f84c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180cabc0 con 0x7f5d180ca240 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1cde7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5d1806c990 0x7f5d1806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1cde7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5d1806c990 0x7f5d1806ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39140/0 (socket says 172.21.15.153:39140) 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.148+0000 7f5d1cde7700 1 -- 172.21.15.153:0/2552720895 learned_addr learned my addr 172.21.15.153:0/2552720895 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1dde9700 1 --2- 172.21.15.153:0/2552720895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d180ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1cde7700 1 -- 172.21.15.153:0/2552720895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 msgr2=0x7f5d180ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1cde7700 1 --2- 172.21.15.153:0/2552720895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d180ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1cde7700 1 -- 172.21.15.153:0/2552720895 >> v1:172.21.15.177:6789/0 conn(0x7f5d1806e9f0 legacy=0x7f5d1806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1cde7700 1 -- 172.21.15.153:0/2552720895 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1806a270 con 0x7f5d1806c990 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1cde7700 1 --2- 172.21.15.153:0/2552720895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806c990 0x7f5d1806ce10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d140098f0 tx=0x7f5d14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbc37ee55089b23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d0ffff700 1 -- 172.21.15.153:0/2552720895 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14004230 con 0x7f5d1806c990 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d0ffff700 1 -- 172.21.15.153:0/2552720895 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14004390 con 0x7f5d1806c990 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d0ffff700 1 -- 172.21.15.153:0/2552720895 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d140036a0 con 0x7f5d1806c990 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1f84c700 1 -- 172.21.15.153:0/2552720895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806c990 msgr2=0x7f5d1806ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/2552720895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806c990 0x7f5d1806ce10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d140098f0 tx=0x7f5d14009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1f84c700 1 -- 172.21.15.153:0/2552720895 shutdown_connections 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.149+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/2552720895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d180ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.150+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/2552720895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806c990 0x7f5d1806ce10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.150+0000 7f5d1f84c700 1 -- 172.21.15.153:0/2552720895 >> 172.21.15.153:0/2552720895 conn(0x7f5d180b12b0 msgr2=0x7f5d180b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.150+0000 7f5d1f84c700 1 -- 172.21.15.153:0/2552720895 shutdown_connections 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.150+0000 7f5d1f84c700 1 -- 172.21.15.153:0/2552720895 wait complete. 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.151+0000 7f5d1f84c700 1 Processor -- start 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.151+0000 7f5d1f84c700 1 -- start start 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.151+0000 7f5d1f84c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d1806c990 0x7f5d180c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.151+0000 7f5d1f84c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806e9f0 0x7f5d180c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.151+0000 7f5d1f84c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d1814d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.151+0000 7f5d1f84c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180c8d50 con 0x7f5d1806e9f0 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1f84c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180c7b80 con 0x7f5d1806c990 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1f84c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181535f0 con 0x7f5d180ca240 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806e9f0 0x7f5d180c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806e9f0 0x7f5d180c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39144/0 (socket says 172.21.15.153:39144) 2023-11-28T13:00:45.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 -- 172.21.15.153:0/158167805 learned_addr learned my addr 172.21.15.153:0/158167805 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:45.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1d5e8700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d1806c990 0x7f5d180c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1dde9700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d1814d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 -- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 msgr2=0x7f5d1814d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d1814d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 -- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d1806c990 msgr2=0x7f5d180c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d1806c990 0x7f5d180c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 -- 172.21.15.153:0/158167805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d14009550 con 0x7f5d1806e9f0 2023-11-28T13:00:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1cde7700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806e9f0 0x7f5d180c7640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009130 tx=0x7f5d14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d0dffb700 1 -- 172.21.15.153:0/158167805 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14003de0 con 0x7f5d1806e9f0 2023-11-28T13:00:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d0dffb700 1 -- 172.21.15.153:0/158167805 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1400b660 con 0x7f5d1806e9f0 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1806a810 con 0x7f5d1806e9f0 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d0dffb700 1 -- 172.21.15.153:0/158167805 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1400ba10 con 0x7f5d1806e9f0 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.152+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1806a530 con 0x7f5d1806e9f0 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.153+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1806aaf0 con 0x7f5d1806e9f0 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.154+0000 7f5d0dffb700 1 -- 172.21.15.153:0/158167805 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d14017020 con 0x7f5d1806e9f0 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.154+0000 7f5d0dffb700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cfc0682f0 0x7f5cfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.154+0000 7f5d0dffb700 1 -- 172.21.15.153:0/158167805 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d14087300 con 0x7f5d1806e9f0 2023-11-28T13:00:45.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.154+0000 7f5d1d5e8700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cfc0682f0 0x7f5cfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.155+0000 7f5d1d5e8700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cfc0682f0 0x7f5cfc06a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d18151a90 tx=0x7f5d08007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.158+0000 7f5d0dffb700 1 -- 172.21.15.153:0/158167805 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d14053b00 con 0x7f5d1806e9f0 2023-11-28T13:00:45.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.344+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d180b4de0 con 0x7f5d1806e9f0 2023-11-28T13:00:45.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.344+0000 7f5d0dffb700 1 -- 172.21.15.153:0/158167805 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d14057120 con 0x7f5d1806e9f0 2023-11-28T13:00:45.346 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:45.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.346+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cfc0682f0 msgr2=0x7f5cfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.346+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cfc0682f0 0x7f5cfc06a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d18151a90 tx=0x7f5d08007ec0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.346+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806e9f0 msgr2=0x7f5d180c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.346+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806e9f0 0x7f5d180c7640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009130 tx=0x7f5d14004750 comp rx=0 tx=0).stop 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 shutdown_connections 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d180ca240 0x7f5d1814d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d1806c990 0x7f5d180c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cfc0682f0 0x7f5cfc06a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 --2- 172.21.15.153:0/158167805 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d1806e9f0 0x7f5d180c7640 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 >> 172.21.15.153:0/158167805 conn(0x7f5d180b12b0 msgr2=0x7f5d180b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 shutdown_connections 2023-11-28T13:00:45.349 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.347+0000 7f5d1f84c700 1 -- 172.21.15.153:0/158167805 wait complete. 2023-11-28T13:00:45.360 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:45.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.482+0000 7f3fab211700 1 Processor -- start 2023-11-28T13:00:45.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.482+0000 7f3fab211700 1 -- start start 2023-11-28T13:00:45.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.482+0000 7f3fab211700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 0x7f3fa4167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fab211700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa415d060 con 0x7f3fa4071040 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fab211700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa415d340 con 0x7f3fa406dee0 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fab211700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4161550 con 0x7f3fa41658c0 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa8fad700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3fa4071040 0x7f3fa4071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57314/0 (socket says 172.21.15.153:57314) 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa8fad700 1 -- 172.21.15.153:0/4281089764 learned_addr learned my addr 172.21.15.153:0/4281089764 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545690544 0 0) 0x7f3fa415d060 con 0x7f3fa4071040 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa97ae700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 0x7f3fa4167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f940035b0 con 0x7f3fa4071040 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3801953367 0 0) 0x7f3f940035b0 con 0x7f3fa4071040 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 msgr2=0x7f3fa4167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 0x7f3fa4167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 >> v1:172.21.15.177:6789/0 conn(0x7f3fa406dee0 legacy=0x7f3fa406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa415d620 con 0x7f3fa4071040 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f3f98001670 con 0x7f3fa4071040 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 >> v1:172.21.15.153:6789/0 conn(0x7f3fa4071040 legacy=0x7f3fa4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94003a30 0x7f3f94003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f94004850 0x7f3f94004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 0x7f3f940052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4161550 con 0x7f3f94004850 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9400bbd0 con 0x7f3f94003a30 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9400bd90 con 0x7f3fa41658c0 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa3fff700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f94004850 0x7f3f94004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa8fad700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94003a30 0x7f3f94003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.483+0000 7f3fa97ae700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 0x7f3f940052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa3fff700 1 -- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 msgr2=0x7f3f940052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa3fff700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa41658c0 0x7f3f940052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa3fff700 1 -- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94003a30 msgr2=0x7f3f94003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa3fff700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94003a30 0x7f3f94003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa3fff700 1 -- 172.21.15.153:0/4281089764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940035b0 con 0x7f3f94004850 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa3fff700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f94004850 0x7f3f94004cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007fd0 tx=0x7f3f94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facddab4d66faab8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f900012f0 con 0x7f3f94004850 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f900052f0 con 0x7f3f94004850 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa37fe700 1 -- 172.21.15.153:0/4281089764 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90006860 con 0x7f3f94004850 2023-11-28T13:00:45.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fab211700 1 -- 172.21.15.153:0/4281089764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f94004850 msgr2=0x7f3f94004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fab211700 1 --2- 172.21.15.153:0/4281089764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f94004850 0x7f3f94004cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007fd0 tx=0x7f3f94005880 comp rx=0 tx=0).stop 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.484+0000 7f3fa97ae700 1 -- 172.21.15.153:0/4281089764 reap_dead start 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.485+0000 7f3fab211700 1 -- 172.21.15.153:0/4281089764 shutdown_connections 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.485+0000 7f3fab211700 1 -- 172.21.15.153:0/4281089764 >> 172.21.15.153:0/4281089764 conn(0x7f3fa40d0000 msgr2=0x7f3fa40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.485+0000 7f3fab211700 1 -- 172.21.15.153:0/4281089764 shutdown_connections 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.485+0000 7f3fab211700 1 -- 172.21.15.153:0/4281089764 wait complete. 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.486+0000 7f3fab211700 1 Processor -- start 2023-11-28T13:00:45.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.486+0000 7f3fab211700 1 -- start start 2023-11-28T13:00:45.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94004850 0x7f3fa4155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa4151300 0x7f3fa41509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fa40c3a60 0x7f3fa4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa40aeab0 con 0x7f3fa40c3a60 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa40aefe0 con 0x7f3f94004850 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4157310 con 0x7f3fa4151300 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fa40c3a60 0x7f3fa4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fa40c3a60 0x7f3fa4157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39166/0 (socket says 172.21.15.153:39166) 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 -- 172.21.15.153:0/919630210 learned_addr learned my addr 172.21.15.153:0/919630210 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa3fff700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa4151300 0x7f3fa41509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa8fad700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94004850 0x7f3fa4155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 -- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa4151300 msgr2=0x7f3fa41509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa4151300 0x7f3fa41509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 -- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94004850 msgr2=0x7f3fa4155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94004850 0x7f3fa4155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 -- 172.21.15.153:0/919630210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa415ccd0 con 0x7f3fa40c3a60 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa97ae700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fa40c3a60 0x7f3fa4157af0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40c3a30 tx=0x7f3f9c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa17fa700 1 -- 172.21.15.153:0/919630210 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00e040 con 0x7f3fa40c3a60 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa17fa700 1 -- 172.21.15.153:0/919630210 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c00f040 con 0x7f3fa40c3a60 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa415d900 con 0x7f3fa40c3a60 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fa17fa700 1 -- 172.21.15.153:0/919630210 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00f950 con 0x7f3fa40c3a60 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.487+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa406a270 con 0x7f3fa40c3a60 2023-11-28T13:00:45.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.488+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa4158c90 con 0x7f3fa40c3a60 2023-11-28T13:00:45.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.489+0000 7f3fa17fa700 1 -- 172.21.15.153:0/919630210 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f9c00fab0 con 0x7f3fa40c3a60 2023-11-28T13:00:45.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.489+0000 7f3fa17fa700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f880682f0 0x7f3f8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.489+0000 7f3fa17fa700 1 -- 172.21.15.153:0/919630210 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f9c0862d0 con 0x7f3fa40c3a60 2023-11-28T13:00:45.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.489+0000 7f3fa8fad700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f880682f0 0x7f3f8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.490+0000 7f3fa8fad700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f880682f0 0x7f3f8806a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3fa415a3e0 tx=0x7f3f98000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.493+0000 7f3fa17fa700 1 -- 172.21.15.153:0/919630210 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3fa4158c90 con 0x7f3fa40c3a60 2023-11-28T13:00:45.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.675+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fa4157920 con 0x7f3fa40c3a60 2023-11-28T13:00:45.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.675+0000 7f3fa17fa700 1 -- 172.21.15.153:0/919630210 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fa4157920 con 0x7f3fa40c3a60 2023-11-28T13:00:45.676 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:45.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f880682f0 msgr2=0x7f3f8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f880682f0 0x7f3f8806a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3fa415a3e0 tx=0x7f3f98000f80 comp rx=0 tx=0).stop 2023-11-28T13:00:45.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fa40c3a60 msgr2=0x7f3fa4157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fa40c3a60 0x7f3fa4157af0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40c3a30 tx=0x7f3f9c011f60 comp rx=0 tx=0).stop 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 shutdown_connections 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f880682f0 0x7f3f8806a7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fa40c3a60 0x7f3fa4157af0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f94004850 0x7f3fa4155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 --2- 172.21.15.153:0/919630210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fa4151300 0x7f3fa41509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 >> 172.21.15.153:0/919630210 conn(0x7f3fa40d0000 msgr2=0x7f3fa406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 shutdown_connections 2023-11-28T13:00:45.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.677+0000 7f3fab211700 1 -- 172.21.15.153:0/919630210 wait complete. 2023-11-28T13:00:45.690 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:45.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.812+0000 7f424f5d7700 1 Processor -- start 2023-11-28T13:00:45.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.812+0000 7f424f5d7700 1 -- start start 2023-11-28T13:00:45.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424f5d7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f42480cd8c0 0x7f4248140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424f5d7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f424806f850 0x7f424806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424f5d7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424813ff00 con 0x7f42480cdc90 2023-11-28T13:00:45.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424f5d7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42481404c0 con 0x7f42480cd8c0 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424f5d7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424815aa80 con 0x7f424806f850 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424d373700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f42480cd8c0 0x7f4248140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424cb72700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f42480cdc90 0x7f42480b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57334/0 (socket says 172.21.15.153:57334) 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424cb72700 1 -- 172.21.15.153:0/2096870131 learned_addr learned my addr 172.21.15.153:0/2096870131 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424d373700 1 -- 172.21.15.153:0/2096870131 >> v2:172.21.15.177:3300/0 conn(0x7f424806f850 msgr2=0x7f424806fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424d373700 1 --2- 172.21.15.153:0/2096870131 >> v2:172.21.15.177:3300/0 conn(0x7f424806f850 0x7f424806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424d373700 1 -- 172.21.15.153:0/2096870131 >> v1:172.21.15.153:6789/0 conn(0x7f42480cdc90 legacy=0x7f42480b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f424d373700 1 -- 172.21.15.153:0/2096870131 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480b3040 con 0x7f42480cd8c0 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.813+0000 7f423ffff700 1 -- 172.21.15.153:0/2096870131 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959420550 0 0) 0x7f424813ff00 con 0x7f42480cdc90 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424d373700 1 --2- 172.21.15.153:0/2096870131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42480cd8c0 0x7f4248140950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f424806ea90 tx=0x7f4238009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b36cdff4957a55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f423ffff700 1 -- 172.21.15.153:0/2096870131 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238005e30 con 0x7f42480cd8c0 2023-11-28T13:00:45.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f423ffff700 1 -- 172.21.15.153:0/2096870131 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238006030 con 0x7f42480cd8c0 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f423ffff700 1 -- 172.21.15.153:0/2096870131 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380063c0 con 0x7f42480cd8c0 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 -- 172.21.15.153:0/2096870131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42480cd8c0 msgr2=0x7f4248140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 --2- 172.21.15.153:0/2096870131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42480cd8c0 0x7f4248140950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f424806ea90 tx=0x7f4238009b00 comp rx=0 tx=0).stop 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 -- 172.21.15.153:0/2096870131 shutdown_connections 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 --2- 172.21.15.153:0/2096870131 >> v2:172.21.15.177:3300/0 conn(0x7f424806f850 0x7f424806fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 --2- 172.21.15.153:0/2096870131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42480cd8c0 0x7f4248140950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 -- 172.21.15.153:0/2096870131 >> 172.21.15.153:0/2096870131 conn(0x7f424806aed0 msgr2=0x7f424806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 -- 172.21.15.153:0/2096870131 shutdown_connections 2023-11-28T13:00:45.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.814+0000 7f424f5d7700 1 -- 172.21.15.153:0/2096870131 wait complete. 2023-11-28T13:00:45.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 Processor -- start 2023-11-28T13:00:45.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 -- start start 2023-11-28T13:00:45.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f424806f850 0x7f42480b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42480cd8c0 0x7f4248153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42480cdc90 0x7f42481542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248154800 con 0x7f424806f850 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480c53f0 con 0x7f42480cdc90 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424f5d7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480c56d0 con 0x7f42480cd8c0 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424d373700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f424806f850 0x7f42480b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424d373700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f424806f850 0x7f42480b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39176/0 (socket says 172.21.15.153:39176) 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.816+0000 7f424d373700 1 -- 172.21.15.153:0/3010266553 learned_addr learned my addr 172.21.15.153:0/3010266553 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424db74700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42480cdc90 0x7f42481542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424cb72700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42480cd8c0 0x7f4248153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424d373700 1 -- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42480cd8c0 msgr2=0x7f4248153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424d373700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42480cd8c0 0x7f4248153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424d373700 1 -- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42480cdc90 msgr2=0x7f42481542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424d373700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42480cdc90 0x7f42481542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424d373700 1 -- 172.21.15.153:0/3010266553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4238009430 con 0x7f424806f850 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424cb72700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42480cd8c0 0x7f4248153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424db74700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42480cdc90 0x7f42481542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424d373700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f424806f850 0x7f42480b17d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c00 tx=0x7f4238004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f423dffb700 1 -- 172.21.15.153:0/3010266553 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238004240 con 0x7f424806f850 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f423dffb700 1 -- 172.21.15.153:0/3010266553 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380043c0 con 0x7f424806f850 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42480b35f0 con 0x7f424806f850 2023-11-28T13:00:45.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f423dffb700 1 -- 172.21.15.153:0/3010266553 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380036a0 con 0x7f424806f850 2023-11-28T13:00:45.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.817+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42480b3310 con 0x7f424806f850 2023-11-28T13:00:45.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.818+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4248140150 con 0x7f424806f850 2023-11-28T13:00:45.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.819+0000 7f423dffb700 1 -- 172.21.15.153:0/3010266553 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f423800f050 con 0x7f424806f850 2023-11-28T13:00:45.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.819+0000 7f423dffb700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f422c0682f0 0x7f422c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.819+0000 7f423dffb700 1 -- 172.21.15.153:0/3010266553 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f42380879c0 con 0x7f424806f850 2023-11-28T13:00:45.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.819+0000 7f424cb72700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f422c0682f0 0x7f422c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.820+0000 7f424cb72700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f422c0682f0 0x7f422c06a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4244007fd0 tx=0x7f4244009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:45.823+0000 7f423dffb700 1 -- 172.21.15.153:0/3010266553 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4238054220 con 0x7f424806f850 2023-11-28T13:00:46.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.004+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4248068690 con 0x7f424806f850 2023-11-28T13:00:46.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.005+0000 7f423dffb700 1 -- 172.21.15.153:0/3010266553 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4238057840 con 0x7f424806f850 2023-11-28T13:00:46.006 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f422c0682f0 msgr2=0x7f422c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f422c0682f0 0x7f422c06a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4244007fd0 tx=0x7f4244009040 comp rx=0 tx=0).stop 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f424806f850 msgr2=0x7f42480b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f424806f850 0x7f42480b17d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c00 tx=0x7f4238004750 comp rx=0 tx=0).stop 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 shutdown_connections 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f422c0682f0 0x7f422c06a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f424806f850 0x7f42480b17d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42480cdc90 0x7f42481542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 --2- 172.21.15.153:0/3010266553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42480cd8c0 0x7f4248153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 >> 172.21.15.153:0/3010266553 conn(0x7f424806aed0 msgr2=0x7f42480b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 shutdown_connections 2023-11-28T13:00:46.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.007+0000 7f424f5d7700 1 -- 172.21.15.153:0/3010266553 wait complete. 2023-11-28T13:00:46.020 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:46.036 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:46.066 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:46.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.143+0000 7eff443bb700 1 Processor -- start 2023-11-28T13:00:46.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.143+0000 7eff443bb700 1 -- start start 2023-11-28T13:00:46.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.144+0000 7eff443bb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7eff3c0d01b0 0x7eff3c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.144+0000 7eff443bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0b20b0 0x7eff3c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.144+0000 7eff443bb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c0b0230 con 0x7eff3c0b29b0 2023-11-28T13:00:46.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff443bb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c15cec0 con 0x7eff3c0d01b0 2023-11-28T13:00:46.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff443bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0b4570 con 0x7eff3c0b20b0 2023-11-28T13:00:46.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42157700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7eff3c0d01b0 0x7eff3c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42958700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7eff3c0b29b0 0x7eff3c0b2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57348/0 (socket says 172.21.15.153:57348) 2023-11-28T13:00:46.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42958700 1 -- 172.21.15.153:0/1912937703 learned_addr learned my addr 172.21.15.153:0/1912937703 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:46.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff41956700 1 --2- 172.21.15.153:0/1912937703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0b20b0 0x7eff3c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42157700 1 -- 172.21.15.153:0/1912937703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0b20b0 msgr2=0x7eff3c0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:46.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42157700 1 --2- 172.21.15.153:0/1912937703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0b20b0 0x7eff3c0d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42157700 1 -- 172.21.15.153:0/1912937703 >> v1:172.21.15.153:6789/0 conn(0x7eff3c0b29b0 legacy=0x7eff3c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42157700 1 -- 172.21.15.153:0/1912937703 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c0b4850 con 0x7eff3c0d01b0 2023-11-28T13:00:46.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff41155700 1 -- 172.21.15.153:0/1912937703 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117662205 0 0) 0x7eff3c0b0230 con 0x7eff3c0b29b0 2023-11-28T13:00:46.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff42157700 1 --2- 172.21.15.153:0/1912937703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0d01b0 0x7eff3c0d0c20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009840 tx=0x7eff2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3642703d2ea4a53c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff41155700 1 -- 172.21.15.153:0/1912937703 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c011830 con 0x7eff3c0d01b0 2023-11-28T13:00:46.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff41155700 1 -- 172.21.15.153:0/1912937703 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c011990 con 0x7eff3c0d01b0 2023-11-28T13:00:46.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.145+0000 7eff41155700 1 -- 172.21.15.153:0/1912937703 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c011d20 con 0x7eff3c0d01b0 2023-11-28T13:00:46.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 -- 172.21.15.153:0/1912937703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0d01b0 msgr2=0x7eff3c0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 --2- 172.21.15.153:0/1912937703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0d01b0 0x7eff3c0d0c20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009840 tx=0x7eff2c009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:46.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 -- 172.21.15.153:0/1912937703 shutdown_connections 2023-11-28T13:00:46.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 --2- 172.21.15.153:0/1912937703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0d01b0 0x7eff3c0d0c20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 --2- 172.21.15.153:0/1912937703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0b20b0 0x7eff3c0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 -- 172.21.15.153:0/1912937703 >> 172.21.15.153:0/1912937703 conn(0x7eff3c09c3f0 msgr2=0x7eff3c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 -- 172.21.15.153:0/1912937703 shutdown_connections 2023-11-28T13:00:46.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.146+0000 7eff443bb700 1 -- 172.21.15.153:0/1912937703 wait complete. 2023-11-28T13:00:46.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.147+0000 7eff443bb700 1 Processor -- start 2023-11-28T13:00:46.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.147+0000 7eff443bb700 1 -- start start 2023-11-28T13:00:46.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff443bb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff3c0b20b0 0x7eff3c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff443bb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0b29b0 0x7eff3c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff443bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0d01b0 0x7eff3c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff443bb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0c6de0 con 0x7eff3c0b29b0 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff443bb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0cc630 con 0x7eff3c0b20b0 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff443bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0cc910 con 0x7eff3c0d01b0 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0b29b0 0x7eff3c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0b29b0 0x7eff3c15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39192/0 (socket says 172.21.15.153:39192) 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 -- 172.21.15.153:0/4086942994 learned_addr learned my addr 172.21.15.153:0/4086942994 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff42958700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0d01b0 0x7eff3c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 -- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0d01b0 msgr2=0x7eff3c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff42157700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff3c0b20b0 0x7eff3c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0d01b0 0x7eff3c15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 -- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff3c0b20b0 msgr2=0x7eff3c15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff3c0b20b0 0x7eff3c15a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 -- 172.21.15.153:0/4086942994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0094a0 con 0x7eff3c0b29b0 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff42958700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0d01b0 0x7eff3c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.148+0000 7eff41956700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0b29b0 0x7eff3c15ae80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff3800e810 tx=0x7eff3800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.149+0000 7eff32ffd700 1 -- 172.21.15.153:0/4086942994 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38013610 con 0x7eff3c0b29b0 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.149+0000 7eff32ffd700 1 -- 172.21.15.153:0/4086942994 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff38013770 con 0x7eff3c0b29b0 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.149+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c0afec0 con 0x7eff3c0b29b0 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.149+0000 7eff32ffd700 1 -- 172.21.15.153:0/4086942994 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38013b20 con 0x7eff3c0b29b0 2023-11-28T13:00:46.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.149+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c0afbe0 con 0x7eff3c0b29b0 2023-11-28T13:00:46.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.150+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c0b2600 con 0x7eff3c0b29b0 2023-11-28T13:00:46.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.150+0000 7eff32ffd700 1 -- 172.21.15.153:0/4086942994 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff38023070 con 0x7eff3c0b29b0 2023-11-28T13:00:46.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.150+0000 7eff32ffd700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff200682f0 0x7eff2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.150+0000 7eff32ffd700 1 -- 172.21.15.153:0/4086942994 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff380859b0 con 0x7eff3c0b29b0 2023-11-28T13:00:46.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.150+0000 7eff42157700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff200682f0 0x7eff2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.151+0000 7eff42157700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff200682f0 0x7eff2006a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7eff3c1f5780 tx=0x7eff2c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.155+0000 7eff32ffd700 1 -- 172.21.15.153:0/4086942994 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff38052190 con 0x7eff3c0b29b0 2023-11-28T13:00:46.339 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.338+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff3c068690 con 0x7eff3c0b29b0 2023-11-28T13:00:46.340 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.339+0000 7eff32ffd700 1 -- 172.21.15.153:0/4086942994 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7eff380557b0 con 0x7eff3c0b29b0 2023-11-28T13:00:46.340 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:46.341 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff200682f0 msgr2=0x7eff2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff200682f0 0x7eff2006a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7eff3c1f5780 tx=0x7eff2c008030 comp rx=0 tx=0).stop 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0b29b0 msgr2=0x7eff3c15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0b29b0 0x7eff3c15ae80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff3800e810 tx=0x7eff3800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 shutdown_connections 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff3c0d01b0 0x7eff3c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff3c0b20b0 0x7eff3c15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff200682f0 0x7eff2006a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 --2- 172.21.15.153:0/4086942994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff3c0b29b0 0x7eff3c15ae80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 >> 172.21.15.153:0/4086942994 conn(0x7eff3c09c3f0 msgr2=0x7eff3c0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 shutdown_connections 2023-11-28T13:00:46.344 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.342+0000 7eff443bb700 1 -- 172.21.15.153:0/4086942994 wait complete. 2023-11-28T13:00:46.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:46.355 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:46.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.519+0000 7f1786779700 1 Processor -- start 2023-11-28T13:00:46.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.519+0000 7f1786779700 1 -- start start 2023-11-28T13:00:46.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f1786779700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f178015cef0 0x7f17800af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f1786779700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f178006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f1786779700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17800cc5f0 con 0x7f178006b8c0 2023-11-28T13:00:46.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f1786779700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178006bd40 con 0x7f178015cef0 2023-11-28T13:00:46.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f1786779700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178015d470 con 0x7f17800cbdf0 2023-11-28T13:00:46.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177ffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f178015cef0 0x7f17800af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f1784d16700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f178006b8c0 0x7f1780071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57366/0 (socket says 172.21.15.153:57366) 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f1784d16700 1 -- 172.21.15.153:0/760745754 learned_addr learned my addr 172.21.15.153:0/760745754 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177f7fe700 1 --2- 172.21.15.153:0/760745754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f178006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177effd700 1 -- 172.21.15.153:0/760745754 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40648348 0 0) 0x7f17800cc5f0 con 0x7f178006b8c0 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177effd700 1 -- 172.21.15.153:0/760745754 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17680035b0 con 0x7f178006b8c0 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177ffff700 1 -- 172.21.15.153:0/760745754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 msgr2=0x7f178006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177ffff700 1 --2- 172.21.15.153:0/760745754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f178006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177ffff700 1 -- 172.21.15.153:0/760745754 >> v1:172.21.15.153:6789/0 conn(0x7f178006b8c0 legacy=0x7f1780071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177ffff700 1 -- 172.21.15.153:0/760745754 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178015d750 con 0x7f178015cef0 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177ffff700 1 --2- 172.21.15.153:0/760745754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178015cef0 0x7f17800af400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1774009840 tx=0x7f1774009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1f831661342c58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177effd700 1 -- 172.21.15.153:0/760745754 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774011830 con 0x7f178015cef0 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177effd700 1 -- 172.21.15.153:0/760745754 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1774011990 con 0x7f178015cef0 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.520+0000 7f177effd700 1 -- 172.21.15.153:0/760745754 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774011d20 con 0x7f178015cef0 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 -- 172.21.15.153:0/760745754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178015cef0 msgr2=0x7f17800af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 --2- 172.21.15.153:0/760745754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178015cef0 0x7f17800af400 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1774009840 tx=0x7f1774009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 -- 172.21.15.153:0/760745754 shutdown_connections 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 --2- 172.21.15.153:0/760745754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178015cef0 0x7f17800af400 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 --2- 172.21.15.153:0/760745754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f178006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 -- 172.21.15.153:0/760745754 >> 172.21.15.153:0/760745754 conn(0x7f17800d0000 msgr2=0x7f17800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 -- 172.21.15.153:0/760745754 shutdown_connections 2023-11-28T13:00:46.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.521+0000 7f1786779700 1 -- 172.21.15.153:0/760745754 wait complete. 2023-11-28T13:00:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.522+0000 7f1786779700 1 Processor -- start 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.522+0000 7f1786779700 1 -- start start 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f1786779700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178006b8c0 0x7f178009d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f1786779700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f1780154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f1786779700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f178015cef0 0x7f1780155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f1786779700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1780155aa0 con 0x7f178006b8c0 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f1786779700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17800b52f0 con 0x7f178015cef0 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f1786779700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178009d890 con 0x7f17800cbdf0 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f177ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178006b8c0 0x7f178009d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f177ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178006b8c0 0x7f178009d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39204/0 (socket says 172.21.15.153:39204) 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f177ffff700 1 -- 172.21.15.153:0/2107108759 learned_addr learned my addr 172.21.15.153:0/2107108759 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.523+0000 7f177f7fe700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f1780154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177ffff700 1 -- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 msgr2=0x7f1780154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177ffff700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f1780154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177ffff700 1 -- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f178015cef0 msgr2=0x7f1780155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f1784d16700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f178015cef0 0x7f1780155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177ffff700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f178015cef0 0x7f1780155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177ffff700 1 -- 172.21.15.153:0/2107108759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740094a0 con 0x7f178006b8c0 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177f7fe700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f1780154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f1784d16700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f178015cef0 0x7f1780155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177ffff700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178006b8c0 0x7f178009d350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17740090b0 tx=0x7f1774004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177cff9700 1 -- 172.21.15.153:0/2107108759 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740036a0 con 0x7f178006b8c0 2023-11-28T13:00:46.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177cff9700 1 -- 172.21.15.153:0/2107108759 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177401e070 con 0x7f178006b8c0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178015dd10 con 0x7f178006b8c0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f177cff9700 1 -- 172.21.15.153:0/2107108759 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774003a20 con 0x7f178006b8c0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.524+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178015da30 con 0x7f178006b8c0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.525+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178015dff0 con 0x7f178006b8c0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.525+0000 7f177cff9700 1 -- 172.21.15.153:0/2107108759 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f177400f050 con 0x7f178006b8c0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.526+0000 7f177cff9700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f176c0682f0 0x7f176c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.526+0000 7f177cff9700 1 -- 172.21.15.153:0/2107108759 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1774086c10 con 0x7f178006b8c0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.526+0000 7f177f7fe700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f176c0682f0 0x7f176c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.526+0000 7f177f7fe700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f176c0682f0 0x7f176c06a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f17800cb660 tx=0x7f1770008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.530+0000 7f177cff9700 1 -- 172.21.15.153:0/2107108759 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1774053470 con 0x7f178006b8c0 2023-11-28T13:00:46.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.712+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17800cf7f0 con 0x7f178006b8c0 2023-11-28T13:00:46.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.713+0000 7f177cff9700 1 -- 172.21.15.153:0/2107108759 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1774020020 con 0x7f178006b8c0 2023-11-28T13:00:46.714 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f176c0682f0 msgr2=0x7f176c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f176c0682f0 0x7f176c06a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f17800cb660 tx=0x7f1770008040 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178006b8c0 msgr2=0x7f178009d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178006b8c0 0x7f178009d350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17740090b0 tx=0x7f1774004750 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 shutdown_connections 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f176c0682f0 0x7f176c06a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f178006b8c0 0x7f178009d350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f178015cef0 0x7f1780155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 --2- 172.21.15.153:0/2107108759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17800cbdf0 0x7f1780154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 >> 172.21.15.153:0/2107108759 conn(0x7f17800d0000 msgr2=0x7f17800afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 shutdown_connections 2023-11-28T13:00:46.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.715+0000 7f1786779700 1 -- 172.21.15.153:0/2107108759 wait complete. 2023-11-28T13:00:46.728 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:46.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.849+0000 7f09782e7700 1 Processor -- start 2023-11-28T13:00:46.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.850+0000 7f09782e7700 1 -- start start 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.850+0000 7f09782e7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f097006c990 0x7f097006ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.850+0000 7f09782e7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f09700ca240 0x7f09700ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.850+0000 7f09782e7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097006ae60 con 0x7f097006e9f0 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f09782e7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097006d350 con 0x7f097006c990 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f09782e7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09700cab70 con 0x7f09700ca240 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975882700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f097006c990 0x7f097006ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0976083700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f097006e9f0 0x7f097006b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57376/0 (socket says 172.21.15.153:57376) 2023-11-28T13:00:46.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0976083700 1 -- 172.21.15.153:0/92767540 learned_addr learned my addr 172.21.15.153:0/92767540 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0976884700 1 --2- 172.21.15.153:0/92767540 >> v2:172.21.15.177:3300/0 conn(0x7f09700ca240 0x7f09700ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975882700 1 -- 172.21.15.153:0/92767540 >> v2:172.21.15.177:3300/0 conn(0x7f09700ca240 msgr2=0x7f09700ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975882700 1 --2- 172.21.15.153:0/92767540 >> v2:172.21.15.177:3300/0 conn(0x7f09700ca240 0x7f09700ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975882700 1 -- 172.21.15.153:0/92767540 >> v1:172.21.15.153:6789/0 conn(0x7f097006e9f0 legacy=0x7f097006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975882700 1 -- 172.21.15.153:0/92767540 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097006a270 con 0x7f097006c990 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975081700 1 -- 172.21.15.153:0/92767540 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138027518 0 0) 0x7f097006ae60 con 0x7f097006e9f0 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975882700 1 --2- 172.21.15.153:0/92767540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097006c990 0x7f097006ce10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0970070490 tx=0x7f096c00acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc448442bbb75ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975081700 1 -- 172.21.15.153:0/92767540 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c00cb70 con 0x7f097006c990 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975081700 1 -- 172.21.15.153:0/92767540 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c00ccd0 con 0x7f097006c990 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.851+0000 7f0975081700 1 -- 172.21.15.153:0/92767540 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c018620 con 0x7f097006c990 2023-11-28T13:00:46.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 -- 172.21.15.153:0/92767540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097006c990 msgr2=0x7f097006ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 --2- 172.21.15.153:0/92767540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097006c990 0x7f097006ce10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0970070490 tx=0x7f096c00acc0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 -- 172.21.15.153:0/92767540 shutdown_connections 2023-11-28T13:00:46.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 --2- 172.21.15.153:0/92767540 >> v2:172.21.15.177:3300/0 conn(0x7f09700ca240 0x7f09700ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 --2- 172.21.15.153:0/92767540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f097006c990 0x7f097006ce10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 -- 172.21.15.153:0/92767540 >> 172.21.15.153:0/92767540 conn(0x7f09700b12b0 msgr2=0x7f09700b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 -- 172.21.15.153:0/92767540 shutdown_connections 2023-11-28T13:00:46.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.852+0000 7f09782e7700 1 -- 172.21.15.153:0/92767540 wait complete. 2023-11-28T13:00:46.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.853+0000 7f09782e7700 1 Processor -- start 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.853+0000 7f09782e7700 1 -- start start 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f09782e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f097006c990 0x7f097015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f09782e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f097006e9f0 0x7f09700c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f09782e7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09700ca240 0x7f097014fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f09782e7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700ad0b0 con 0x7f09700ca240 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f09782e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700c8e00 con 0x7f097006c990 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f09782e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700c8b20 con 0x7f097006e9f0 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09700ca240 0x7f097014fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09700ca240 0x7f097014fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39220/0 (socket says 172.21.15.153:39220) 2023-11-28T13:00:46.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 -- 172.21.15.153:0/2470664344 learned_addr learned my addr 172.21.15.153:0/2470664344 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976083700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f097006c990 0x7f097015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0975882700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f097006e9f0 0x7f09700c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 -- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f097006e9f0 msgr2=0x7f09700c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f097006e9f0 0x7f09700c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 -- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f097006c990 msgr2=0x7f097015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f097006c990 0x7f097015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 -- 172.21.15.153:0/2470664344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c00a640 con 0x7f09700ca240 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976083700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f097006c990 0x7f097015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0975882700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f097006e9f0 0x7f09700c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.854+0000 7f0976884700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09700ca240 0x7f097014fa80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f096800cd70 tx=0x7f096800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.855+0000 7f0966ffd700 1 -- 172.21.15.153:0/2470664344 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968003a20 con 0x7f09700ca240 2023-11-28T13:00:46.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.855+0000 7f0966ffd700 1 -- 172.21.15.153:0/2470664344 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968003b80 con 0x7f09700ca240 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.855+0000 7f0966ffd700 1 -- 172.21.15.153:0/2470664344 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968004020 con 0x7f09700ca240 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.855+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097006a810 con 0x7f09700ca240 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.855+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097006a530 con 0x7f09700ca240 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.855+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097006aaf0 con 0x7f09700ca240 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.856+0000 7f0966ffd700 1 -- 172.21.15.153:0/2470664344 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f096801c020 con 0x7f09700ca240 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.856+0000 7f0966ffd700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f09540682f0 0x7f095406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.856+0000 7f0966ffd700 1 -- 172.21.15.153:0/2470664344 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0968012070 con 0x7f09700ca240 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.856+0000 7f0976083700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f09540682f0 0x7f095406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.857+0000 7f0976083700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f09540682f0 0x7f095406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0960000f60 tx=0x7f0960009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:46.861+0000 7f0966ffd700 1 -- 172.21.15.153:0/2470664344 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0968052cd0 con 0x7f09700ca240 2023-11-28T13:00:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.042+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09700ad1f0 con 0x7f09700ca240 2023-11-28T13:00:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.043+0000 7f0966ffd700 1 -- 172.21.15.153:0/2470664344 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f09700ad1f0 con 0x7f09700ca240 2023-11-28T13:00:47.044 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f09540682f0 msgr2=0x7f095406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f09540682f0 0x7f095406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0960000f60 tx=0x7f0960009040 comp rx=0 tx=0).stop 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09700ca240 msgr2=0x7f097014fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09700ca240 0x7f097014fa80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f096800cd70 tx=0x7f096800a570 comp rx=0 tx=0).stop 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 shutdown_connections 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f09540682f0 0x7f095406a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09700ca240 0x7f097014fa80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f097006c990 0x7f097015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 --2- 172.21.15.153:0/2470664344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f097006e9f0 0x7f09700c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 >> 172.21.15.153:0/2470664344 conn(0x7f09700b12b0 msgr2=0x7f09700b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 shutdown_connections 2023-11-28T13:00:47.047 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.045+0000 7f09782e7700 1 -- 172.21.15.153:0/2470664344 wait complete. 2023-11-28T13:00:47.058 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:47.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.180+0000 7efec9c21700 1 Processor -- start 2023-11-28T13:00:47.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.180+0000 7efec9c21700 1 -- start start 2023-11-28T13:00:47.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec9c21700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efec406e9f0 0x7efec406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec9c21700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec406ae60 con 0x7efec406d0a0 2023-11-28T13:00:47.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec9c21700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec406a270 con 0x7efec406b830 2023-11-28T13:00:47.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec9c21700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec406a530 con 0x7efec406e9f0 2023-11-28T13:00:47.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec3fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7efec406d0a0 0x7efec40ca240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57390/0 (socket says 172.21.15.153:57390) 2023-11-28T13:00:47.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec3fff700 1 -- 172.21.15.153:0/3891065003 learned_addr learned my addr 172.21.15.153:0/3891065003 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:47.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec37fe700 1 --2- 172.21.15.153:0/3891065003 >> v2:172.21.15.177:3300/0 conn(0x7efec406e9f0 0x7efec406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715711131 0 0) 0x7efec406ae60 con 0x7efec406d0a0 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.181+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeac0035b0 con 0x7efec406d0a0 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657893669 0 0) 0x7efec406a270 con 0x7efec406b830 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec406ae60 con 0x7efec406b830 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 360833336 0 0) 0x7efeac0035b0 con 0x7efec406d0a0 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 >> v2:172.21.15.177:3300/0 conn(0x7efec406e9f0 msgr2=0x7efec406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 --2- 172.21.15.153:0/3891065003 >> v2:172.21.15.177:3300/0 conn(0x7efec406e9f0 0x7efec406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 >> v1:172.21.15.177:6789/0 conn(0x7efec406b830 legacy=0x7efec406c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec406a810 con 0x7efec406d0a0 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7efebc0019f0 con 0x7efec406d0a0 2023-11-28T13:00:47.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 >> v1:172.21.15.153:6789/0 conn(0x7efec406d0a0 legacy=0x7efec40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac003be0 0x7efeac003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efeac004850 0x7efeac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efeac005200 0x7efeac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec406a530 con 0x7efeac005200 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac00c0e0 con 0x7efeac003be0 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac00c270 con 0x7efeac004850 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efeac005200 0x7efeac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec37fe700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac003be0 0x7efeac003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec2ffd700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efeac004850 0x7efeac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 -- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efeac004850 msgr2=0x7efeac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efeac004850 0x7efeac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 -- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac003be0 msgr2=0x7efeac003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:47.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac003be0 0x7efeac003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 -- 172.21.15.153:0/3891065003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0035b0 con 0x7efeac005200 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 -- 172.21.15.153:0/3891065003 reap_dead start 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec3fff700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efeac005200 0x7efeac00ba10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efebc00a6e0 tx=0x7efeac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7875bf15146c48 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.182+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0072d0 con 0x7efeac005200 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc007430 con 0x7efeac005200 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec27fc700 1 -- 172.21.15.153:0/3891065003 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0077e0 con 0x7efeac005200 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec9c21700 1 -- 172.21.15.153:0/3891065003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efeac005200 msgr2=0x7efeac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec9c21700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efeac005200 0x7efeac00ba10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efebc00a6e0 tx=0x7efeac005880 comp rx=0 tx=0).stop 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec9c21700 1 -- 172.21.15.153:0/3891065003 shutdown_connections 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec9c21700 1 --2- 172.21.15.153:0/3891065003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efeac005200 0x7efeac00ba10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec9c21700 1 -- 172.21.15.153:0/3891065003 >> 172.21.15.153:0/3891065003 conn(0x7efec40b12b0 msgr2=0x7efec40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:47.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec9c21700 1 -- 172.21.15.153:0/3891065003 shutdown_connections 2023-11-28T13:00:47.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.183+0000 7efec9c21700 1 -- 172.21.15.153:0/3891065003 wait complete. 2023-11-28T13:00:47.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.184+0000 7efec9c21700 1 Processor -- start 2023-11-28T13:00:47.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec9c21700 1 -- start start 2023-11-28T13:00:47.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec9c21700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac005200 0x7efec415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec9c21700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec40c7d70 0x7efec40ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec9c21700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec40c81c0 0x7efec41f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec9c21700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41493d0 con 0x7efec40c81c0 2023-11-28T13:00:47.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec9c21700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4149690 con 0x7efeac005200 2023-11-28T13:00:47.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec9c21700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec4149970 con 0x7efec40c7d70 2023-11-28T13:00:47.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec40c81c0 0x7efec41f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec40c81c0 0x7efec41f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39240/0 (socket says 172.21.15.153:39240) 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.185+0000 7efec3fff700 1 -- 172.21.15.153:0/2590657848 learned_addr learned my addr 172.21.15.153:0/2590657848 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec3fff700 1 -- 172.21.15.153:0/2590657848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec40c7d70 msgr2=0x7efec40ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec3fff700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec40c7d70 0x7efec40ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec3fff700 1 -- 172.21.15.153:0/2590657848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac005200 msgr2=0x7efec415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec3fff700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac005200 0x7efec415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec3fff700 1 -- 172.21.15.153:0/2590657848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec406edc0 con 0x7efec40c81c0 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec3fff700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec40c81c0 0x7efec41f4120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efec4159580 tx=0x7efec406add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efeabfff700 1 -- 172.21.15.153:0/2590657848 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0079b0 con 0x7efec40c81c0 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efeabfff700 1 -- 172.21.15.153:0/2590657848 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc007b10 con 0x7efec40c81c0 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec406aaf0 con 0x7efec40c81c0 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efeabfff700 1 -- 172.21.15.153:0/2590657848 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc01f630 con 0x7efec40c81c0 2023-11-28T13:00:47.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.186+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec406ef50 con 0x7efec40c81c0 2023-11-28T13:00:47.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.187+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec40c9c60 con 0x7efec40c81c0 2023-11-28T13:00:47.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.187+0000 7efeabfff700 1 -- 172.21.15.153:0/2590657848 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efebc010070 con 0x7efec40c81c0 2023-11-28T13:00:47.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.188+0000 7efeabfff700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efeb00682f0 0x7efeb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.188+0000 7efeabfff700 1 -- 172.21.15.153:0/2590657848 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efebc0870e0 con 0x7efec40c81c0 2023-11-28T13:00:47.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.188+0000 7efec37fe700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efeb00682f0 0x7efeb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.188+0000 7efec37fe700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efeb00682f0 0x7efeb006a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009ae0 tx=0x7efeb80093c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.192+0000 7efeabfff700 1 -- 172.21.15.153:0/2590657848 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efebc053860 con 0x7efec40c81c0 2023-11-28T13:00:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.373+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efec40cd5b0 con 0x7efec40c81c0 2023-11-28T13:00:47.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.373+0000 7efeabfff700 1 -- 172.21.15.153:0/2590657848 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7efebc056e80 con 0x7efec40c81c0 2023-11-28T13:00:47.374 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efeb00682f0 msgr2=0x7efeb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efeb00682f0 0x7efeb006a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009ae0 tx=0x7efeb80093c0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec40c81c0 msgr2=0x7efec41f4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec40c81c0 0x7efec41f4120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efec4159580 tx=0x7efec406add0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 shutdown_connections 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efeb00682f0 0x7efeb006a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efec40c81c0 0x7efec41f4120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeac005200 0x7efec415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 --2- 172.21.15.153:0/2590657848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec40c7d70 0x7efec40ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 >> 172.21.15.153:0/2590657848 conn(0x7efec40b12b0 msgr2=0x7efec406dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 shutdown_connections 2023-11-28T13:00:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.375+0000 7efec9c21700 1 -- 172.21.15.153:0/2590657848 wait complete. 2023-11-28T13:00:47.388 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:47.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.511+0000 7f457f9c8700 1 Processor -- start 2023-11-28T13:00:47.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.512+0000 7f457f9c8700 1 -- start start 2023-11-28T13:00:47.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.512+0000 7f457f9c8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f457806ec90 0x7f4578065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.512+0000 7f457f9c8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f457806d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.512+0000 7f457f9c8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4578066580 con 0x7f45780cec70 2023-11-28T13:00:47.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.512+0000 7f457f9c8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45780cded0 con 0x7f457806ec90 2023-11-28T13:00:47.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.512+0000 7f457f9c8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780ce1b0 con 0x7f45780cb0e0 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457df65700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f457806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457d764700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f457806ec90 0x7f4578065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457cf63700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f45780cec70 0x7f45780ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:34024/0 (socket says 172.21.15.153:34024) 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457cf63700 1 -- 172.21.15.153:0/1727212341 learned_addr learned my addr 172.21.15.153:0/1727212341 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457df65700 1 -- 172.21.15.153:0/1727212341 >> v2:172.21.15.177:3300/0 conn(0x7f457806ec90 msgr2=0x7f4578065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457df65700 1 --2- 172.21.15.153:0/1727212341 >> v2:172.21.15.177:3300/0 conn(0x7f457806ec90 0x7f4578065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457df65700 1 -- 172.21.15.153:0/1727212341 >> v1:172.21.15.177:6789/0 conn(0x7f45780cec70 legacy=0x7f45780ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457df65700 1 -- 172.21.15.153:0/1727212341 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780ce490 con 0x7f45780cb0e0 2023-11-28T13:00:47.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.513+0000 7f457df65700 1 --2- 172.21.15.153:0/1727212341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f457806d230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4570009d90 tx=0x7f457000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90742a853358b929 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.514+0000 7f456ffff700 1 -- 172.21.15.153:0/1727212341 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457000e070 con 0x7f45780cb0e0 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.514+0000 7f456ffff700 1 -- 172.21.15.153:0/1727212341 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457000bb20 con 0x7f45780cb0e0 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.514+0000 7f456ffff700 1 -- 172.21.15.153:0/1727212341 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570012430 con 0x7f45780cb0e0 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 -- 172.21.15.153:0/1727212341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 msgr2=0x7f457806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1727212341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f457806d230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4570009d90 tx=0x7f457000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 -- 172.21.15.153:0/1727212341 shutdown_connections 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1727212341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f457806d230 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1727212341 >> v2:172.21.15.177:3300/0 conn(0x7f457806ec90 0x7f4578065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 -- 172.21.15.153:0/1727212341 >> 172.21.15.153:0/1727212341 conn(0x7f45780b12b0 msgr2=0x7f45780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:47.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 -- 172.21.15.153:0/1727212341 shutdown_connections 2023-11-28T13:00:47.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.515+0000 7f457f9c8700 1 -- 172.21.15.153:0/1727212341 wait complete. 2023-11-28T13:00:47.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.516+0000 7f457f9c8700 1 Processor -- start 2023-11-28T13:00:47.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.516+0000 7f457f9c8700 1 -- start start 2023-11-28T13:00:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457f9c8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f457806ec90 0x7f45780c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457f9c8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f45780c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457f9c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45780cec70 0x7f457814cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457f9c8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780c3a10 con 0x7f45780cec70 2023-11-28T13:00:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457f9c8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780c74b0 con 0x7f457806ec90 2023-11-28T13:00:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457f9c8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457814d040 con 0x7f45780cb0e0 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45780cec70 0x7f457814cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45780cec70 0x7f457814cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39254/0 (socket says 172.21.15.153:39254) 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 -- 172.21.15.153:0/1427941939 learned_addr learned my addr 172.21.15.153:0/1427941939 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457d764700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f457806ec90 0x7f45780c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457cf63700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f45780c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 -- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 msgr2=0x7f45780c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f45780c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 -- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f457806ec90 msgr2=0x7f45780c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f457806ec90 0x7f45780c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 -- 172.21.15.153:0/1427941939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570009a70 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457df65700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45780cec70 0x7f457814cb00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4570014040 tx=0x7f4570004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f456dffb700 1 -- 172.21.15.153:0/1427941939 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570012430 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f456dffb700 1 -- 172.21.15.153:0/1427941939 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4570020070 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45780cb6c0 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.517+0000 7f456dffb700 1 -- 172.21.15.153:0/1427941939 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570003ea0 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.518+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45780ce770 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.519+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457806f060 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.519+0000 7f456dffb700 1 -- 172.21.15.153:0/1427941939 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4570004040 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.519+0000 7f456dffb700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f455c0682f0 0x7f455c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.519+0000 7f456dffb700 1 -- 172.21.15.153:0/1427941939 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4570022070 con 0x7f45780cec70 2023-11-28T13:00:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.519+0000 7f457d764700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f455c0682f0 0x7f455c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.520+0000 7f457d764700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f455c0682f0 0x7f455c06a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4574009b40 tx=0x7f4574009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.524+0000 7f456dffb700 1 -- 172.21.15.153:0/1427941939 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4570054800 con 0x7f45780cec70 2023-11-28T13:00:47.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.707+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45780c3b50 con 0x7f45780cec70 2023-11-28T13:00:47.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.707+0000 7f456dffb700 1 -- 172.21.15.153:0/1427941939 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4570057e20 con 0x7f45780cec70 2023-11-28T13:00:47.708 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:47.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.709+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f455c0682f0 msgr2=0x7f455c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.709+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f455c0682f0 0x7f455c06a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4574009b40 tx=0x7f4574009210 comp rx=0 tx=0).stop 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.709+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45780cec70 msgr2=0x7f457814cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.709+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45780cec70 0x7f457814cb00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4570014040 tx=0x7f4570004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.709+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 shutdown_connections 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.709+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f455c0682f0 0x7f455c06a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.710+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45780cec70 0x7f457814cb00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.710+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f457806ec90 0x7f45780c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.710+0000 7f457f9c8700 1 --2- 172.21.15.153:0/1427941939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45780cb0e0 0x7f45780c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.710+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 >> 172.21.15.153:0/1427941939 conn(0x7f45780b12b0 msgr2=0x7f45780b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.710+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 shutdown_connections 2023-11-28T13:00:47.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.710+0000 7f457f9c8700 1 -- 172.21.15.153:0/1427941939 wait complete. 2023-11-28T13:00:47.756 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:00:47.756 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T13:00:47.877 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.876+0000 7fc86b13e700 1 Processor -- start 2023-11-28T13:00:47.877 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.876+0000 7fc86b13e700 1 -- start start 2023-11-28T13:00:47.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc86b13e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 0x7fc86406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc86b13e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86406ae60 con 0x7fc86406b830 2023-11-28T13:00:47.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc86b13e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86406a270 con 0x7fc86406d0a0 2023-11-28T13:00:47.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc86b13e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86406a530 con 0x7fc86406e9f0 2023-11-28T13:00:47.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc863fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc86406b830 0x7fc86406c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57424/0 (socket says 172.21.15.153:57424) 2023-11-28T13:00:47.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc863fff700 1 -- 172.21.15.153:0/2380437381 learned_addr learned my addr 172.21.15.153:0/2380437381 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:47.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477950027 0 0) 0x7fc86406ae60 con 0x7fc86406b830 2023-11-28T13:00:47.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc868eda700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 0x7fc86406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.877+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8540035b0 con 0x7fc86406b830 2023-11-28T13:00:47.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1199602132 0 0) 0x7fc8540035b0 con 0x7fc86406b830 2023-11-28T13:00:47.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 msgr2=0x7fc86406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:47.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 0x7fc86406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 >> v1:172.21.15.177:6789/0 conn(0x7fc86406d0a0 legacy=0x7fc8640ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86406a810 con 0x7fc86406b830 2023-11-28T13:00:47.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fc850001670 con 0x7fc86406b830 2023-11-28T13:00:47.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 >> v1:172.21.15.153:6789/0 conn(0x7fc86406b830 legacy=0x7fc86406c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854003a30 0x7fc854003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc854004850 0x7fc854004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 0x7fc854005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86406a530 con 0x7fc854004850 2023-11-28T13:00:47.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85400bac0 con 0x7fc854003a30 2023-11-28T13:00:47.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85400bc50 con 0x7fc86406e9f0 2023-11-28T13:00:47.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc863fff700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc854004850 0x7fc854004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc868eda700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854003a30 0x7fc854003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8696db700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 0x7fc854005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc863fff700 1 -- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 msgr2=0x7fc854005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:47.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc863fff700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86406e9f0 0x7fc854005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc863fff700 1 -- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854003a30 msgr2=0x7fc854003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc863fff700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854003a30 0x7fc854003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc863fff700 1 -- 172.21.15.153:0/2380437381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540035b0 con 0x7fc854004850 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc863fff700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc854004850 0x7fc854004cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c2a0 tx=0x7fc854005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44eea4c20b4c3362 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850007c10 con 0x7fc854004850 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85000cdc0 con 0x7fc854004850 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.878+0000 7fc8637fe700 1 -- 172.21.15.153:0/2380437381 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850005ba0 con 0x7fc854004850 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.879+0000 7fc86b13e700 1 -- 172.21.15.153:0/2380437381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc854004850 msgr2=0x7fc854004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.879+0000 7fc86b13e700 1 --2- 172.21.15.153:0/2380437381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc854004850 0x7fc854004cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c2a0 tx=0x7fc854005880 comp rx=0 tx=0).stop 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.879+0000 7fc8696db700 1 -- 172.21.15.153:0/2380437381 reap_dead start 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.879+0000 7fc86b13e700 1 -- 172.21.15.153:0/2380437381 shutdown_connections 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.879+0000 7fc86b13e700 1 -- 172.21.15.153:0/2380437381 >> 172.21.15.153:0/2380437381 conn(0x7fc8640b12b0 msgr2=0x7fc8640b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:47.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.879+0000 7fc86b13e700 1 -- 172.21.15.153:0/2380437381 shutdown_connections 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.879+0000 7fc86b13e700 1 -- 172.21.15.153:0/2380437381 wait complete. 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.881+0000 7fc86b13e700 1 Processor -- start 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.881+0000 7fc86b13e700 1 -- start start 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc86b13e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854004850 0x7fc8640b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc86b13e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8640c2f90 0x7fc864164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc86b13e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8641651e0 0x7fc8640b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc86b13e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640b3db0 con 0x7fc8640c2f90 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc86b13e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640c5eb0 con 0x7fc854004850 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc86b13e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640c6190 con 0x7fc8641651e0 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8640c2f90 0x7fc864164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8640c2f90 0x7fc864164ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39272/0 (socket says 172.21.15.153:39272) 2023-11-28T13:00:47.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 -- 172.21.15.153:0/4083496499 learned_addr learned my addr 172.21.15.153:0/4083496499 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc868eda700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854004850 0x7fc8640b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc8696db700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8641651e0 0x7fc8640b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 -- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8641651e0 msgr2=0x7fc8640b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8641651e0 0x7fc8640b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 -- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854004850 msgr2=0x7fc8640b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854004850 0x7fc8640b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 -- 172.21.15.153:0/4083496499 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86406edc0 con 0x7fc8640c2f90 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc8696db700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8641651e0 0x7fc8640b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc863fff700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8640c2f90 0x7fc864164ca0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8540047a0 tx=0x7fc86406add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc8617fa700 1 -- 172.21.15.153:0/4083496499 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850001640 con 0x7fc8640c2f90 2023-11-28T13:00:47.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc8617fa700 1 -- 172.21.15.153:0/4083496499 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8500070d0 con 0x7fc8640c2f90 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc8617fa700 1 -- 172.21.15.153:0/4083496499 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850007440 con 0x7fc8640c2f90 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.882+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86406aaf0 con 0x7fc8640c2f90 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.883+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86406ef50 con 0x7fc8640c2f90 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.883+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc864071670 con 0x7fc8640c2f90 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.884+0000 7fc8617fa700 1 -- 172.21.15.153:0/4083496499 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc850010070 con 0x7fc8640c2f90 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.884+0000 7fc8617fa700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc8480682f0 0x7fc84806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.884+0000 7fc8617fa700 1 -- 172.21.15.153:0/4083496499 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8500870f0 con 0x7fc8640c2f90 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.884+0000 7fc868eda700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc8480682f0 0x7fc84806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:47.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.885+0000 7fc868eda700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc8480682f0 0x7fc84806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc858009c60 tx=0x7fc858009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:47.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:47.888+0000 7fc8617fa700 1 -- 172.21.15.153:0/4083496499 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8500538d0 con 0x7fc8640c2f90 2023-11-28T13:00:48.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.143+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fc8640cf2d0 con 0x7fc8640c2f90 2023-11-28T13:00:48.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.518+0000 7fc8617fa700 1 -- 172.21.15.153:0/4083496499 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v49) v1 ==== 145+0+0 (secure 0 0 0) 0x7fc85001a380 con 0x7fc8640c2f90 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc8480682f0 msgr2=0x7fc84806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc8480682f0 0x7fc84806a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc858009c60 tx=0x7fc858009210 comp rx=0 tx=0).stop 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8640c2f90 msgr2=0x7fc864164ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8640c2f90 0x7fc864164ca0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8540047a0 tx=0x7fc86406add0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 shutdown_connections 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8641651e0 0x7fc8640b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc854004850 0x7fc8640b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc8480682f0 0x7fc84806a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 --2- 172.21.15.153:0/4083496499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc8640c2f90 0x7fc864164ca0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 >> 172.21.15.153:0/4083496499 conn(0x7fc8640b12b0 msgr2=0x7fc8640c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:48.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 shutdown_connections 2023-11-28T13:00:48.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.521+0000 7fc86b13e700 1 -- 172.21.15.153:0/4083496499 wait complete. 2023-11-28T13:00:48.535 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:00:48.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.657+0000 7f9f909ae700 1 Processor -- start 2023-11-28T13:00:48.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.657+0000 7f9f909ae700 1 -- start start 2023-11-28T13:00:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f909ae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f8c06dee0 0x7f9f8c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f909ae700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9f8c160540 0x7f9f8c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f909ae700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c15d060 con 0x7f9f8c071040 2023-11-28T13:00:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f909ae700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8c15d340 con 0x7f9f8c160540 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f909ae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8c15d620 con 0x7f9f8c06dee0 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f8ad9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9f8c160540 0x7f9f8c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f8ad9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9f8c160540 0x7f9f8c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39282/0 (socket says 172.21.15.153:39282) 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.658+0000 7f9f8ad9d700 1 -- 172.21.15.153:0/2423443926 learned_addr learned my addr 172.21.15.153:0/2423443926 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8ad9d700 1 -- 172.21.15.153:0/2423443926 >> v2:172.21.15.177:3300/0 conn(0x7f9f8c06dee0 msgr2=0x7f9f8c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8ad9d700 1 --2- 172.21.15.153:0/2423443926 >> v2:172.21.15.177:3300/0 conn(0x7f9f8c06dee0 0x7f9f8c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8ad9d700 1 -- 172.21.15.153:0/2423443926 >> v1:172.21.15.177:6789/0 conn(0x7f9f8c071040 legacy=0x7f9f8c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8ad9d700 1 -- 172.21.15.153:0/2423443926 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c15d900 con 0x7f9f8c160540 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8ad9d700 1 --2- 172.21.15.153:0/2423443926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c160540 0x7f9f8c160980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c06fb00 tx=0x7f9f8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304b2da76e189dc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8959a700 1 -- 172.21.15.153:0/2423443926 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400e070 con 0x7f9f8c160540 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8959a700 1 -- 172.21.15.153:0/2423443926 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f84003a20 con 0x7f9f8c160540 2023-11-28T13:00:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f8959a700 1 -- 172.21.15.153:0/2423443926 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84003dd0 con 0x7f9f8c160540 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f909ae700 1 -- 172.21.15.153:0/2423443926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c160540 msgr2=0x7f9f8c160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.659+0000 7f9f909ae700 1 --2- 172.21.15.153:0/2423443926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c160540 0x7f9f8c160980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c06fb00 tx=0x7f9f8400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.660+0000 7f9f909ae700 1 -- 172.21.15.153:0/2423443926 shutdown_connections 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.660+0000 7f9f909ae700 1 --2- 172.21.15.153:0/2423443926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c160540 0x7f9f8c160980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.660+0000 7f9f909ae700 1 --2- 172.21.15.153:0/2423443926 >> v2:172.21.15.177:3300/0 conn(0x7f9f8c06dee0 0x7f9f8c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.660+0000 7f9f909ae700 1 -- 172.21.15.153:0/2423443926 >> 172.21.15.153:0/2423443926 conn(0x7f9f8c0d0000 msgr2=0x7f9f8c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.660+0000 7f9f909ae700 1 -- 172.21.15.153:0/2423443926 shutdown_connections 2023-11-28T13:00:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.660+0000 7f9f909ae700 1 -- 172.21.15.153:0/2423443926 wait complete. 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.661+0000 7f9f909ae700 1 Processor -- start 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.661+0000 7f9f909ae700 1 -- start start 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f909ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8c06dee0 0x7f9f8c148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f909ae700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c071040 0x7f9f8c15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f909ae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f8c160540 0x7f9f8c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f909ae700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b5fd0 con 0x7f9f8c071040 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f909ae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0c4e30 con 0x7f9f8c160540 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f909ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b4f30 con 0x7f9f8c06dee0 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f89d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c071040 0x7f9f8c15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f89d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c071040 0x7f9f8c15f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39286/0 (socket says 172.21.15.153:39286) 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.662+0000 7f9f89d9b700 1 -- 172.21.15.153:0/4280352229 learned_addr learned my addr 172.21.15.153:0/4280352229 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f8ad9d700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f8c160540 0x7f9f8c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f8a59c700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8c06dee0 0x7f9f8c148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f89d9b700 1 -- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8c06dee0 msgr2=0x7f9f8c148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f89d9b700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8c06dee0 0x7f9f8c148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f89d9b700 1 -- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f8c160540 msgr2=0x7f9f8c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f89d9b700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f8c160540 0x7f9f8c15f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f89d9b700 1 -- 172.21.15.153:0/4280352229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f84009b30 con 0x7f9f8c071040 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f8a59c700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8c06dee0 0x7f9f8c148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f8ad9d700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f8c160540 0x7f9f8c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f89d9b700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c071040 0x7f9f8c15f470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00b4a0 tx=0x7f9f7c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f7affd700 1 -- 172.21.15.153:0/4280352229 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c011830 con 0x7f9f8c071040 2023-11-28T13:00:48.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f7affd700 1 -- 172.21.15.153:0/4280352229 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c011990 con 0x7f9f8c071040 2023-11-28T13:00:48.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c15ccd0 con 0x7f9f8c071040 2023-11-28T13:00:48.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f7affd700 1 -- 172.21.15.153:0/4280352229 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c011d40 con 0x7f9f8c071040 2023-11-28T13:00:48.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.663+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c06e870 con 0x7f9f8c071040 2023-11-28T13:00:48.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.664+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c06a270 con 0x7f9f8c071040 2023-11-28T13:00:48.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.665+0000 7f9f7affd700 1 -- 172.21.15.153:0/4280352229 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f7c021070 con 0x7f9f8c071040 2023-11-28T13:00:48.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.665+0000 7f9f7affd700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9f6c0682f0 0x7f9f6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.665+0000 7f9f7affd700 1 -- 172.21.15.153:0/4280352229 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f7c085b80 con 0x7f9f8c071040 2023-11-28T13:00:48.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.665+0000 7f9f8a59c700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9f6c0682f0 0x7f9f6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.665+0000 7f9f8a59c700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9f6c0682f0 0x7f9f6c06a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0cfe90 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:48.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.669+0000 7f9f7affd700 1 -- 172.21.15.153:0/4280352229 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9f7c052360 con 0x7f9f8c071040 2023-11-28T13:00:48.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.850+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9f8c0720a0 con 0x7f9f8c071040 2023-11-28T13:00:48.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.854+0000 7f9f7affd700 1 -- 172.21.15.153:0/4280352229 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9f7c0150c0 con 0x7f9f8c071040 2023-11-28T13:00:48.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9f6c0682f0 msgr2=0x7f9f6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9f6c0682f0 0x7f9f6c06a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0cfe90 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c071040 msgr2=0x7f9f8c15f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c071040 0x7f9f8c15f470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00b4a0 tx=0x7f9f7c00b860 comp rx=0 tx=0).stop 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 shutdown_connections 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f8c160540 0x7f9f8c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8c06dee0 0x7f9f8c148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9f6c0682f0 0x7f9f6c06a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 --2- 172.21.15.153:0/4280352229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f8c071040 0x7f9f8c15f470 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 >> 172.21.15.153:0/4280352229 conn(0x7f9f8c0d0000 msgr2=0x7f9f8c15e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 shutdown_connections 2023-11-28T13:00:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:48.856+0000 7f9f909ae700 1 -- 172.21.15.153:0/4280352229 wait complete. 2023-11-28T13:00:48.870 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:00:48.870 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:00:48.905 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:00:48.954 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:00:48.954 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:48.954 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:48.972 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:48.972 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:49.030 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:49.030 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:49.089 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:49.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.249+0000 7fcbcfc57700 1 Processor -- start 2023-11-28T13:00:49.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.249+0000 7fcbcfc57700 1 -- start start 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcfc57700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcbc80d01b0 0x7fcbc80d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcfc57700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcbc80b20b0 0x7fcbc80d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcfc57700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcfc57700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc815cec0 con 0x7fcbc80d01b0 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcfc57700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc80b0230 con 0x7fcbc80b20b0 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcfc57700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b4570 con 0x7fcbc80b29b0 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcbc80d01b0 0x7fcbc80d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcbc80d01b0 0x7fcbc80d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39292/0 (socket says 172.21.15.153:39292) 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 -- 172.21.15.153:0/2320163616 learned_addr learned my addr 172.21.15.153:0/2320163616 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:49.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd1f2700 1 --2- 172.21.15.153:0/2320163616 >> v2:172.21.15.177:3300/0 conn(0x7fcbc80b20b0 0x7fcbc80d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbce1f4700 1 --2- 172.21.15.153:0/2320163616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 -- 172.21.15.153:0/2320163616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 msgr2=0x7fcbc80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 --2- 172.21.15.153:0/2320163616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 -- 172.21.15.153:0/2320163616 >> v2:172.21.15.177:3300/0 conn(0x7fcbc80b20b0 msgr2=0x7fcbc80d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 --2- 172.21.15.153:0/2320163616 >> v2:172.21.15.177:3300/0 conn(0x7fcbc80b20b0 0x7fcbc80d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 -- 172.21.15.153:0/2320163616 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80b4850 con 0x7fcbc80d01b0 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbce1f4700 1 --2- 172.21.15.153:0/2320163616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd1f2700 1 --2- 172.21.15.153:0/2320163616 >> v2:172.21.15.177:3300/0 conn(0x7fcbc80b20b0 0x7fcbc80d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:49.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.250+0000 7fcbcd9f3700 1 --2- 172.21.15.153:0/2320163616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80d0c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80ca0b0 tx=0x7fcbc4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c43d3f98114a353 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcc9f1700 1 -- 172.21.15.153:0/2320163616 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc400b870 con 0x7fcbc80d01b0 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcc9f1700 1 -- 172.21.15.153:0/2320163616 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc400b9d0 con 0x7fcbc80d01b0 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcc9f1700 1 -- 172.21.15.153:0/2320163616 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc400bd60 con 0x7fcbc80d01b0 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 -- 172.21.15.153:0/2320163616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 msgr2=0x7fcbc80d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/2320163616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80d0c20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80ca0b0 tx=0x7fcbc4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 -- 172.21.15.153:0/2320163616 shutdown_connections 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/2320163616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/2320163616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80d0c20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/2320163616 >> v2:172.21.15.177:3300/0 conn(0x7fcbc80b20b0 0x7fcbc80d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 -- 172.21.15.153:0/2320163616 >> 172.21.15.153:0/2320163616 conn(0x7fcbc809c3f0 msgr2=0x7fcbc80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 -- 172.21.15.153:0/2320163616 shutdown_connections 2023-11-28T13:00:49.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.251+0000 7fcbcfc57700 1 -- 172.21.15.153:0/2320163616 wait complete. 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 Processor -- start 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 -- start start 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc80b20b0 0x7fcbc8158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc81e5b20 con 0x7fcbc80d01b0 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b6db0 con 0x7fcbc80b20b0 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbcfc57700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b7090 con 0x7fcbc80b29b0 2023-11-28T13:00:49.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbce1f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.253+0000 7fcbce1f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80b88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39300/0 (socket says 172.21.15.153:39300) 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbce1f4700 1 -- 172.21.15.153:0/598704532 learned_addr learned my addr 172.21.15.153:0/598704532 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbcd1f2700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbcd9f3700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc80b20b0 0x7fcbc8158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbce1f4700 1 -- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 msgr2=0x7fcbc80b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbce1f4700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbce1f4700 1 -- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc80b20b0 msgr2=0x7fcbc8158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbce1f4700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc80b20b0 0x7fcbc8158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbce1f4700 1 -- 172.21.15.153:0/598704532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4009550 con 0x7fcbc80d01b0 2023-11-28T13:00:49.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbce1f4700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80b88a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0007ef0 tx=0x7fcbc000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbbe7fc700 1 -- 172.21.15.153:0/598704532 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000dd80 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbbe7fc700 1 -- 172.21.15.153:0/598704532 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc000f040 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbbe7fc700 1 -- 172.21.15.153:0/598704532 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0003c60 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc80afec0 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.254+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc80afbe0 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.255+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc80b2600 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.255+0000 7fcbbe7fc700 1 -- 172.21.15.153:0/598704532 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbc001c030 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.256+0000 7fcbbe7fc700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbac068320 0x7fcbac06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.256+0000 7fcbbe7fc700 1 -- 172.21.15.153:0/598704532 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbc0085ba0 con 0x7fcbc80d01b0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.256+0000 7fcbcd9f3700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbac068320 0x7fcbac06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.256+0000 7fcbcd9f3700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbac068320 0x7fcbac06a7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4014040 tx=0x7fcbc4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.260+0000 7fcbbe7fc700 1 -- 172.21.15.153:0/598704532 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbc0052400 con 0x7fcbc80d01b0 2023-11-28T13:00:49.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.444+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc8068690 con 0x7fcbc80d01b0 2023-11-28T13:00:49.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.445+0000 7fcbbe7fc700 1 -- 172.21.15.153:0/598704532 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fcbc0055a20 con 0x7fcbc80d01b0 2023-11-28T13:00:49.447 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:49.447 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:49.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbac068320 msgr2=0x7fcbac06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbac068320 0x7fcbac06a7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4014040 tx=0x7fcbc4008030 comp rx=0 tx=0).stop 2023-11-28T13:00:49.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 msgr2=0x7fcbc80b88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80b88a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0007ef0 tx=0x7fcbc000d3b0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 shutdown_connections 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbac068320 0x7fcbac06a7d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbc80d01b0 0x7fcbc80b88a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbc80b20b0 0x7fcbc8158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 --2- 172.21.15.153:0/598704532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbc80b29b0 0x7fcbc80b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 >> 172.21.15.153:0/598704532 conn(0x7fcbc809c3f0 msgr2=0x7fcbc80b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 shutdown_connections 2023-11-28T13:00:49.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.447+0000 7fcbcfc57700 1 -- 172.21.15.153:0/598704532 wait complete. 2023-11-28T13:00:49.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:49.462 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:49.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.585+0000 7f8686632700 1 Processor -- start 2023-11-28T13:00:49.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.585+0000 7f8686632700 1 -- start start 2023-11-28T13:00:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f8686632700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f868006dee0 0x7f868006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f8686632700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8680160540 0x7f8680160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f8686632700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868015d060 con 0x7f8680071040 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f8686632700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868015d340 con 0x7f868006dee0 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f8686632700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868015d620 con 0x7f8680160540 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f868006dee0 0x7f868006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8680071040 0x7f8680071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57468/0 (socket says 172.21.15.153:57468) 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867ffff700 1 -- 172.21.15.153:0/657416577 learned_addr learned my addr 172.21.15.153:0/657416577 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f8684bcf700 1 --2- 172.21.15.153:0/657416577 >> v2:172.21.15.177:3300/0 conn(0x7f8680160540 0x7f8680160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867f7fe700 1 -- 172.21.15.153:0/657416577 >> v2:172.21.15.177:3300/0 conn(0x7f8680160540 msgr2=0x7f8680160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867f7fe700 1 --2- 172.21.15.153:0/657416577 >> v2:172.21.15.177:3300/0 conn(0x7f8680160540 0x7f8680160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867f7fe700 1 -- 172.21.15.153:0/657416577 >> v1:172.21.15.153:6789/0 conn(0x7f8680071040 legacy=0x7f8680071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867effd700 1 -- 172.21.15.153:0/657416577 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796422780 0 0) 0x7f868015d060 con 0x7f8680071040 2023-11-28T13:00:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867f7fe700 1 -- 172.21.15.153:0/657416577 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868015d900 con 0x7f868006dee0 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867f7fe700 1 --2- 172.21.15.153:0/657416577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f868006e330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f86800c3a10 tx=0x7f8670009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e00c46670eb490 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.586+0000 7f867effd700 1 -- 172.21.15.153:0/657416577 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670011830 con 0x7f868006dee0 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f867effd700 1 -- 172.21.15.153:0/657416577 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8670011990 con 0x7f868006dee0 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f867effd700 1 -- 172.21.15.153:0/657416577 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670011d20 con 0x7f868006dee0 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 -- 172.21.15.153:0/657416577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 msgr2=0x7f868006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 --2- 172.21.15.153:0/657416577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f868006e330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f86800c3a10 tx=0x7f8670009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 -- 172.21.15.153:0/657416577 shutdown_connections 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 --2- 172.21.15.153:0/657416577 >> v2:172.21.15.177:3300/0 conn(0x7f8680160540 0x7f8680160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 --2- 172.21.15.153:0/657416577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f868006e330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 -- 172.21.15.153:0/657416577 >> 172.21.15.153:0/657416577 conn(0x7f86800d0000 msgr2=0x7f86800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 -- 172.21.15.153:0/657416577 shutdown_connections 2023-11-28T13:00:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.587+0000 7f8686632700 1 -- 172.21.15.153:0/657416577 wait complete. 2023-11-28T13:00:49.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.589+0000 7f8686632700 1 Processor -- start 2023-11-28T13:00:49.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.589+0000 7f8686632700 1 -- start start 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8686632700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f8680158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8686632700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8680071040 0x7f868015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8686632700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8680160540 0x7f868015f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8686632700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b4770 con 0x7f868006dee0 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8686632700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b3b60 con 0x7f8680071040 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8686632700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b3e40 con 0x7f8680160540 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f8680158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f8680158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39310/0 (socket says 172.21.15.153:39310) 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 -- 172.21.15.153:0/4180740361 learned_addr learned my addr 172.21.15.153:0/4180740361 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867f7fe700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8680071040 0x7f868015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8684bcf700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8680160540 0x7f868015f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 -- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8680160540 msgr2=0x7f868015f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8680160540 0x7f868015f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 -- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8680071040 msgr2=0x7f868015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8680071040 0x7f868015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 -- 172.21.15.153:0/4180740361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86700094a0 con 0x7f868006dee0 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8684bcf700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8680160540 0x7f868015f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867ffff700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f8680158040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f867400c550 tx=0x7f867400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867cff9700 1 -- 172.21.15.153:0/4180740361 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674004c80 con 0x7f868006dee0 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f867cff9700 1 -- 172.21.15.153:0/4180740361 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674004e00 con 0x7f868006dee0 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.590+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868015ccd0 con 0x7f868006dee0 2023-11-28T13:00:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.591+0000 7f867cff9700 1 -- 172.21.15.153:0/4180740361 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674007310 con 0x7f868006dee0 2023-11-28T13:00:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.591+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868006e870 con 0x7f868006dee0 2023-11-28T13:00:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.592+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868006a270 con 0x7f868006dee0 2023-11-28T13:00:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.592+0000 7f867cff9700 1 -- 172.21.15.153:0/4180740361 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f867401a020 con 0x7f868006dee0 2023-11-28T13:00:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.592+0000 7f867cff9700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f866c0682f0 0x7f866c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.592+0000 7f867cff9700 1 -- 172.21.15.153:0/4180740361 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8674085800 con 0x7f868006dee0 2023-11-28T13:00:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.592+0000 7f867f7fe700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f866c0682f0 0x7f866c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.593+0000 7f867f7fe700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f866c0682f0 0x7f866c06a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f86700090b0 tx=0x7f867001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.596+0000 7f867cff9700 1 -- 172.21.15.153:0/4180740361 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8674052000 con 0x7f868006dee0 2023-11-28T13:00:49.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.782+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86800720a0 con 0x7f868006dee0 2023-11-28T13:00:49.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.783+0000 7f867cff9700 1 -- 172.21.15.153:0/4180740361 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8674055620 con 0x7f868006dee0 2023-11-28T13:00:49.784 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.784+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f866c0682f0 msgr2=0x7f866c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.784+0000 7f8686632700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f866c0682f0 0x7f866c06a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f86700090b0 tx=0x7f867001c040 comp rx=0 tx=0).stop 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.784+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 msgr2=0x7f8680158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.784+0000 7f8686632700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f8680158040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f867400c550 tx=0x7f867400c860 comp rx=0 tx=0).stop 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 shutdown_connections 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f866c0682f0 0x7f866c06a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f868006dee0 0x7f8680158040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8680160540 0x7f868015f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 --2- 172.21.15.153:0/4180740361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8680071040 0x7f868015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 >> 172.21.15.153:0/4180740361 conn(0x7f86800d0000 msgr2=0x7f868015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:49.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 shutdown_connections 2023-11-28T13:00:49.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.785+0000 7f8686632700 1 -- 172.21.15.153:0/4180740361 wait complete. 2023-11-28T13:00:49.797 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:49.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.918+0000 7f25a0b4a700 1 Processor -- start 2023-11-28T13:00:49.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.918+0000 7f25a0b4a700 1 -- start start 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f25a0b4a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f259c071040 0x7f259c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f25a0b4a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c160540 0x7f259c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f25a0b4a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c071950 con 0x7f259c06dee0 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f25a0b4a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c15d060 con 0x7f259c071040 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f25a0b4a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c15d340 con 0x7f259c160540 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f259a59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f259c071040 0x7f259c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f2599d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f259c06dee0 0x7f259c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57482/0 (socket says 172.21.15.153:57482) 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.919+0000 7f2599d9b700 1 -- 172.21.15.153:0/2084605536 learned_addr learned my addr 172.21.15.153:0/2084605536 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259ad9d700 1 --2- 172.21.15.153:0/2084605536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c160540 0x7f259c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259a59c700 1 -- 172.21.15.153:0/2084605536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c160540 msgr2=0x7f259c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259a59c700 1 --2- 172.21.15.153:0/2084605536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c160540 0x7f259c1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259a59c700 1 -- 172.21.15.153:0/2084605536 >> v1:172.21.15.153:6789/0 conn(0x7f259c06dee0 legacy=0x7f259c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259a59c700 1 -- 172.21.15.153:0/2084605536 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c15d620 con 0x7f259c071040 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259ad9d700 1 --2- 172.21.15.153:0/2084605536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c160540 0x7f259c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259a59c700 1 --2- 172.21.15.153:0/2084605536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c071410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2590009840 tx=0x7f2590009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ee262c2c63469a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259959a700 1 -- 172.21.15.153:0/2084605536 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590011830 con 0x7f259c071040 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259959a700 1 -- 172.21.15.153:0/2084605536 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590011990 con 0x7f259c071040 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.920+0000 7f259959a700 1 -- 172.21.15.153:0/2084605536 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590011d20 con 0x7f259c071040 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 -- 172.21.15.153:0/2084605536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 msgr2=0x7f259c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/2084605536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c071410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2590009840 tx=0x7f2590009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:49.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 -- 172.21.15.153:0/2084605536 shutdown_connections 2023-11-28T13:00:49.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/2084605536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c160540 0x7f259c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/2084605536 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c071410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 -- 172.21.15.153:0/2084605536 >> 172.21.15.153:0/2084605536 conn(0x7f259c0d0000 msgr2=0x7f259c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:49.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 -- 172.21.15.153:0/2084605536 shutdown_connections 2023-11-28T13:00:49.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.921+0000 7f25a0b4a700 1 -- 172.21.15.153:0/2084605536 wait complete. 2023-11-28T13:00:49.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.922+0000 7f25a0b4a700 1 Processor -- start 2023-11-28T13:00:49.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.922+0000 7f25a0b4a700 1 -- start start 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c06dee0 0x7f259c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f259c160540 0x7f259c0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0b32b0 con 0x7f259c071040 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c3aa0 con 0x7f259c160540 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c15b550 con 0x7f259c06dee0 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c0c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39326/0 (socket says 172.21.15.153:39326) 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 -- 172.21.15.153:0/359959691 learned_addr learned my addr 172.21.15.153:0/359959691 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f259a59c700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c06dee0 0x7f259c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f259ad9d700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f259c160540 0x7f259c0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 -- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c06dee0 msgr2=0x7f259c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c06dee0 0x7f259c15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 -- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f259c160540 msgr2=0x7f259c0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f259c160540 0x7f259c0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 -- 172.21.15.153:0/359959691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900094a0 con 0x7f259c071040 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f259ad9d700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f259c160540 0x7f259c0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f2599d9b700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c0c5eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c480 tx=0x7f258c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f258affd700 1 -- 172.21.15.153:0/359959691 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c005d70 con 0x7f259c071040 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f258affd700 1 -- 172.21.15.153:0/359959691 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c0070d0 con 0x7f259c071040 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f258affd700 1 -- 172.21.15.153:0/359959691 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c007440 con 0x7f259c071040 2023-11-28T13:00:49.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c15ccd0 con 0x7f259c071040 2023-11-28T13:00:49.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.923+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c15d900 con 0x7f259c071040 2023-11-28T13:00:49.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.924+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c06a270 con 0x7f259c071040 2023-11-28T13:00:49.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.925+0000 7f258affd700 1 -- 172.21.15.153:0/359959691 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f258c019030 con 0x7f259c071040 2023-11-28T13:00:49.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.925+0000 7f258affd700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f257c0682f0 0x7f257c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:49.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.925+0000 7f258affd700 1 -- 172.21.15.153:0/359959691 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f258c084df0 con 0x7f259c071040 2023-11-28T13:00:49.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.925+0000 7f259a59c700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f257c0682f0 0x7f257c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:49.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.926+0000 7f259a59c700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f257c0682f0 0x7f257c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25900090b0 tx=0x7f259001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:49.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:49.930+0000 7f258affd700 1 -- 172.21.15.153:0/359959691 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f258c04b5e0 con 0x7f259c071040 2023-11-28T13:00:50.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.108+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f259c06c360 con 0x7f259c071040 2023-11-28T13:00:50.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.108+0000 7f258affd700 1 -- 172.21.15.153:0/359959691 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f258c0170a0 con 0x7f259c071040 2023-11-28T13:00:50.110 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:50.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f257c0682f0 msgr2=0x7f257c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f257c0682f0 0x7f257c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25900090b0 tx=0x7f259001c040 comp rx=0 tx=0).stop 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 msgr2=0x7f259c0c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c0c5eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c480 tx=0x7f258c00c790 comp rx=0 tx=0).stop 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 shutdown_connections 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f259c160540 0x7f259c0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f259c06dee0 0x7f259c15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f257c0682f0 0x7f257c06a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 --2- 172.21.15.153:0/359959691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f259c071040 0x7f259c0c5eb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 >> 172.21.15.153:0/359959691 conn(0x7f259c0d0000 msgr2=0x7f259c15e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:50.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 shutdown_connections 2023-11-28T13:00:50.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.111+0000 7f25a0b4a700 1 -- 172.21.15.153:0/359959691 wait complete. 2023-11-28T13:00:50.123 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:50.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.246+0000 7fe79b677700 1 Processor -- start 2023-11-28T13:00:50.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.247+0000 7fe79b677700 1 -- start start 2023-11-28T13:00:50.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.247+0000 7fe79b677700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794071040 0x7fe794071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.247+0000 7fe79b677700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe794071950 con 0x7fe79406dee0 2023-11-28T13:00:50.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.247+0000 7fe79b677700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79415d060 con 0x7fe794160540 2023-11-28T13:00:50.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.247+0000 7fe79b677700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79415d340 con 0x7fe794071040 2023-11-28T13:00:50.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe798c12700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe79406dee0 0x7fe79406e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57496/0 (socket says 172.21.15.153:57496) 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe798c12700 1 -- 172.21.15.153:0/1817821379 learned_addr learned my addr 172.21.15.153:0/1817821379 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe799413700 1 --2- 172.21.15.153:0/1817821379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794071040 0x7fe794071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe78bfff700 1 -- 172.21.15.153:0/1817821379 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066079038 0 0) 0x7fe794071950 con 0x7fe79406dee0 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe78bfff700 1 -- 172.21.15.153:0/1817821379 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7800035b0 con 0x7fe79406dee0 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe799413700 1 -- 172.21.15.153:0/1817821379 >> v1:172.21.15.177:6789/0 conn(0x7fe794160540 legacy=0x7fe7941609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe799413700 1 -- 172.21.15.153:0/1817821379 >> v1:172.21.15.153:6789/0 conn(0x7fe79406dee0 legacy=0x7fe79406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe799413700 1 -- 172.21.15.153:0/1817821379 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79415d620 con 0x7fe794071040 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe799413700 1 --2- 172.21.15.153:0/1817821379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794071040 0x7fe794071410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe784009870 tx=0x7fe784009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=667264ca26a7fd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe78bfff700 1 -- 172.21.15.153:0/1817821379 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784011830 con 0x7fe794071040 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe78bfff700 1 -- 172.21.15.153:0/1817821379 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784011990 con 0x7fe794071040 2023-11-28T13:00:50.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.248+0000 7fe78bfff700 1 -- 172.21.15.153:0/1817821379 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784011d20 con 0x7fe794071040 2023-11-28T13:00:50.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.249+0000 7fe79b677700 1 -- 172.21.15.153:0/1817821379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794071040 msgr2=0x7fe794071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.249+0000 7fe79b677700 1 --2- 172.21.15.153:0/1817821379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794071040 0x7fe794071410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe784009870 tx=0x7fe784009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:50.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.249+0000 7fe79b677700 1 -- 172.21.15.153:0/1817821379 shutdown_connections 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.249+0000 7fe79b677700 1 --2- 172.21.15.153:0/1817821379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794071040 0x7fe794071410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.249+0000 7fe79b677700 1 -- 172.21.15.153:0/1817821379 >> 172.21.15.153:0/1817821379 conn(0x7fe7940d0000 msgr2=0x7fe7940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.249+0000 7fe79b677700 1 -- 172.21.15.153:0/1817821379 shutdown_connections 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.249+0000 7fe79b677700 1 -- 172.21.15.153:0/1817821379 wait complete. 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.250+0000 7fe79b677700 1 Processor -- start 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.250+0000 7fe79b677700 1 -- start start 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe79b677700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe79406dee0 0x7fe79415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe79b677700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe794071040 0x7fe7940b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe79b677700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794160540 0x7fe7940b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe79b677700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940b4190 con 0x7fe79406dee0 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe79b677700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940b3510 con 0x7fe794071040 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe79b677700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940b37f0 con 0x7fe794160540 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe79406dee0 0x7fe79415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe79406dee0 0x7fe79415a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39338/0 (socket says 172.21.15.153:39338) 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 -- 172.21.15.153:0/464417804 learned_addr learned my addr 172.21.15.153:0/464417804 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe798c12700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe794071040 0x7fe7940b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 -- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794160540 msgr2=0x7fe7940b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799c14700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794160540 0x7fe7940b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794160540 0x7fe7940b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 -- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe794071040 msgr2=0x7fe7940b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe794071040 0x7fe7940b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.251+0000 7fe799413700 1 -- 172.21.15.153:0/464417804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7840094a0 con 0x7fe79406dee0 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.252+0000 7fe799413700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe79406dee0 0x7fe79415a310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe784009bc0 tx=0x7fe784004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.252+0000 7fe789ffb700 1 -- 172.21.15.153:0/464417804 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7840038a0 con 0x7fe79406dee0 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.252+0000 7fe789ffb700 1 -- 172.21.15.153:0/464417804 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78401e070 con 0x7fe79406dee0 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.252+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79415ccd0 con 0x7fe79406dee0 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.252+0000 7fe789ffb700 1 -- 172.21.15.153:0/464417804 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784003c40 con 0x7fe79406dee0 2023-11-28T13:00:50.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.252+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79415d900 con 0x7fe79406dee0 2023-11-28T13:00:50.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.253+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79406a270 con 0x7fe79406dee0 2023-11-28T13:00:50.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.253+0000 7fe789ffb700 1 -- 172.21.15.153:0/464417804 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe78400f050 con 0x7fe79406dee0 2023-11-28T13:00:50.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.254+0000 7fe789ffb700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe7780682f0 0x7fe77806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.254+0000 7fe789ffb700 1 -- 172.21.15.153:0/464417804 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe784086c70 con 0x7fe79406dee0 2023-11-28T13:00:50.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.254+0000 7fe798c12700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe7780682f0 0x7fe77806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.254+0000 7fe798c12700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe7780682f0 0x7fe77806a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe7940b1430 tx=0x7fe790009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.258+0000 7fe789ffb700 1 -- 172.21.15.153:0/464417804 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe784053450 con 0x7fe79406dee0 2023-11-28T13:00:50.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.438+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7940b4400 con 0x7fe79406dee0 2023-11-28T13:00:50.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.439+0000 7fe789ffb700 1 -- 172.21.15.153:0/464417804 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe784015020 con 0x7fe79406dee0 2023-11-28T13:00:50.440 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe7780682f0 msgr2=0x7fe77806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe7780682f0 0x7fe77806a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe7940b1430 tx=0x7fe790009040 comp rx=0 tx=0).stop 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe79406dee0 msgr2=0x7fe79415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe79406dee0 0x7fe79415a310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe784009bc0 tx=0x7fe784004750 comp rx=0 tx=0).stop 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 shutdown_connections 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe7780682f0 0x7fe77806a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe79406dee0 0x7fe79415a310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe794160540 0x7fe7940b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 --2- 172.21.15.153:0/464417804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe794071040 0x7fe7940b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 >> 172.21.15.153:0/464417804 conn(0x7fe7940d0000 msgr2=0x7fe79415f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 shutdown_connections 2023-11-28T13:00:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.441+0000 7fe79b677700 1 -- 172.21.15.153:0/464417804 wait complete. 2023-11-28T13:00:50.454 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.578+0000 7f47c4f76700 1 Processor -- start 2023-11-28T13:00:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.578+0000 7f47c4f76700 1 -- start start 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47c4f76700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f47c006dee0 0x7f47c006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47c4f76700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c015d060 con 0x7f47c0160540 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47c4f76700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c015d340 con 0x7f47c0071040 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47c4f76700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c015d620 con 0x7f47c006dee0 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47bed9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f47c0160540 0x7f47c0160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57514/0 (socket says 172.21.15.153:57514) 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47bed9d700 1 -- 172.21.15.153:0/2076724449 learned_addr learned my addr 172.21.15.153:0/2076724449 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/2076724449 >> v2:172.21.15.177:3300/0 conn(0x7f47c006dee0 0x7f47c006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.579+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246849288 0 0) 0x7f47c015d060 con 0x7f47c0160540 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a80035b0 con 0x7f47c0160540 2023-11-28T13:00:50.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2892354826 0 0) 0x7f47a80035b0 con 0x7f47c0160540 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 >> v2:172.21.15.177:3300/0 conn(0x7f47c006dee0 msgr2=0x7f47c006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 --2- 172.21.15.153:0/2076724449 >> v2:172.21.15.177:3300/0 conn(0x7f47c006dee0 0x7f47c006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 >> v1:172.21.15.177:6789/0 conn(0x7f47c0071040 legacy=0x7f47c0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c015d900 con 0x7f47c0160540 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f47b80019f0 con 0x7f47c0160540 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 >> v1:172.21.15.153:6789/0 conn(0x7f47c0160540 legacy=0x7f47c0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8003a00 0x7f47a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8004850 0x7f47a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47a8005320 0x7f47a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c015d620 con 0x7f47a8004850 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a800c250 con 0x7f47a8003a00 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a800c410 con 0x7f47a8005320 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8004850 0x7f47a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47be59c700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8003a00 0x7f47a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bed9d700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47a8005320 0x7f47a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bdd9b700 1 -- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47a8005320 msgr2=0x7f47a800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47a8005320 0x7f47a800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bdd9b700 1 -- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8003a00 msgr2=0x7f47a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8003a00 0x7f47a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bdd9b700 1 -- 172.21.15.153:0/2076724449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80035b0 con 0x7f47a8004850 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bed9d700 1 -- 172.21.15.153:0/2076724449 reap_dead start 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.580+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8004850 0x7f47a8004cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009b60 tx=0x7f47a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ba17bf57f556f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000e6c0 con 0x7f47a8004850 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b000e820 con 0x7f47a8004850 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47bd59a700 1 -- 172.21.15.153:0/2076724449 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000eb90 con 0x7f47a8004850 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47c4f76700 1 -- 172.21.15.153:0/2076724449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8004850 msgr2=0x7f47a8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47c4f76700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8004850 0x7f47a8004cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009b60 tx=0x7f47a8005880 comp rx=0 tx=0).stop 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47c4f76700 1 -- 172.21.15.153:0/2076724449 shutdown_connections 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47c4f76700 1 --2- 172.21.15.153:0/2076724449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47a8004850 0x7f47a8004cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47c4f76700 1 -- 172.21.15.153:0/2076724449 >> 172.21.15.153:0/2076724449 conn(0x7f47c00d0000 msgr2=0x7f47c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47c4f76700 1 -- 172.21.15.153:0/2076724449 shutdown_connections 2023-11-28T13:00:50.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.581+0000 7f47c4f76700 1 -- 172.21.15.153:0/2076724449 wait complete. 2023-11-28T13:00:50.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.582+0000 7f47c4f76700 1 Processor -- start 2023-11-28T13:00:50.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47c4f76700 1 -- start start 2023-11-28T13:00:50.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47c4f76700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8004850 0x7f47c0153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47c4f76700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c0155a00 0x7f47c0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47c4f76700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c015e0f0 0x7f47c015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47c4f76700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c015ead0 con 0x7f47c0155a00 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47c4f76700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01539a0 con 0x7f47a8004850 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47c4f76700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01f2680 con 0x7f47c015e0f0 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c0155a00 0x7f47c0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c0155a00 0x7f47c0154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39360/0 (socket says 172.21.15.153:39360) 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.583+0000 7f47bdd9b700 1 -- 172.21.15.153:0/4049589813 learned_addr learned my addr 172.21.15.153:0/4049589813 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47bed9d700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c015e0f0 0x7f47c015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47be59c700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8004850 0x7f47c0153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47bdd9b700 1 -- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c015e0f0 msgr2=0x7f47c015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c015e0f0 0x7f47c015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47bdd9b700 1 -- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8004850 msgr2=0x7f47c0153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8004850 0x7f47c0153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47bdd9b700 1 -- 172.21.15.153:0/4049589813 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c006a270 con 0x7f47c0155a00 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47be59c700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8004850 0x7f47c0153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47bdd9b700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c0155a00 0x7f47c0154d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47a800c220 tx=0x7f47b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47aeffd700 1 -- 172.21.15.153:0/4049589813 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000e6c0 con 0x7f47c0155a00 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47aeffd700 1 -- 172.21.15.153:0/4049589813 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0003ad0 con 0x7f47c0155a00 2023-11-28T13:00:50.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c015ccd0 con 0x7f47c0155a00 2023-11-28T13:00:50.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47aeffd700 1 -- 172.21.15.153:0/4049589813 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0003e80 con 0x7f47c0155a00 2023-11-28T13:00:50.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.584+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c01f2c10 con 0x7f47c0155a00 2023-11-28T13:00:50.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.585+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c006a400 con 0x7f47c0155a00 2023-11-28T13:00:50.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.585+0000 7f47aeffd700 1 -- 172.21.15.153:0/4049589813 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47b0019030 con 0x7f47c0155a00 2023-11-28T13:00:50.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.586+0000 7f47aeffd700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a00682f0 0x7f47a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.586+0000 7f47be59c700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a00682f0 0x7f47a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.586+0000 7f47aeffd700 1 -- 172.21.15.153:0/4049589813 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47b00874b0 con 0x7f47c0155a00 2023-11-28T13:00:50.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.586+0000 7f47be59c700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a00682f0 0x7f47a006a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47c00af310 tx=0x7f47b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.590+0000 7f47aeffd700 1 -- 172.21.15.153:0/4049589813 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47c006a400 con 0x7f47c0155a00 2023-11-28T13:00:50.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.771+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47c006a400 con 0x7f47c0155a00 2023-11-28T13:00:50.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.772+0000 7f47aeffd700 1 -- 172.21.15.153:0/4049589813 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47b0017020 con 0x7f47c0155a00 2023-11-28T13:00:50.772 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:50.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.773+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a00682f0 msgr2=0x7f47a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.773+0000 7f47c4f76700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a00682f0 0x7f47a006a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47c00af310 tx=0x7f47b4009040 comp rx=0 tx=0).stop 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.773+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c0155a00 msgr2=0x7f47c0154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.773+0000 7f47c4f76700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c0155a00 0x7f47c0154d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47a800c220 tx=0x7f47b00047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 shutdown_connections 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c015e0f0 0x7f47c015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a8004850 0x7f47c0153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a00682f0 0x7f47a006a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 --2- 172.21.15.153:0/4049589813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c0155a00 0x7f47c0154d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 >> 172.21.15.153:0/4049589813 conn(0x7f47c00d0000 msgr2=0x7f47c01577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 shutdown_connections 2023-11-28T13:00:50.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.774+0000 7f47c4f76700 1 -- 172.21.15.153:0/4049589813 wait complete. 2023-11-28T13:00:50.787 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:00:50.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.907+0000 7f3b7d3c5700 1 Processor -- start 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.907+0000 7f3b7d3c5700 1 -- start start 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b7d3c5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3b7815c9b0 0x7f3b780b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b7d3c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 0x7f3b780cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b7d3c5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b780d1950 con 0x7f3b78161340 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b7d3c5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b780b5800 con 0x7f3b7815c9b0 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b7d3c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780ce140 con 0x7f3b780cd7c0 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b767fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3b78161340 0x7f3b780d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57530/0 (socket says 172.21.15.153:57530) 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b767fc700 1 -- 172.21.15.153:0/2206317739 learned_addr learned my addr 172.21.15.153:0/2206317739 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:50.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b76ffd700 1 --2- 172.21.15.153:0/2206317739 >> v2:172.21.15.177:3300/0 conn(0x7f3b7815c9b0 0x7f3b780b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b777fe700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 0x7f3b780cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106329302 0 0) 0x7f3b780d1950 con 0x7f3b78161340 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b600035b0 con 0x7f3b78161340 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2949549879 0 0) 0x7f3b600035b0 con 0x7f3b78161340 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 msgr2=0x7f3b780cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 0x7f3b780cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 >> v2:172.21.15.177:3300/0 conn(0x7f3b7815c9b0 msgr2=0x7f3b780b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 --2- 172.21.15.153:0/2206317739 >> v2:172.21.15.177:3300/0 conn(0x7f3b7815c9b0 0x7f3b780b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780d24f0 con 0x7f3b78161340 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f3b6c001670 con 0x7f3b78161340 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 >> v1:172.21.15.153:6789/0 conn(0x7f3b78161340 legacy=0x7f3b780d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 0x7f3b60003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b60003f80 0x7f3b60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b60004d90 0x7f3b60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780b5800 con 0x7f3b60004d90 2023-11-28T13:00:50.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780ce140 con 0x7f3b60003f80 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.908+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6000ba40 con 0x7f3b780cd7c0 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b777fe700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b60004d90 0x7f3b60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b76ffd700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 0x7f3b60003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b767fc700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b60003f80 0x7f3b60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b777fe700 1 -- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 msgr2=0x7f3b60003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b777fe700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780cd7c0 0x7f3b60003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b777fe700 1 -- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b60003f80 msgr2=0x7f3b60004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b777fe700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b60003f80 0x7f3b60004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b777fe700 1 -- 172.21.15.153:0/2206317739 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600035b0 con 0x7f3b60004d90 2023-11-28T13:00:50.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b777fe700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b60004d90 0x7f3b60005200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009fc0 tx=0x7f3b60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9034be8dacc3dcdd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000ebd0 con 0x7f3b60004d90 2023-11-28T13:00:50.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7000f040 con 0x7f3b60004d90 2023-11-28T13:00:50.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.909+0000 7f3b75ffb700 1 -- 172.21.15.153:0/2206317739 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70014480 con 0x7f3b60004d90 2023-11-28T13:00:50.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.910+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/2206317739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b60004d90 msgr2=0x7f3b60005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.910+0000 7f3b7d3c5700 1 --2- 172.21.15.153:0/2206317739 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b60004d90 0x7f3b60005200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009fc0 tx=0x7f3b60005880 comp rx=0 tx=0).stop 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.910+0000 7f3b777fe700 1 -- 172.21.15.153:0/2206317739 reap_dead start 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.910+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/2206317739 shutdown_connections 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.910+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/2206317739 >> 172.21.15.153:0/2206317739 conn(0x7f3b78072600 msgr2=0x7f3b780cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.910+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/2206317739 shutdown_connections 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.910+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/2206317739 wait complete. 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.911+0000 7f3b7d3c5700 1 Processor -- start 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.911+0000 7f3b7d3c5700 1 -- start start 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b780b7920 0x7f3b780c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780b7cf0 0x7f3b781e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b78149f30 0x7f3b7814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780c9300 con 0x7f3b780b7920 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780c95e0 con 0x7f3b78149f30 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780c98c0 con 0x7f3b780b7cf0 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b780b7920 0x7f3b780c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b780b7920 0x7f3b780c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39378/0 (socket says 172.21.15.153:39378) 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 -- 172.21.15.153:0/3286727122 learned_addr learned my addr 172.21.15.153:0/3286727122 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b767fc700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780b7cf0 0x7f3b781e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b777fe700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b78149f30 0x7f3b7814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 -- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780b7cf0 msgr2=0x7f3b781e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780b7cf0 0x7f3b781e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 -- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b78149f30 msgr2=0x7f3b7814a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b78149f30 0x7f3b7814a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 -- 172.21.15.153:0/3286727122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780d1300 con 0x7f3b780b7920 2023-11-28T13:00:50.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b777fe700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b78149f30 0x7f3b7814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:50.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b76ffd700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b780b7920 0x7f3b780c8d00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b68011aa0 tx=0x7f3b68011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b5f7fe700 1 -- 172.21.15.153:0/3286727122 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800f9e0 con 0x7f3b780b7920 2023-11-28T13:00:50.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b5f7fe700 1 -- 172.21.15.153:0/3286727122 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6800fb40 con 0x7f3b780b7920 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b780b5520 con 0x7f3b780b7920 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b5f7fe700 1 -- 172.21.15.153:0/3286727122 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68014440 con 0x7f3b780b7920 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.912+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b780d15e0 con 0x7f3b780b7920 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.913+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b780745c0 con 0x7f3b780b7920 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.914+0000 7f3b5f7fe700 1 -- 172.21.15.153:0/3286727122 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b680145a0 con 0x7f3b780b7920 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.914+0000 7f3b5f7fe700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3b64068320 0x7f3b6406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.914+0000 7f3b5f7fe700 1 -- 172.21.15.153:0/3286727122 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b68086610 con 0x7f3b780b7920 2023-11-28T13:00:50.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.914+0000 7f3b767fc700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3b64068320 0x7f3b6406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.914+0000 7f3b767fc700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3b64068320 0x7f3b6406a7d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0048d0 tx=0x7f3b6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:50.918+0000 7f3b5f7fe700 1 -- 172.21.15.153:0/3286727122 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3b68052de0 con 0x7f3b780b7920 2023-11-28T13:00:51.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.112+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3b780d08a0 con 0x7f3b780b7920 2023-11-28T13:00:51.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.112+0000 7f3b5f7fe700 1 -- 172.21.15.153:0/3286727122 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3b68056400 con 0x7f3b780b7920 2023-11-28T13:00:51.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.114+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3b64068320 msgr2=0x7f3b6406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.114+0000 7f3b7d3c5700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3b64068320 0x7f3b6406a7d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0048d0 tx=0x7f3b6c009040 comp rx=0 tx=0).stop 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.114+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b780b7920 msgr2=0x7f3b780c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.114+0000 7f3b7d3c5700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b780b7920 0x7f3b780c8d00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3b68011aa0 tx=0x7f3b68011e60 comp rx=0 tx=0).stop 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 shutdown_connections 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3b64068320 0x7f3b6406a7d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b780b7920 0x7f3b780c8d00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b78149f30 0x7f3b7814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 --2- 172.21.15.153:0/3286727122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b780b7cf0 0x7f3b781e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 >> 172.21.15.153:0/3286727122 conn(0x7f3b78072600 msgr2=0x7f3b78160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:51.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 shutdown_connections 2023-11-28T13:00:51.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.115+0000 7f3b7d3c5700 1 -- 172.21.15.153:0/3286727122 wait complete. 2023-11-28T13:00:51.117 INFO:teuthology.orchestra.run.smithi153.stderr:listed 0 entries 2023-11-28T13:00:51.128 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:00:51.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.128 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T13:00:51.170 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:00:51.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.171 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:00:51.231 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:51.231 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:51.253 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:51.254 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:51.314 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:51.314 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:51.374 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:51.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.493+0000 7fbe0d34f700 1 Processor -- start 2023-11-28T13:00:51.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.493+0000 7fbe0d34f700 1 -- start start 2023-11-28T13:00:51.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe0d34f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbe080cd7c0 0x7fbe080cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe0d34f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe080d1950 con 0x7fbe08161340 2023-11-28T13:00:51.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe0d34f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe080ce140 con 0x7fbe0815c9b0 2023-11-28T13:00:51.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe0d34f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe080b5800 con 0x7fbe080cd7c0 2023-11-28T13:00:51.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe067fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbe08161340 0x7fbe080d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57552/0 (socket says 172.21.15.153:57552) 2023-11-28T13:00:51.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe067fc700 1 -- 172.21.15.153:0/1235675393 learned_addr learned my addr 172.21.15.153:0/1235675393 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798933262 0 0) 0x7fbe080d1950 con 0x7fbe08161340 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf00035b0 con 0x7fbe08161340 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe077fe700 1 --2- 172.21.15.153:0/1235675393 >> v2:172.21.15.177:3300/0 conn(0x7fbe080cd7c0 0x7fbe080cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 508157024 0 0) 0x7fbdf00035b0 con 0x7fbe08161340 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 >> v2:172.21.15.177:3300/0 conn(0x7fbe080cd7c0 msgr2=0x7fbe080cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 --2- 172.21.15.153:0/1235675393 >> v2:172.21.15.177:3300/0 conn(0x7fbe080cd7c0 0x7fbe080cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 >> v1:172.21.15.177:6789/0 conn(0x7fbe0815c9b0 legacy=0x7fbe080b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080d24f0 con 0x7fbe08161340 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fbdf8001670 con 0x7fbe08161340 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe077fe700 1 --2- 172.21.15.153:0/1235675393 >> v2:172.21.15.177:3300/0 conn(0x7fbe080cd7c0 0x7fbe080cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.494+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 >> v1:172.21.15.153:6789/0 conn(0x7fbe08161340 legacy=0x7fbe080d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf0003a00 0x7fbdf0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0004850 0x7fbdf0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf0005320 0x7fbdf000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080b5800 con 0x7fbdf0005320 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf000c1e0 con 0x7fbdf0003a00 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf000c3a0 con 0x7fbdf0004850 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf0005320 0x7fbdf000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe06ffd700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf0003a00 0x7fbdf0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe067fc700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0004850 0x7fbdf0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 -- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0004850 msgr2=0x7fbdf0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0004850 0x7fbdf0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 -- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf0003a00 msgr2=0x7fbdf0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf0003a00 0x7fbdf0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 -- 172.21.15.153:0/1235675393 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00035b0 con 0x7fbdf0005320 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 -- 172.21.15.153:0/1235675393 reap_dead start 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe077fe700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf0005320 0x7fbdf000ba10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe000073e0 tx=0x7fbdf0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e820078a7d21cdc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00009490 con 0x7fbdf0005320 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00014400 con 0x7fbdf0005320 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.495+0000 7fbe05ffb700 1 -- 172.21.15.153:0/1235675393 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000147b0 con 0x7fbdf0005320 2023-11-28T13:00:51.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.496+0000 7fbe0d34f700 1 -- 172.21.15.153:0/1235675393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf0005320 msgr2=0x7fbdf000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.496+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf0005320 0x7fbdf000ba10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbe000073e0 tx=0x7fbdf0005880 comp rx=0 tx=0).stop 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.496+0000 7fbe0d34f700 1 -- 172.21.15.153:0/1235675393 shutdown_connections 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.496+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/1235675393 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdf0005320 0x7fbdf000ba10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.496+0000 7fbe0d34f700 1 -- 172.21.15.153:0/1235675393 >> 172.21.15.153:0/1235675393 conn(0x7fbe08072600 msgr2=0x7fbe080cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.496+0000 7fbe0d34f700 1 -- 172.21.15.153:0/1235675393 shutdown_connections 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.496+0000 7fbe0d34f700 1 -- 172.21.15.153:0/1235675393 wait complete. 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.497+0000 7fbe0d34f700 1 Processor -- start 2023-11-28T13:00:51.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.497+0000 7fbe0d34f700 1 -- start start 2023-11-28T13:00:51.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe0d34f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0005320 0x7fbe081566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe0d34f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe080ca240 0x7fbe080cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe0d34f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe08152920 0x7fbe080cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe0d34f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080cb660 con 0x7fbe08152920 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe0d34f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0815d190 con 0x7fbe080ca240 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe0d34f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080cb860 con 0x7fbdf0005320 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe08152920 0x7fbe080cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe08152920 0x7fbe080cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39402/0 (socket says 172.21.15.153:39402) 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 -- 172.21.15.153:0/358634173 learned_addr learned my addr 172.21.15.153:0/358634173 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe06ffd700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0005320 0x7fbe081566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe067fc700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe080ca240 0x7fbe080cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 -- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0005320 msgr2=0x7fbe081566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0005320 0x7fbe081566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 -- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe080ca240 msgr2=0x7fbe080cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe080ca240 0x7fbe080cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 -- 172.21.15.153:0/358634173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080d1300 con 0x7fbe08152920 2023-11-28T13:00:51.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe067fc700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe080ca240 0x7fbe080cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:51.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.498+0000 7fbe077fe700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe08152920 0x7fbe080cc5e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbe080d05d0 tx=0x7fbe00004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:51.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.499+0000 7fbdef7fe700 1 -- 172.21.15.153:0/358634173 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00007000 con 0x7fbe08152920 2023-11-28T13:00:51.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.499+0000 7fbdef7fe700 1 -- 172.21.15.153:0/358634173 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00009450 con 0x7fbe08152920 2023-11-28T13:00:51.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.499+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe080b5520 con 0x7fbe08152920 2023-11-28T13:00:51.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.499+0000 7fbdef7fe700 1 -- 172.21.15.153:0/358634173 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00003e10 con 0x7fbe08152920 2023-11-28T13:00:51.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.499+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe080d15e0 con 0x7fbe08152920 2023-11-28T13:00:51.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.500+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe080745c0 con 0x7fbe08152920 2023-11-28T13:00:51.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.500+0000 7fbdef7fe700 1 -- 172.21.15.153:0/358634173 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe00012070 con 0x7fbe08152920 2023-11-28T13:00:51.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.500+0000 7fbdef7fe700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbdf40682f0 0x7fbdf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.500+0000 7fbdef7fe700 1 -- 172.21.15.153:0/358634173 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe000871a0 con 0x7fbe08152920 2023-11-28T13:00:51.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.500+0000 7fbe06ffd700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbdf40682f0 0x7fbdf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.501+0000 7fbe06ffd700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbdf40682f0 0x7fbdf406a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc007fd0 tx=0x7fbdfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:51.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.505+0000 7fbdef7fe700 1 -- 172.21.15.153:0/358634173 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe00053a00 con 0x7fbe08152920 2023-11-28T13:00:51.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.689+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe080d08a0 con 0x7fbe08152920 2023-11-28T13:00:51.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.690+0000 7fbdef7fe700 1 -- 172.21.15.153:0/358634173 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7fbe00057020 con 0x7fbe08152920 2023-11-28T13:00:51.691 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:51.691 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbdf40682f0 msgr2=0x7fbdf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbdf40682f0 0x7fbdf406a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc007fd0 tx=0x7fbdfc009040 comp rx=0 tx=0).stop 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe08152920 msgr2=0x7fbe080cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe08152920 0x7fbe080cc5e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbe080d05d0 tx=0x7fbe00004b40 comp rx=0 tx=0).stop 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 shutdown_connections 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbdf40682f0 0x7fbdf406a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe08152920 0x7fbe080cc5e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf0005320 0x7fbe081566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 --2- 172.21.15.153:0/358634173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe080ca240 0x7fbe080cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 >> 172.21.15.153:0/358634173 conn(0x7fbe08072600 msgr2=0x7fbe080b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:51.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 shutdown_connections 2023-11-28T13:00:51.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.693+0000 7fbe0d34f700 1 -- 172.21.15.153:0/358634173 wait complete. 2023-11-28T13:00:51.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:51.706 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:51.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.828+0000 7f9de2057700 1 Processor -- start 2023-11-28T13:00:51.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.828+0000 7f9de2057700 1 -- start start 2023-11-28T13:00:51.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.829+0000 7f9de2057700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9ddc0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.829+0000 7f9de2057700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc0b0230 con 0x7f9ddc0d01b0 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.829+0000 7f9de2057700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc0b4570 con 0x7f9ddc0d1330 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.829+0000 7f9de2057700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc15cec0 con 0x7f9ddc0b20b0 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.829+0000 7f9ddb7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9ddc0d01b0 0x7f9ddc0d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57570/0 (socket says 172.21.15.153:57570) 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.829+0000 7f9ddb7fe700 1 -- 172.21.15.153:0/3881631520 learned_addr learned my addr 172.21.15.153:0/3881631520 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.829+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81869537 0 0) 0x7f9ddc0b0230 con 0x7f9ddc0d01b0 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddbfff700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9ddc0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc80035b0 con 0x7f9ddc0d01b0 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 880808472 0 0) 0x7f9dc80035b0 con 0x7f9ddc0d01b0 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 msgr2=0x7f9ddc0b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:51.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9ddc0b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 >> v1:172.21.15.177:6789/0 conn(0x7f9ddc0d1330 legacy=0x7f9ddc0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0b4850 con 0x7f9ddc0d01b0 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f9dc4001670 con 0x7f9ddc0d01b0 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddbfff700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9ddc0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 >> v1:172.21.15.153:6789/0 conn(0x7f9ddc0d01b0 legacy=0x7f9ddc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc8003a30 0x7f9dc8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc8004850 0x7f9dc8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9dc8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc15cec0 con 0x7f9dc8003a30 2023-11-28T13:00:51.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc800baf0 con 0x7f9dc8004850 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc800bcb0 con 0x7f9ddc0b20b0 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc8003a30 0x7f9dc8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddaffd700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc8004850 0x7f9dc8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddbfff700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9dc8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddb7fe700 1 -- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 msgr2=0x7f9dc8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9dc8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddb7fe700 1 -- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc8004850 msgr2=0x7f9dc8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dc8004850 0x7f9dc8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddb7fe700 1 -- 172.21.15.153:0/3881631520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80035b0 con 0x7f9dc8003a30 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddbfff700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ddc0b20b0 0x7f9dc8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc8003a30 0x7f9dc8003e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400c240 tx=0x7f9dc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa7c4127fd486f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:51.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4007c10 con 0x7f9dc8003a30 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc400ce00 con 0x7f9dc8003a30 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.830+0000 7f9dda7fc700 1 -- 172.21.15.153:0/3881631520 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4005ba0 con 0x7f9dc8003a30 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.831+0000 7f9de2057700 1 -- 172.21.15.153:0/3881631520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc8003a30 msgr2=0x7f9dc8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.831+0000 7f9de2057700 1 --2- 172.21.15.153:0/3881631520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9dc8003a30 0x7f9dc8003e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400c240 tx=0x7f9dc8005880 comp rx=0 tx=0).stop 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.831+0000 7f9ddbfff700 1 -- 172.21.15.153:0/3881631520 reap_dead start 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.831+0000 7f9de2057700 1 -- 172.21.15.153:0/3881631520 shutdown_connections 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.831+0000 7f9de2057700 1 -- 172.21.15.153:0/3881631520 >> 172.21.15.153:0/3881631520 conn(0x7f9ddc09c3f0 msgr2=0x7f9ddc0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.831+0000 7f9de2057700 1 -- 172.21.15.153:0/3881631520 shutdown_connections 2023-11-28T13:00:51.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.831+0000 7f9de2057700 1 -- 172.21.15.153:0/3881631520 wait complete. 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.832+0000 7f9de2057700 1 Processor -- start 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.832+0000 7f9de2057700 1 -- start start 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9de2057700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc8003a30 0x7f9ddc15f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9de2057700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc15fb00 0x7f9ddc15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9de2057700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ddc0b7e90 0x7f9ddc0b8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9de2057700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc0cc330 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9de2057700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc0b8870 con 0x7f9ddc15fb00 2023-11-28T13:00:51.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9de2057700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc0cb460 con 0x7f9dc8003a30 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9ddbfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ddc0b7e90 0x7f9ddc0b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9ddbfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ddc0b7e90 0x7f9ddc0b8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39422/0 (socket says 172.21.15.153:39422) 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9ddbfff700 1 -- 172.21.15.153:0/725897 learned_addr learned my addr 172.21.15.153:0/725897 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc8003a30 0x7f9ddc15f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.833+0000 7f9ddaffd700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc15fb00 0x7f9ddc15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddbfff700 1 -- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc8003a30 msgr2=0x7f9ddc15f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddbfff700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc8003a30 0x7f9ddc15f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddbfff700 1 -- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc15fb00 msgr2=0x7f9ddc15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddbfff700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc15fb00 0x7f9ddc15b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddbfff700 1 -- 172.21.15.153:0/725897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc0afec0 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddaffd700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc15fb00 0x7f9ddc15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:51.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc8003a30 0x7f9ddc15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9ddbfff700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ddc0b7e90 0x7f9ddc0b8330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0b2530 tx=0x7f9dd4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9dd3fff700 1 -- 172.21.15.153:0/725897 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4007c00 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9dd3fff700 1 -- 172.21.15.153:0/725897 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd400f050 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9dd3fff700 1 -- 172.21.15.153:0/725897 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4014410 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ddc0afbe0 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.834+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc0b2600 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.835+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc0cc470 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.835+0000 7f9dd3fff700 1 -- 172.21.15.153:0/725897 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9dd4014570 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.835+0000 7f9dd3fff700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9dbc0682f0 0x7f9dbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.836+0000 7f9dd3fff700 1 -- 172.21.15.153:0/725897 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9dd4086040 con 0x7f9ddc0b7e90 2023-11-28T13:00:51.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.836+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9dbc0682f0 0x7f9dbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:51.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.836+0000 7f9ddb7fe700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9dbc0682f0 0x7f9dbc06a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4015040 tx=0x7f9dc4007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:51.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:51.840+0000 7f9dd3fff700 1 -- 172.21.15.153:0/725897 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ddc0cc470 con 0x7f9ddc0b7e90 2023-11-28T13:00:52.021 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.020+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ddc0cc470 con 0x7f9ddc0b7e90 2023-11-28T13:00:52.021 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.021+0000 7f9dd3fff700 1 -- 172.21.15.153:0/725897 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ddc0cc470 con 0x7f9ddc0b7e90 2023-11-28T13:00:52.021 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:52.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.022+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9dbc0682f0 msgr2=0x7f9dbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.022+0000 7f9de2057700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9dbc0682f0 0x7f9dbc06a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4015040 tx=0x7f9dc4007000 comp rx=0 tx=0).stop 2023-11-28T13:00:52.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.022+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ddc0b7e90 msgr2=0x7f9ddc0b8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.022+0000 7f9de2057700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ddc0b7e90 0x7f9ddc0b8330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0b2530 tx=0x7f9dd4004740 comp rx=0 tx=0).stop 2023-11-28T13:00:52.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 shutdown_connections 2023-11-28T13:00:52.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9dbc0682f0 0x7f9dbc06a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ddc0b7e90 0x7f9ddc0b8330 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dc8003a30 0x7f9ddc15f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 --2- 172.21.15.153:0/725897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc15fb00 0x7f9ddc15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 >> 172.21.15.153:0/725897 conn(0x7f9ddc09c3f0 msgr2=0x7f9ddc06ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:52.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 shutdown_connections 2023-11-28T13:00:52.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.023+0000 7f9de2057700 1 -- 172.21.15.153:0/725897 wait complete. 2023-11-28T13:00:52.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:52.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.162+0000 7f74f2075700 1 Processor -- start 2023-11-28T13:00:52.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.162+0000 7f74f2075700 1 -- start start 2023-11-28T13:00:52.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.162+0000 7f74f2075700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f74ec06dee0 0x7f74ec06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.162+0000 7f74f2075700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f74ec160540 0x7f74ec160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.162+0000 7f74f2075700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec15d060 con 0x7f74ec071040 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.162+0000 7f74f2075700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec15d340 con 0x7f74ec06dee0 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.162+0000 7f74f2075700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec15d620 con 0x7f74ec160540 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f74ec06dee0 0x7f74ec06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f74ec06dee0 0x7f74ec06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39428/0 (socket says 172.21.15.153:39428) 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 -- 172.21.15.153:0/141683749 learned_addr learned my addr 172.21.15.153:0/141683749 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74ebfff700 1 --2- 172.21.15.153:0/141683749 >> v2:172.21.15.177:3300/0 conn(0x7f74ec160540 0x7f74ec160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 -- 172.21.15.153:0/141683749 >> v2:172.21.15.177:3300/0 conn(0x7f74ec160540 msgr2=0x7f74ec160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:52.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 --2- 172.21.15.153:0/141683749 >> v2:172.21.15.177:3300/0 conn(0x7f74ec160540 0x7f74ec160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 -- 172.21.15.153:0/141683749 >> v1:172.21.15.177:6789/0 conn(0x7f74ec071040 legacy=0x7f74ec071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 -- 172.21.15.153:0/141683749 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec15d900 con 0x7f74ec06dee0 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74eaffd700 1 --2- 172.21.15.153:0/141683749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec06dee0 0x7f74ec06e330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0c3a10 tx=0x7f74e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ecf3e8854fc3e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74ea7fc700 1 -- 172.21.15.153:0/141683749 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0004230 con 0x7f74ec06dee0 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74ea7fc700 1 -- 172.21.15.153:0/141683749 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e0004390 con 0x7f74ec06dee0 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.163+0000 7f74ea7fc700 1 -- 172.21.15.153:0/141683749 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e00036a0 con 0x7f74ec06dee0 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 -- 172.21.15.153:0/141683749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec06dee0 msgr2=0x7f74ec06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 --2- 172.21.15.153:0/141683749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec06dee0 0x7f74ec06e330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0c3a10 tx=0x7f74e0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 -- 172.21.15.153:0/141683749 shutdown_connections 2023-11-28T13:00:52.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 --2- 172.21.15.153:0/141683749 >> v2:172.21.15.177:3300/0 conn(0x7f74ec160540 0x7f74ec160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 --2- 172.21.15.153:0/141683749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec06dee0 0x7f74ec06e330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 -- 172.21.15.153:0/141683749 >> 172.21.15.153:0/141683749 conn(0x7f74ec0d0000 msgr2=0x7f74ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 -- 172.21.15.153:0/141683749 shutdown_connections 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.164+0000 7f74f2075700 1 -- 172.21.15.153:0/141683749 wait complete. 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.165+0000 7f74f2075700 1 Processor -- start 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.165+0000 7f74f2075700 1 -- start start 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74f2075700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74ec06dee0 0x7f74ec0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74f2075700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74ec071040 0x7f74ec1f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74f2075700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec160540 0x7f74ec0b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74f2075700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0c5e10 con 0x7f74ec160540 2023-11-28T13:00:52.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74f2075700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0b2e30 con 0x7f74ec071040 2023-11-28T13:00:52.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74f2075700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0b3110 con 0x7f74ec06dee0 2023-11-28T13:00:52.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec160540 0x7f74ec0b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec160540 0x7f74ec0b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39436/0 (socket says 172.21.15.153:39436) 2023-11-28T13:00:52.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 -- 172.21.15.153:0/26837794 learned_addr learned my addr 172.21.15.153:0/26837794 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:52.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74eaffd700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74ec071040 0x7f74ec1f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74eb7fe700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74ec06dee0 0x7f74ec0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 -- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74ec06dee0 msgr2=0x7f74ec0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74ec06dee0 0x7f74ec0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 -- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74ec071040 msgr2=0x7f74ec1f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74ec071040 0x7f74ec1f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 -- 172.21.15.153:0/26837794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0009550 con 0x7f74ec160540 2023-11-28T13:00:52.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74eb7fe700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74ec06dee0 0x7f74ec0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:52.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.166+0000 7f74ebfff700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec160540 0x7f74ec0b35a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007e90 tx=0x7f74e400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.167+0000 7f74d3fff700 1 -- 172.21.15.153:0/26837794 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e400ede0 con 0x7f74ec160540 2023-11-28T13:00:52.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.167+0000 7f74d3fff700 1 -- 172.21.15.153:0/26837794 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4014420 con 0x7f74ec160540 2023-11-28T13:00:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.167+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec15ccd0 con 0x7f74ec160540 2023-11-28T13:00:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.167+0000 7f74d3fff700 1 -- 172.21.15.153:0/26837794 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e40147d0 con 0x7f74ec160540 2023-11-28T13:00:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.167+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec06e870 con 0x7f74ec160540 2023-11-28T13:00:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.168+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec06a270 con 0x7f74ec160540 2023-11-28T13:00:52.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.168+0000 7f74d3fff700 1 -- 172.21.15.153:0/26837794 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f74e4012050 con 0x7f74ec160540 2023-11-28T13:00:52.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.168+0000 7f74d3fff700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f74d80682f0 0x7f74d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.168+0000 7f74d3fff700 1 -- 172.21.15.153:0/26837794 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74e4086250 con 0x7f74ec160540 2023-11-28T13:00:52.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.168+0000 7f74eb7fe700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f74d80682f0 0x7f74d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.169+0000 7f74eb7fe700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f74d80682f0 0x7f74d806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0cf080 tx=0x7f74dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.173+0000 7f74d3fff700 1 -- 172.21.15.153:0/26837794 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f74e4052ab0 con 0x7f74ec160540 2023-11-28T13:00:52.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.354+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f74ec0b3250 con 0x7f74ec160540 2023-11-28T13:00:52.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.355+0000 7f74d3fff700 1 -- 172.21.15.153:0/26837794 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f74e40560d0 con 0x7f74ec160540 2023-11-28T13:00:52.356 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f74d80682f0 msgr2=0x7f74d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f74d80682f0 0x7f74d806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0cf080 tx=0x7f74dc008040 comp rx=0 tx=0).stop 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec160540 msgr2=0x7f74ec0b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec160540 0x7f74ec0b35a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007e90 tx=0x7f74e400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 shutdown_connections 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f74d80682f0 0x7f74d806a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ec160540 0x7f74ec0b35a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74ec06dee0 0x7f74ec0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 --2- 172.21.15.153:0/26837794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74ec071040 0x7f74ec1f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 >> 172.21.15.153:0/26837794 conn(0x7f74ec0d0000 msgr2=0x7f74ec06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:52.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 shutdown_connections 2023-11-28T13:00:52.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.358+0000 7f74f2075700 1 -- 172.21.15.153:0/26837794 wait complete. 2023-11-28T13:00:52.371 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:52.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.493+0000 7fc773f86700 1 Processor -- start 2023-11-28T13:00:52.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.493+0000 7fc773f86700 1 -- start start 2023-11-28T13:00:52.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc773f86700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc76c071040 0x7fc76c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc773f86700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c071950 con 0x7fc76c06dee0 2023-11-28T13:00:52.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc773f86700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c15d060 con 0x7fc76c160540 2023-11-28T13:00:52.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc773f86700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c15d340 con 0x7fc76c071040 2023-11-28T13:00:52.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc771d22700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc76c071040 0x7fc76c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc771d22700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc76c071040 0x7fc76c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39440/0 (socket says 172.21.15.153:39440) 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc771521700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc76c06dee0 0x7fc76c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57604/0 (socket says 172.21.15.153:57604) 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc771d22700 1 -- 172.21.15.153:0/3671830809 learned_addr learned my addr 172.21.15.153:0/3671830809 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc771d22700 1 -- 172.21.15.153:0/3671830809 >> v1:172.21.15.177:6789/0 conn(0x7fc76c160540 legacy=0x7fc76c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc771d22700 1 -- 172.21.15.153:0/3671830809 >> v1:172.21.15.153:6789/0 conn(0x7fc76c06dee0 legacy=0x7fc76c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc770d20700 1 -- 172.21.15.153:0/3671830809 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321150507 0 0) 0x7fc76c071950 con 0x7fc76c06dee0 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.494+0000 7fc771d22700 1 -- 172.21.15.153:0/3671830809 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c15d620 con 0x7fc76c071040 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc771d22700 1 --2- 172.21.15.153:0/3671830809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c071410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0098f0 tx=0x7fc75c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848f328b98445257 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc770d20700 1 -- 172.21.15.153:0/3671830809 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c004230 con 0x7fc76c071040 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc770d20700 1 -- 172.21.15.153:0/3671830809 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c004390 con 0x7fc76c071040 2023-11-28T13:00:52.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc770d20700 1 -- 172.21.15.153:0/3671830809 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0036a0 con 0x7fc76c071040 2023-11-28T13:00:52.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc773f86700 1 -- 172.21.15.153:0/3671830809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 msgr2=0x7fc76c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc773f86700 1 --2- 172.21.15.153:0/3671830809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c071410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0098f0 tx=0x7fc75c009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:52.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc773f86700 1 -- 172.21.15.153:0/3671830809 shutdown_connections 2023-11-28T13:00:52.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc773f86700 1 --2- 172.21.15.153:0/3671830809 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c071410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc773f86700 1 -- 172.21.15.153:0/3671830809 >> 172.21.15.153:0/3671830809 conn(0x7fc76c0d0000 msgr2=0x7fc76c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:52.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc773f86700 1 -- 172.21.15.153:0/3671830809 shutdown_connections 2023-11-28T13:00:52.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.495+0000 7fc773f86700 1 -- 172.21.15.153:0/3671830809 wait complete. 2023-11-28T13:00:52.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.497+0000 7fc773f86700 1 Processor -- start 2023-11-28T13:00:52.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.497+0000 7fc773f86700 1 -- start start 2023-11-28T13:00:52.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc773f86700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc76c06dee0 0x7fc76c0c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc773f86700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc773f86700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c160540 0x7fc76c14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc773f86700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c52b0 con 0x7fc76c071040 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc773f86700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c5590 con 0x7fc76c06dee0 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc773f86700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c5870 con 0x7fc76c160540 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c0cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39448/0 (socket says 172.21.15.153:39448) 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 -- 172.21.15.153:0/2557346483 learned_addr learned my addr 172.21.15.153:0/2557346483 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771d22700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc76c06dee0 0x7fc76c0c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc772523700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c160540 0x7fc76c14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 -- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c160540 msgr2=0x7fc76c14a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c160540 0x7fc76c14a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 -- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc76c06dee0 msgr2=0x7fc76c0c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc76c06dee0 0x7fc76c0c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 -- 172.21.15.153:0/2557346483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c009550 con 0x7fc76c071040 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc772523700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c160540 0x7fc76c14a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771d22700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc76c06dee0 0x7fc76c0c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc771521700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c0cf170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c550 tx=0x7fc76800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc7627fc700 1 -- 172.21.15.153:0/2557346483 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768004c80 con 0x7fc76c071040 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc7627fc700 1 -- 172.21.15.153:0/2557346483 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc768004e00 con 0x7fc76c071040 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.498+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c15ccd0 con 0x7fc76c071040 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.499+0000 7fc7627fc700 1 -- 172.21.15.153:0/2557346483 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768007310 con 0x7fc76c071040 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.499+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c15d900 con 0x7fc76c071040 2023-11-28T13:00:52.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.500+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c06a270 con 0x7fc76c071040 2023-11-28T13:00:52.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.500+0000 7fc7627fc700 1 -- 172.21.15.153:0/2557346483 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc76801a020 con 0x7fc76c071040 2023-11-28T13:00:52.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.500+0000 7fc7627fc700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7500682f0 0x7fc75006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.500+0000 7fc7627fc700 1 -- 172.21.15.153:0/2557346483 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc768085850 con 0x7fc76c071040 2023-11-28T13:00:52.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.500+0000 7fc771d22700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7500682f0 0x7fc75006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.501+0000 7fc771d22700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7500682f0 0x7fc75006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009130 tx=0x7fc75c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.505+0000 7fc7627fc700 1 -- 172.21.15.153:0/2557346483 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc768052160 con 0x7fc76c071040 2023-11-28T13:00:52.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.691+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc76c0c53f0 con 0x7fc76c071040 2023-11-28T13:00:52.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.691+0000 7fc7627fc700 1 -- 172.21.15.153:0/2557346483 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc76c0c53f0 con 0x7fc76c071040 2023-11-28T13:00:52.692 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:52.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7500682f0 msgr2=0x7fc75006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7500682f0 0x7fc75006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009130 tx=0x7fc75c008030 comp rx=0 tx=0).stop 2023-11-28T13:00:52.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 msgr2=0x7fc76c0cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c0cf170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc76800c550 tx=0x7fc76800c910 comp rx=0 tx=0).stop 2023-11-28T13:00:52.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 shutdown_connections 2023-11-28T13:00:52.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76c160540 0x7fc76c14a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc76c06dee0 0x7fc76c0c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7500682f0 0x7fc75006a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 --2- 172.21.15.153:0/2557346483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc76c071040 0x7fc76c0cf170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 >> 172.21.15.153:0/2557346483 conn(0x7fc76c0d0000 msgr2=0x7fc76c15e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:52.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 shutdown_connections 2023-11-28T13:00:52.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.693+0000 7fc773f86700 1 -- 172.21.15.153:0/2557346483 wait complete. 2023-11-28T13:00:52.706 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:52.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.828+0000 7fb596eb1700 1 Processor -- start 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.828+0000 7fb596eb1700 1 -- start start 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb596eb1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb590160740 0x7fb5900cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb596eb1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb59006b060 0x7fb59006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb596eb1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5900cc030 con 0x7fb590160b10 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb596eb1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5900ca720 con 0x7fb590160740 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb596eb1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900c9e10 con 0x7fb59006b060 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb594c4d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb590160740 0x7fb5900cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb594c4d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb590160740 0x7fb5900cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39454/0 (socket says 172.21.15.153:39454) 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb594c4d700 1 -- 172.21.15.153:0/1852043800 learned_addr learned my addr 172.21.15.153:0/1852043800 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:52.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.829+0000 7fb59544e700 1 --2- 172.21.15.153:0/1852043800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb59006b060 0x7fb59006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb594c4d700 1 -- 172.21.15.153:0/1852043800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb59006b060 msgr2=0x7fb59006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb594c4d700 1 --2- 172.21.15.153:0/1852043800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb59006b060 0x7fb59006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb594c4d700 1 -- 172.21.15.153:0/1852043800 >> v1:172.21.15.177:6789/0 conn(0x7fb590160b10 legacy=0x7fb5900cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb594c4d700 1 -- 172.21.15.153:0/1852043800 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59006efc0 con 0x7fb590160740 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb594c4d700 1 --2- 172.21.15.153:0/1852043800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590160740 0x7fb5900cbaf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb5840098f0 tx=0x7fb584009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd37119b9ff15e7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb58f7fe700 1 -- 172.21.15.153:0/1852043800 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58400b8a0 con 0x7fb590160740 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb58f7fe700 1 -- 172.21.15.153:0/1852043800 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58400ba00 con 0x7fb590160740 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb58f7fe700 1 -- 172.21.15.153:0/1852043800 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58400bd90 con 0x7fb590160740 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb596eb1700 1 -- 172.21.15.153:0/1852043800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590160740 msgr2=0x7fb5900cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1852043800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590160740 0x7fb5900cbaf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb5840098f0 tx=0x7fb584009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb596eb1700 1 -- 172.21.15.153:0/1852043800 shutdown_connections 2023-11-28T13:00:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1852043800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb59006b060 0x7fb59006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1852043800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590160740 0x7fb5900cbaf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.830+0000 7fb596eb1700 1 -- 172.21.15.153:0/1852043800 >> 172.21.15.153:0/1852043800 conn(0x7fb5900b12b0 msgr2=0x7fb5900b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.831+0000 7fb596eb1700 1 -- 172.21.15.153:0/1852043800 shutdown_connections 2023-11-28T13:00:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.831+0000 7fb596eb1700 1 -- 172.21.15.153:0/1852043800 wait complete. 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.832+0000 7fb596eb1700 1 Processor -- start 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.832+0000 7fb596eb1700 1 -- start start 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb59006b060 0x7fb590155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb590160740 0x7fb59015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb590160b10 0x7fb59015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590156170 con 0x7fb59006b060 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59015a4f0 con 0x7fb590160b10 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59015a7d0 con 0x7fb590160740 2023-11-28T13:00:52.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb59006b060 0x7fb590155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb59006b060 0x7fb590155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39458/0 (socket says 172.21.15.153:39458) 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 -- 172.21.15.153:0/1934512315 learned_addr learned my addr 172.21.15.153:0/1934512315 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb59544e700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb590160b10 0x7fb59015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb58ffff700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb590160740 0x7fb59015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 -- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb590160740 msgr2=0x7fb59015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb590160740 0x7fb59015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 -- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb590160b10 msgr2=0x7fb59015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb590160b10 0x7fb59015f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 -- 172.21.15.153:0/1934512315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584009550 con 0x7fb59006b060 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb594c4d700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb59006b060 0x7fb590155c30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb584009c40 tx=0x7fb584004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb58d7fa700 1 -- 172.21.15.153:0/1934512315 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840044d0 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb58d7fa700 1 -- 172.21.15.153:0/1934512315 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58401f070 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5900ca3b0 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb58d7fa700 1 -- 172.21.15.153:0/1934512315 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840038e0 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.833+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5900ca0d0 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.834+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb590160f60 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.835+0000 7fb58d7fa700 1 -- 172.21.15.153:0/1934512315 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb58400f050 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.835+0000 7fb58d7fa700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb5740682f0 0x7fb57406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.835+0000 7fb58d7fa700 1 -- 172.21.15.153:0/1934512315 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb584021070 con 0x7fb59006b060 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.835+0000 7fb58ffff700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb5740682f0 0x7fb57406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:52.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.835+0000 7fb58ffff700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb5740682f0 0x7fb57406a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb59015a2a0 tx=0x7fb57c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:52.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:52.839+0000 7fb58d7fa700 1 -- 172.21.15.153:0/1934512315 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb584053cf0 con 0x7fb59006b060 2023-11-28T13:00:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.022+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb590072670 con 0x7fb59006b060 2023-11-28T13:00:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.022+0000 7fb58d7fa700 1 -- 172.21.15.153:0/1934512315 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb584015020 con 0x7fb59006b060 2023-11-28T13:00:53.023 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:53.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb5740682f0 msgr2=0x7fb57406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb5740682f0 0x7fb57406a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb59015a2a0 tx=0x7fb57c009040 comp rx=0 tx=0).stop 2023-11-28T13:00:53.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb59006b060 msgr2=0x7fb590155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb59006b060 0x7fb590155c30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb584009c40 tx=0x7fb584004750 comp rx=0 tx=0).stop 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 shutdown_connections 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb5740682f0 0x7fb57406a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb59006b060 0x7fb590155c30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb590160b10 0x7fb59015f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 --2- 172.21.15.153:0/1934512315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb590160740 0x7fb59015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 >> 172.21.15.153:0/1934512315 conn(0x7fb5900b12b0 msgr2=0x7fb5900af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 shutdown_connections 2023-11-28T13:00:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.024+0000 7fb596eb1700 1 -- 172.21.15.153:0/1934512315 wait complete. 2023-11-28T13:00:53.038 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:00:53.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.159+0000 7f72e41ad700 1 Processor -- start 2023-11-28T13:00:53.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.159+0000 7f72e41ad700 1 -- start start 2023-11-28T13:00:53.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e41ad700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f72dc071040 0x7f72dc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e41ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc160540 0x7f72dc1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e41ad700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc071950 con 0x7f72dc06dee0 2023-11-28T13:00:53.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e41ad700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc15d060 con 0x7f72dc071040 2023-11-28T13:00:53.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e41ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc15d340 con 0x7f72dc160540 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e1748700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f72dc06dee0 0x7f72dc06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57630/0 (socket says 172.21.15.153:57630) 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e1748700 1 -- 172.21.15.153:0/3055876679 learned_addr learned my addr 172.21.15.153:0/3055876679 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e1f49700 1 --2- 172.21.15.153:0/3055876679 >> v2:172.21.15.177:3300/0 conn(0x7f72dc071040 0x7f72dc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e274a700 1 --2- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc160540 0x7f72dc1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e0f47700 1 -- 172.21.15.153:0/3055876679 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586130383 0 0) 0x7f72dc071950 con 0x7f72dc06dee0 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e0f47700 1 -- 172.21.15.153:0/3055876679 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c80035b0 con 0x7f72dc06dee0 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e1f49700 1 -- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc160540 msgr2=0x7f72dc1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e1f49700 1 --2- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc160540 0x7f72dc1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e1f49700 1 -- 172.21.15.153:0/3055876679 >> v1:172.21.15.153:6789/0 conn(0x7f72dc06dee0 legacy=0x7f72dc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e1f49700 1 -- 172.21.15.153:0/3055876679 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc15d620 con 0x7f72dc071040 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.160+0000 7f72e274a700 1 --2- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc160540 0x7f72dc1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e1f49700 1 --2- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc071040 0x7f72dc071410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009840 tx=0x7f72d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ea916a423788338 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e0f47700 1 -- 172.21.15.153:0/3055876679 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8011830 con 0x7f72dc071040 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e0f47700 1 -- 172.21.15.153:0/3055876679 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d8011990 con 0x7f72dc071040 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e0f47700 1 -- 172.21.15.153:0/3055876679 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8011d00 con 0x7f72dc071040 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 -- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc071040 msgr2=0x7f72dc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 --2- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc071040 0x7f72dc071410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009840 tx=0x7f72d8009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 -- 172.21.15.153:0/3055876679 shutdown_connections 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 --2- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc160540 0x7f72dc1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 --2- 172.21.15.153:0/3055876679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc071040 0x7f72dc071410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 -- 172.21.15.153:0/3055876679 >> 172.21.15.153:0/3055876679 conn(0x7f72dc0d0000 msgr2=0x7f72dc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 -- 172.21.15.153:0/3055876679 shutdown_connections 2023-11-28T13:00:53.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.161+0000 7f72e41ad700 1 -- 172.21.15.153:0/3055876679 wait complete. 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 Processor -- start 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 -- start start 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc06dee0 0x7f72dc0c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc071040 0x7f72dc0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc160540 0x7f72dc1e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1e5c10 con 0x7f72dc160540 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0c5960 con 0x7f72dc06dee0 2023-11-28T13:00:53.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e41ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc159210 con 0x7f72dc071040 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.163+0000 7f72e274a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc160540 0x7f72dc1e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc160540 0x7f72dc1e56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39476/0 (socket says 172.21.15.153:39476) 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 -- 172.21.15.153:0/1195760265 learned_addr learned my addr 172.21.15.153:0/1195760265 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e1f49700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc06dee0 0x7f72dc0c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e1748700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc071040 0x7f72dc0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 -- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc071040 msgr2=0x7f72dc0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc071040 0x7f72dc0c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 -- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc06dee0 msgr2=0x7f72dc0c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc06dee0 0x7f72dc0c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 -- 172.21.15.153:0/1195760265 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d4009a70 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e274a700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc160540 0x7f72dc1e56d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f72d400efc0 tx=0x7f72d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72d27fc700 1 -- 172.21.15.153:0/1195760265 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d400ca90 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72d27fc700 1 -- 172.21.15.153:0/1195760265 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d400f040 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72d27fc700 1 -- 172.21.15.153:0/1195760265 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d400ce30 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d80094a0 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.164+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc15ccd0 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.165+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc15d900 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.166+0000 7f72d27fc700 1 -- 172.21.15.153:0/1195760265 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f72d4014480 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.166+0000 7f72d27fc700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f72c00682f0 0x7f72c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.166+0000 7f72d27fc700 1 -- 172.21.15.153:0/1195760265 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72d40864b0 con 0x7f72dc160540 2023-11-28T13:00:53.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.166+0000 7f72e1f49700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f72c00682f0 0x7f72c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.166+0000 7f72e1f49700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f72c00682f0 0x7f72c006a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.170+0000 7f72d27fc700 1 -- 172.21.15.153:0/1195760265 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f72d4052c30 con 0x7f72dc160540 2023-11-28T13:00:53.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.370+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f72dc06a270 con 0x7f72dc160540 2023-11-28T13:00:53.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.370+0000 7f72d27fc700 1 -- 172.21.15.153:0/1195760265 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f72d4056250 con 0x7f72dc160540 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f72c00682f0 msgr2=0x7f72c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f72c00682f0 0x7f72c006a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d801c040 comp rx=0 tx=0).stop 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc160540 msgr2=0x7f72dc1e56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc160540 0x7f72dc1e56d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f72d400efc0 tx=0x7f72d400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 shutdown_connections 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f72c00682f0 0x7f72c006a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc160540 0x7f72dc1e56d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72dc06dee0 0x7f72dc0c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 --2- 172.21.15.153:0/1195760265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72dc071040 0x7f72dc0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 >> 172.21.15.153:0/1195760265 conn(0x7f72dc0d0000 msgr2=0x7f72dc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 shutdown_connections 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.373+0000 7f72e41ad700 1 -- 172.21.15.153:0/1195760265 wait complete. 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi153.stderr:listed 0 entries 2023-11-28T13:00:53.386 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T13:00:53.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.387 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T13:00:53.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T13:00:53.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.435 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T13:00:53.554 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:53.555 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:53.574 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:53.574 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:53.633 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:00:53.633 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:53.691 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.851+0000 7f7eba069700 1 Processor -- start 2023-11-28T13:00:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.851+0000 7f7eba069700 1 -- start start 2023-11-28T13:00:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.851+0000 7f7eba069700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7eb406e500 0x7f7eb415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eba069700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7eb40ae9d0 0x7f7eb406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eba069700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb406ee10 con 0x7f7eb415e540 2023-11-28T13:00:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eba069700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb415bf10 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eba069700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb40aee40 con 0x7f7eb406e500 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7eb40ae9d0 0x7f7eb406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7eb40ae9d0 0x7f7eb406e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39482/0 (socket says 172.21.15.153:39482) 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/3234402503 learned_addr learned my addr 172.21.15.153:0/3234402503 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb37fe700 1 --2- 172.21.15.153:0/3234402503 >> v2:172.21.15.177:3300/0 conn(0x7f7eb406e500 0x7f7eb415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/3234402503 >> v2:172.21.15.177:3300/0 conn(0x7f7eb406e500 msgr2=0x7f7eb415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 --2- 172.21.15.153:0/3234402503 >> v2:172.21.15.177:3300/0 conn(0x7f7eb406e500 0x7f7eb415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/3234402503 >> v1:172.21.15.177:6789/0 conn(0x7f7eb415e540 legacy=0x7f7eb415e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/3234402503 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb406c930 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb2ffd700 1 --2- 172.21.15.153:0/3234402503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb406e8d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40703a0 tx=0x7f7ea8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5591d1e8da9246b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb27fc700 1 -- 172.21.15.153:0/3234402503 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800b870 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb27fc700 1 -- 172.21.15.153:0/3234402503 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea800b9d0 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.852+0000 7f7eb27fc700 1 -- 172.21.15.153:0/3234402503 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800bd60 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 -- 172.21.15.153:0/3234402503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 msgr2=0x7f7eb406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 --2- 172.21.15.153:0/3234402503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb406e8d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7eb40703a0 tx=0x7f7ea8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 -- 172.21.15.153:0/3234402503 shutdown_connections 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 --2- 172.21.15.153:0/3234402503 >> v2:172.21.15.177:3300/0 conn(0x7f7eb406e500 0x7f7eb415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 --2- 172.21.15.153:0/3234402503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb406e8d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 -- 172.21.15.153:0/3234402503 >> 172.21.15.153:0/3234402503 conn(0x7f7eb40b12b0 msgr2=0x7f7eb40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 -- 172.21.15.153:0/3234402503 shutdown_connections 2023-11-28T13:00:53.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.853+0000 7f7eba069700 1 -- 172.21.15.153:0/3234402503 wait complete. 2023-11-28T13:00:53.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.854+0000 7f7eba069700 1 Processor -- start 2023-11-28T13:00:53.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.854+0000 7f7eba069700 1 -- start start 2023-11-28T13:00:53.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eba069700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb406e500 0x7f7eb4153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eba069700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb41e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eba069700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7eb415e540 0x7f7eb40c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eba069700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb41e5950 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eba069700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40c7f70 con 0x7f7eb406e500 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eba069700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40c8250 con 0x7f7eb415e540 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb41e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb41e4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39486/0 (socket says 172.21.15.153:39486) 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/2310860730 learned_addr learned my addr 172.21.15.153:0/2310860730 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb37fe700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb406e500 0x7f7eb4153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb3fff700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7eb415e540 0x7f7eb40c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7eb415e540 msgr2=0x7f7eb40c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7eb415e540 0x7f7eb40c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb406e500 msgr2=0x7f7eb4153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb406e500 0x7f7eb4153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb2ffd700 1 -- 172.21.15.153:0/2310860730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea8009550 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.855+0000 7f7eb37fe700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb406e500 0x7f7eb4153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.856+0000 7f7eb2ffd700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb41e4e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8000c00 tx=0x7f7ea8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.856+0000 7f7e9bfff700 1 -- 172.21.15.153:0/2310860730 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8004540 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.856+0000 7f7e9bfff700 1 -- 172.21.15.153:0/2310860730 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea80036a0 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.856+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb40664c0 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.856+0000 7f7e9bfff700 1 -- 172.21.15.153:0/2310860730 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8003a30 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.856+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb406cc10 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.857+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb406cef0 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.857+0000 7f7e9bfff700 1 -- 172.21.15.153:0/2310860730 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7ea800f050 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.858+0000 7f7e9bfff700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea00682f0 0x7f7ea006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.858+0000 7f7eb37fe700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea00682f0 0x7f7ea006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.858+0000 7f7e9bfff700 1 -- 172.21.15.153:0/2310860730 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ea8087530 con 0x7f7eb40ae9d0 2023-11-28T13:00:53.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.858+0000 7f7eb37fe700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea00682f0 0x7f7ea006a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4007a50 tx=0x7f7ea4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:53.862+0000 7f7e9bfff700 1 -- 172.21.15.153:0/2310860730 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7ea8053d90 con 0x7f7eb40ae9d0 2023-11-28T13:00:54.045 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.045+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7eb4065fc0 con 0x7f7eb40ae9d0 2023-11-28T13:00:54.047 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.046+0000 7f7e9bfff700 1 -- 172.21.15.153:0/2310860730 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7f7ea80573b0 con 0x7f7eb40ae9d0 2023-11-28T13:00:54.048 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:54.048 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:54.050 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea00682f0 msgr2=0x7f7ea006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.050 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea00682f0 0x7f7ea006a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4007a50 tx=0x7f7ea4008040 comp rx=0 tx=0).stop 2023-11-28T13:00:54.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 msgr2=0x7f7eb41e4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb41e4e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8000c00 tx=0x7f7ea8004750 comp rx=0 tx=0).stop 2023-11-28T13:00:54.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 shutdown_connections 2023-11-28T13:00:54.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7eb415e540 0x7f7eb40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eb406e500 0x7f7eb4153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7ea00682f0 0x7f7ea006a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 --2- 172.21.15.153:0/2310860730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7eb40ae9d0 0x7f7eb41e4e90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 >> 172.21.15.153:0/2310860730 conn(0x7f7eb40b12b0 msgr2=0x7f7eb40ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:54.053 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 shutdown_connections 2023-11-28T13:00:54.054 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.048+0000 7f7eba069700 1 -- 172.21.15.153:0/2310860730 wait complete. 2023-11-28T13:00:54.062 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:54.063 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:54.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.187+0000 7fe76676e700 1 Processor -- start 2023-11-28T13:00:54.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.187+0000 7fe76676e700 1 -- start start 2023-11-28T13:00:54.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe76676e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe760071040 0x7fe760071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe76676e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe760160540 0x7fe7601609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe76676e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe760071950 con 0x7fe76006dee0 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe76676e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76015d060 con 0x7fe760071040 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe76676e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76015d340 con 0x7fe760160540 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe764d0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe760160540 0x7fe7601609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe75ffff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe760071040 0x7fe760071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe764d0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe760160540 0x7fe7601609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:57106/0 (socket says 172.21.15.153:57106) 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe764d0b700 1 -- 172.21.15.153:0/48833591 learned_addr learned my addr 172.21.15.153:0/48833591 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.188+0000 7fe75f7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe76006dee0 0x7fe76006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:34270/0 (socket says 172.21.15.153:34270) 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75ffff700 1 -- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe760160540 msgr2=0x7fe7601609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75ffff700 1 --2- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe760160540 0x7fe7601609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75ffff700 1 -- 172.21.15.153:0/48833591 >> v1:172.21.15.177:6789/0 conn(0x7fe76006dee0 legacy=0x7fe76006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75ffff700 1 -- 172.21.15.153:0/48833591 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76015d620 con 0x7fe760071040 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe764d0b700 1 --2- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe760160540 0x7fe7601609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75ffff700 1 --2- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 0x7fe760071410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe754009840 tx=0x7fe754009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe4e93f1b8ac37a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75effd700 1 -- 172.21.15.153:0/48833591 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011830 con 0x7fe760071040 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75effd700 1 -- 172.21.15.153:0/48833591 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754011990 con 0x7fe760071040 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.189+0000 7fe75effd700 1 -- 172.21.15.153:0/48833591 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011d20 con 0x7fe760071040 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 -- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 msgr2=0x7fe760071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 --2- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 0x7fe760071410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe754009840 tx=0x7fe754009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 -- 172.21.15.153:0/48833591 shutdown_connections 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 --2- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe760160540 0x7fe7601609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 --2- 172.21.15.153:0/48833591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 0x7fe760071410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 -- 172.21.15.153:0/48833591 >> 172.21.15.153:0/48833591 conn(0x7fe7600d0000 msgr2=0x7fe7600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:54.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 -- 172.21.15.153:0/48833591 shutdown_connections 2023-11-28T13:00:54.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.190+0000 7fe76676e700 1 -- 172.21.15.153:0/48833591 wait complete. 2023-11-28T13:00:54.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.191+0000 7fe76676e700 1 Processor -- start 2023-11-28T13:00:54.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.191+0000 7fe76676e700 1 -- start start 2023-11-28T13:00:54.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe76676e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe76006dee0 0x7fe760157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe76676e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 0x7fe7600b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe76676e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe760160540 0x7fe7600c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe76676e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600b38e0 con 0x7fe760160540 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe76676e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600b3b80 con 0x7fe760071040 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe76676e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600b3e60 con 0x7fe76006dee0 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe760160540 0x7fe7600c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe760160540 0x7fe7600c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39502/0 (socket says 172.21.15.153:39502) 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 -- 172.21.15.153:0/1514925717 learned_addr learned my addr 172.21.15.153:0/1514925717 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe75f7fe700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 0x7fe7600b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe75ffff700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe76006dee0 0x7fe760157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 -- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe76006dee0 msgr2=0x7fe760157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe76006dee0 0x7fe760157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 -- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 msgr2=0x7fe7600b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 0x7fe7600b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 -- 172.21.15.153:0/1514925717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758009b30 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe764d0b700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe760160540 0x7fe7600c5b10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe75800efc0 tx=0x7fe75800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.192+0000 7fe75cff9700 1 -- 172.21.15.153:0/1514925717 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe758018070 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.193+0000 7fe75cff9700 1 -- 172.21.15.153:0/1514925717 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75800f040 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.193+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7540094a0 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.193+0000 7fe75cff9700 1 -- 172.21.15.153:0/1514925717 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe758003c20 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.193+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76015ccd0 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.194+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76015d900 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.194+0000 7fe75cff9700 1 -- 172.21.15.153:0/1514925717 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe75800c880 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.194+0000 7fe75cff9700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe74c0682f0 0x7fe74c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.194+0000 7fe75cff9700 1 -- 172.21.15.153:0/1514925717 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe758085f50 con 0x7fe760160540 2023-11-28T13:00:54.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.194+0000 7fe75ffff700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe74c0682f0 0x7fe74c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.195+0000 7fe75ffff700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe74c0682f0 0x7fe74c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe754000c00 tx=0x7fe75401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.198+0000 7fe75cff9700 1 -- 172.21.15.153:0/1514925717 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe7580527b0 con 0x7fe760160540 2023-11-28T13:00:54.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.380+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe76006a270 con 0x7fe760160540 2023-11-28T13:00:54.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.380+0000 7fe75cff9700 1 -- 172.21.15.153:0/1514925717 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe758055dd0 con 0x7fe760160540 2023-11-28T13:00:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe74c0682f0 msgr2=0x7fe74c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe74c0682f0 0x7fe74c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe754000c00 tx=0x7fe75401c040 comp rx=0 tx=0).stop 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe760160540 msgr2=0x7fe7600c5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe760160540 0x7fe7600c5b10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe75800efc0 tx=0x7fe75800c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 shutdown_connections 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe74c0682f0 0x7fe74c06a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe760160540 0x7fe7600c5b10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe76006dee0 0x7fe760157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 --2- 172.21.15.153:0/1514925717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe760071040 0x7fe7600b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 >> 172.21.15.153:0/1514925717 conn(0x7fe7600d0000 msgr2=0x7fe76015e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 shutdown_connections 2023-11-28T13:00:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.383+0000 7fe76676e700 1 -- 172.21.15.153:0/1514925717 wait complete. 2023-11-28T13:00:54.396 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:54.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.519+0000 7f98fc45d700 1 Processor -- start 2023-11-28T13:00:54.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.519+0000 7f98fc45d700 1 -- start start 2023-11-28T13:00:54.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.519+0000 7f98fc45d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f98f40d01b0 0x7f98f40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.519+0000 7f98fc45d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f98f40b20b0 0x7f98f40d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fc45d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40b29b0 0x7f98f40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fc45d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f415cec0 con 0x7f98f40d01b0 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fc45d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40b0230 con 0x7f98f40b20b0 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fc45d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40b4570 con 0x7f98f40b29b0 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f98f40d01b0 0x7f98f40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f98f40d01b0 0x7f98f40d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39506/0 (socket says 172.21.15.153:39506) 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 -- 172.21.15.153:0/4142533707 learned_addr learned my addr 172.21.15.153:0/4142533707 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98f99f8700 1 --2- 172.21.15.153:0/4142533707 >> v2:172.21.15.177:3300/0 conn(0x7f98f40b20b0 0x7f98f40d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa9fa700 1 --2- 172.21.15.153:0/4142533707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40b29b0 0x7f98f40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 -- 172.21.15.153:0/4142533707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40b29b0 msgr2=0x7f98f40b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:54.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 --2- 172.21.15.153:0/4142533707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40b29b0 0x7f98f40b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 -- 172.21.15.153:0/4142533707 >> v2:172.21.15.177:3300/0 conn(0x7f98f40b20b0 msgr2=0x7f98f40d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 --2- 172.21.15.153:0/4142533707 >> v2:172.21.15.177:3300/0 conn(0x7f98f40b20b0 0x7f98f40d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 -- 172.21.15.153:0/4142533707 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40b4850 con 0x7f98f40d01b0 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98fa1f9700 1 --2- 172.21.15.153:0/4142533707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40d01b0 0x7f98f40d0c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f98f40ca0b0 tx=0x7f98f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6a233e7cb7ebcf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98f91f7700 1 -- 172.21.15.153:0/4142533707 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f000b870 con 0x7f98f40d01b0 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98f91f7700 1 -- 172.21.15.153:0/4142533707 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f000b9d0 con 0x7f98f40d01b0 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.520+0000 7f98f91f7700 1 -- 172.21.15.153:0/4142533707 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f000bd60 con 0x7f98f40d01b0 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 -- 172.21.15.153:0/4142533707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40d01b0 msgr2=0x7f98f40d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 --2- 172.21.15.153:0/4142533707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40d01b0 0x7f98f40d0c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f98f40ca0b0 tx=0x7f98f0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 -- 172.21.15.153:0/4142533707 shutdown_connections 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 --2- 172.21.15.153:0/4142533707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40b29b0 0x7f98f40b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 --2- 172.21.15.153:0/4142533707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40d01b0 0x7f98f40d0c20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 --2- 172.21.15.153:0/4142533707 >> v2:172.21.15.177:3300/0 conn(0x7f98f40b20b0 0x7f98f40d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 -- 172.21.15.153:0/4142533707 >> 172.21.15.153:0/4142533707 conn(0x7f98f409c3f0 msgr2=0x7f98f40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:54.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 -- 172.21.15.153:0/4142533707 shutdown_connections 2023-11-28T13:00:54.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.521+0000 7f98fc45d700 1 -- 172.21.15.153:0/4142533707 wait complete. 2023-11-28T13:00:54.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.522+0000 7f98fc45d700 1 Processor -- start 2023-11-28T13:00:54.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.522+0000 7f98fc45d700 1 -- start start 2023-11-28T13:00:54.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fc45d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40b20b0 0x7f98f4149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fc45d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98f40b29b0 0x7f98f40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fc45d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40d01b0 0x7f98f41f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fc45d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41f6890 con 0x7f98f40b20b0 2023-11-28T13:00:54.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fc45d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4149e50 con 0x7f98f40b29b0 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fc45d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41f5c10 con 0x7f98f40d01b0 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40b20b0 0x7f98f4149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40b20b0 0x7f98f4149910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39510/0 (socket says 172.21.15.153:39510) 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 -- 172.21.15.153:0/101877782 learned_addr learned my addr 172.21.15.153:0/101877782 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98f99f8700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98f40b29b0 0x7f98f40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa9fa700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40d01b0 0x7f98f41f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 -- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40d01b0 msgr2=0x7f98f41f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40d01b0 0x7f98f41f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 -- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98f40b29b0 msgr2=0x7f98f40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98f40b29b0 0x7f98f40cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.523+0000 7f98fa1f9700 1 -- 172.21.15.153:0/101877782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0009550 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.524+0000 7f98fa1f9700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40b20b0 0x7f98f4149910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.524+0000 7f98eaffd700 1 -- 172.21.15.153:0/101877782 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0004520 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.524+0000 7f98eaffd700 1 -- 172.21.15.153:0/101877782 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f001e070 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.524+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f40afec0 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.524+0000 7f98eaffd700 1 -- 172.21.15.153:0/101877782 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00038c0 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.524+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f40afbe0 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.525+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f40b2600 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.525+0000 7f98eaffd700 1 -- 172.21.15.153:0/101877782 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f98f000f050 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.525+0000 7f98eaffd700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f98d80682f0 0x7f98d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.525+0000 7f98eaffd700 1 -- 172.21.15.153:0/101877782 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f98f0020070 con 0x7f98f40b20b0 2023-11-28T13:00:54.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.526+0000 7f98f99f8700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f98d80682f0 0x7f98d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.526+0000 7f98f99f8700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f98d80682f0 0x7f98d806a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98f4158dc0 tx=0x7f98e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.530+0000 7f98eaffd700 1 -- 172.21.15.153:0/101877782 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f98f0053a40 con 0x7f98f40b20b0 2023-11-28T13:00:54.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.714+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98f41f69d0 con 0x7f98f40b20b0 2023-11-28T13:00:54.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.715+0000 7f98eaffd700 1 -- 172.21.15.153:0/101877782 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98f0057060 con 0x7f98f40b20b0 2023-11-28T13:00:54.716 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:54.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f98d80682f0 msgr2=0x7f98d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f98d80682f0 0x7f98d806a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98f4158dc0 tx=0x7f98e4008040 comp rx=0 tx=0).stop 2023-11-28T13:00:54.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40b20b0 msgr2=0x7f98f4149910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40b20b0 0x7f98f4149910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f98f0000c00 tx=0x7f98f0004750 comp rx=0 tx=0).stop 2023-11-28T13:00:54.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 shutdown_connections 2023-11-28T13:00:54.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f98d80682f0 0x7f98d806a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f98f40b20b0 0x7f98f4149910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98f40d01b0 0x7f98f41f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 --2- 172.21.15.153:0/101877782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98f40b29b0 0x7f98f40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 >> 172.21.15.153:0/101877782 conn(0x7f98f409c3f0 msgr2=0x7f98f40cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:54.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 shutdown_connections 2023-11-28T13:00:54.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.717+0000 7f98fc45d700 1 -- 172.21.15.153:0/101877782 wait complete. 2023-11-28T13:00:54.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:54.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.852+0000 7f6419b86700 1 Processor -- start 2023-11-28T13:00:54.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.852+0000 7f6419b86700 1 -- start start 2023-11-28T13:00:54.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.853+0000 7f6419b86700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6414159860 0x7f64140c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.853+0000 7f6419b86700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64140ccc00 con 0x7f64140b0d80 2023-11-28T13:00:54.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.853+0000 7f6419b86700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64140ccec0 con 0x7f6414159490 2023-11-28T13:00:54.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.853+0000 7f6419b86700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64140cd1a0 con 0x7f6414159860 2023-11-28T13:00:54.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.853+0000 7f6413fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f64140b0d80 0x7f64140c9a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57684/0 (socket says 172.21.15.153:57684) 2023-11-28T13:00:54.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.853+0000 7f6413fff700 1 -- 172.21.15.153:0/305575004 learned_addr learned my addr 172.21.15.153:0/305575004 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:54.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6412ffd700 1 --2- 172.21.15.153:0/305575004 >> v2:172.21.15.177:3300/0 conn(0x7f6414159860 0x7f64140c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038452634 0 0) 0x7f64140ccc00 con 0x7f64140b0d80 2023-11-28T13:00:54.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63fc0035b0 con 0x7f64140b0d80 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1489909830 0 0) 0x7f63fc0035b0 con 0x7f64140b0d80 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 >> v2:172.21.15.177:3300/0 conn(0x7f6414159860 msgr2=0x7f64140c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 --2- 172.21.15.153:0/305575004 >> v2:172.21.15.177:3300/0 conn(0x7f6414159860 0x7f64140c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 >> v1:172.21.15.177:6789/0 conn(0x7f6414159490 legacy=0x7f64140b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140cd480 con 0x7f64140b0d80 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f640c0019f0 con 0x7f64140b0d80 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 >> v1:172.21.15.153:6789/0 conn(0x7f64140b0d80 legacy=0x7f64140c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63fc003a00 0x7f63fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc004850 0x7f63fc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63fc005200 0x7f63fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64140cd1a0 con 0x7f63fc005200 2023-11-28T13:00:54.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc00c080 con 0x7f63fc004850 2023-11-28T13:00:54.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc00c240 con 0x7f63fc003a00 2023-11-28T13:00:54.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6413fff700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63fc005200 0x7f63fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6412ffd700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc004850 0x7f63fc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f64137fe700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63fc003a00 0x7f63fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6413fff700 1 -- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63fc003a00 msgr2=0x7f63fc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6413fff700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63fc003a00 0x7f63fc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6413fff700 1 -- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc004850 msgr2=0x7f63fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6413fff700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc004850 0x7f63fc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6413fff700 1 -- 172.21.15.153:0/305575004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc0035b0 con 0x7f63fc005200 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.854+0000 7f6413fff700 1 -- 172.21.15.153:0/305575004 reap_dead start 2023-11-28T13:00:54.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6413fff700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63fc005200 0x7f63fc00ba10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f640c00a830 tx=0x7f63fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d11bfbcdc6838d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00a2f0 con 0x7f63fc005200 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c012600 con 0x7f63fc005200 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f64127fc700 1 -- 172.21.15.153:0/305575004 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c0129b0 con 0x7f63fc005200 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6419b86700 1 -- 172.21.15.153:0/305575004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63fc005200 msgr2=0x7f63fc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6419b86700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63fc005200 0x7f63fc00ba10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f640c00a830 tx=0x7f63fc005880 comp rx=0 tx=0).stop 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6419b86700 1 -- 172.21.15.153:0/305575004 shutdown_connections 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6419b86700 1 --2- 172.21.15.153:0/305575004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63fc005200 0x7f63fc00ba10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6419b86700 1 -- 172.21.15.153:0/305575004 >> 172.21.15.153:0/305575004 conn(0x7f64140cb670 msgr2=0x7f64140cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6419b86700 1 -- 172.21.15.153:0/305575004 shutdown_connections 2023-11-28T13:00:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.855+0000 7f6419b86700 1 -- 172.21.15.153:0/305575004 wait complete. 2023-11-28T13:00:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.856+0000 7f6419b86700 1 Processor -- start 2023-11-28T13:00:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.857+0000 7f6419b86700 1 -- start start 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.857+0000 7f6419b86700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc005200 0x7f641406ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.857+0000 7f6419b86700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64140c49c0 0x7f64140c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.857+0000 7f6419b86700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64140c6630 0x7f64140c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.857+0000 7f6419b86700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141f0960 con 0x7f64140c6630 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.857+0000 7f6419b86700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64141efd90 con 0x7f63fc005200 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.857+0000 7f6419b86700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64141f0030 con 0x7f64140c49c0 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64140c6630 0x7f64140c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64140c6630 0x7f64140c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39532/0 (socket says 172.21.15.153:39532) 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f64137fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc005200 0x7f641406ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 -- 172.21.15.153:0/1232960467 learned_addr learned my addr 172.21.15.153:0/1232960467 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6412ffd700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64140c49c0 0x7f64140c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 -- 172.21.15.153:0/1232960467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64140c49c0 msgr2=0x7f64140c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64140c49c0 0x7f64140c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 -- 172.21.15.153:0/1232960467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc005200 msgr2=0x7f641406ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc005200 0x7f641406ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 -- 172.21.15.153:0/1232960467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140cdc80 con 0x7f64140c6630 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6413fff700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64140c6630 0x7f64140c6a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f64140cad10 tx=0x7f640c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f63fbfff700 1 -- 172.21.15.153:0/1232960467 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c001250 con 0x7f64140c6630 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f63fbfff700 1 -- 172.21.15.153:0/1232960467 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c01d420 con 0x7f64140c6630 2023-11-28T13:00:54.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64140cd760 con 0x7f64140c6630 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f63fbfff700 1 -- 172.21.15.153:0/1232960467 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c01d7d0 con 0x7f64140c6630 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.858+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64140cde10 con 0x7f64140c6630 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.859+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64140745c0 con 0x7f64140c6630 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.860+0000 7f63fbfff700 1 -- 172.21.15.153:0/1232960467 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f640c028030 con 0x7f64140c6630 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.860+0000 7f63fbfff700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f64000682f0 0x7f640006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.860+0000 7f63fbfff700 1 -- 172.21.15.153:0/1232960467 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f640c0873c0 con 0x7f64140c6630 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.860+0000 7f64137fe700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f64000682f0 0x7f640006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.861+0000 7f64137fe700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f64000682f0 0x7f640006a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6408007fd0 tx=0x7f6408009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:54.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:54.864+0000 7f63fbfff700 1 -- 172.21.15.153:0/1232960467 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f640c053c20 con 0x7f64140c6630 2023-11-28T13:00:55.045 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.045+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64140ce640 con 0x7f64140c6630 2023-11-28T13:00:55.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.045+0000 7f63fbfff700 1 -- 172.21.15.153:0/1232960467 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f640c057240 con 0x7f64140c6630 2023-11-28T13:00:55.046 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:55.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f64000682f0 msgr2=0x7f640006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f64000682f0 0x7f640006a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6408007fd0 tx=0x7f6408009040 comp rx=0 tx=0).stop 2023-11-28T13:00:55.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64140c6630 msgr2=0x7f64140c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64140c6630 0x7f64140c6a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f64140cad10 tx=0x7f640c001d80 comp rx=0 tx=0).stop 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 shutdown_connections 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f64000682f0 0x7f640006a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64140c6630 0x7f64140c6a60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63fc005200 0x7f641406ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 --2- 172.21.15.153:0/1232960467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64140c49c0 0x7f64140c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 >> 172.21.15.153:0/1232960467 conn(0x7f64140cb670 msgr2=0x7f6414159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 shutdown_connections 2023-11-28T13:00:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.047+0000 7f6419b86700 1 -- 172.21.15.153:0/1232960467 wait complete. 2023-11-28T13:00:55.060 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:55.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.182+0000 7f714e92d700 1 Processor -- start 2023-11-28T13:00:55.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.182+0000 7f714e92d700 1 -- start start 2023-11-28T13:00:55.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714e92d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f714806dee0 0x7f714806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714e92d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148160540 0x7f71481609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714e92d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f714815d060 con 0x7f7148071040 2023-11-28T13:00:55.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714e92d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714815d340 con 0x7f714806dee0 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714e92d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714815d620 con 0x7f7148160540 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f71477fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f714806dee0 0x7f714806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714ceca700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148160540 0x7f71481609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f71477fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f714806dee0 0x7f714806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:49004/0 (socket says 172.21.15.153:49004) 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f71477fe700 1 -- 172.21.15.153:0/759871194 learned_addr learned my addr 172.21.15.153:0/759871194 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f7147fff700 1 --1- 172.21.15.153:0/759871194 >> v1:172.21.15.177:6789/0 conn(0x7f7148071040 0x7f7148071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:34314/0 (socket says 172.21.15.153:34314) 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714ceca700 1 -- 172.21.15.153:0/759871194 >> v2:172.21.15.177:3300/0 conn(0x7f714806dee0 msgr2=0x7f714806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714ceca700 1 --2- 172.21.15.153:0/759871194 >> v2:172.21.15.177:3300/0 conn(0x7f714806dee0 0x7f714806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714ceca700 1 -- 172.21.15.153:0/759871194 >> v1:172.21.15.177:6789/0 conn(0x7f7148071040 legacy=0x7f7148071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f714ceca700 1 -- 172.21.15.153:0/759871194 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714815d900 con 0x7f7148160540 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.183+0000 7f71477fe700 1 --2- 172.21.15.153:0/759871194 >> v2:172.21.15.177:3300/0 conn(0x7f714806dee0 0x7f714806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714ceca700 1 --2- 172.21.15.153:0/759871194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148160540 0x7f71481609c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7140009d90 tx=0x7f714000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4cf849454d6c506 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f7146ffd700 1 -- 172.21.15.153:0/759871194 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714000e070 con 0x7f7148160540 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f7146ffd700 1 -- 172.21.15.153:0/759871194 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714000bb20 con 0x7f7148160540 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f7146ffd700 1 -- 172.21.15.153:0/759871194 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140012430 con 0x7f7148160540 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 -- 172.21.15.153:0/759871194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148160540 msgr2=0x7f71481609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 --2- 172.21.15.153:0/759871194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148160540 0x7f71481609c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7140009d90 tx=0x7f714000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 -- 172.21.15.153:0/759871194 shutdown_connections 2023-11-28T13:00:55.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 --2- 172.21.15.153:0/759871194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148160540 0x7f71481609c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 --2- 172.21.15.153:0/759871194 >> v2:172.21.15.177:3300/0 conn(0x7f714806dee0 0x7f714806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 -- 172.21.15.153:0/759871194 >> 172.21.15.153:0/759871194 conn(0x7f71480d0000 msgr2=0x7f71480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:55.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 -- 172.21.15.153:0/759871194 shutdown_connections 2023-11-28T13:00:55.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.184+0000 7f714e92d700 1 -- 172.21.15.153:0/759871194 wait complete. 2023-11-28T13:00:55.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.185+0000 7f714e92d700 1 Processor -- start 2023-11-28T13:00:55.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.186+0000 7f714e92d700 1 -- start start 2023-11-28T13:00:55.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.186+0000 7f714e92d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f714806dee0 0x7f714814dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.186+0000 7f714e92d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148071040 0x7f71480b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.186+0000 7f714e92d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7148160540 0x7f71480b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.186+0000 7f714e92d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b2ed0 con 0x7f7148160540 2023-11-28T13:00:55.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.186+0000 7f714e92d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b3f00 con 0x7f714806dee0 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.186+0000 7f714e92d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714814e4e0 con 0x7f7148071040 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7148160540 0x7f71480b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7148160540 0x7f71480b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39546/0 (socket says 172.21.15.153:39546) 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 -- 172.21.15.153:0/3989775423 learned_addr learned my addr 172.21.15.153:0/3989775423 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f7147fff700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f714806dee0 0x7f714814dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f71477fe700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148071040 0x7f71480b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 -- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148071040 msgr2=0x7f71480b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148071040 0x7f71480b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 -- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f714806dee0 msgr2=0x7f714814dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f714806dee0 0x7f714814dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 -- 172.21.15.153:0/3989775423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7138009550 con 0x7f7148160540 2023-11-28T13:00:55.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714ceca700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7148160540 0x7f71480b4ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7140014040 tx=0x7f7140004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:55.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f7144ff9700 1 -- 172.21.15.153:0/3989775423 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140003ea0 con 0x7f7148160540 2023-11-28T13:00:55.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f7144ff9700 1 -- 172.21.15.153:0/3989775423 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7140020070 con 0x7f7148160540 2023-11-28T13:00:55.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f7144ff9700 1 -- 172.21.15.153:0/3989775423 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71400098f0 con 0x7f7148160540 2023-11-28T13:00:55.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7140009a70 con 0x7f7148160540 2023-11-28T13:00:55.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.187+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714815ccd0 con 0x7f7148160540 2023-11-28T13:00:55.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.188+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714806e870 con 0x7f7148160540 2023-11-28T13:00:55.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.189+0000 7f7144ff9700 1 -- 172.21.15.153:0/3989775423 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f714002e070 con 0x7f7148160540 2023-11-28T13:00:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.189+0000 7f7144ff9700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f71340682f0 0x7f713406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.189+0000 7f7144ff9700 1 -- 172.21.15.153:0/3989775423 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7140022070 con 0x7f7148160540 2023-11-28T13:00:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.189+0000 7f7147fff700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f71340682f0 0x7f713406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.189+0000 7f7147fff700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f71340682f0 0x7f713406a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f713c000f60 tx=0x7f713c007ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.193+0000 7f7144ff9700 1 -- 172.21.15.153:0/3989775423 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f71400540b0 con 0x7f7148160540 2023-11-28T13:00:55.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.373+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f714806a270 con 0x7f7148160540 2023-11-28T13:00:55.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.373+0000 7f7144ff9700 1 -- 172.21.15.153:0/3989775423 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f714002c0a0 con 0x7f7148160540 2023-11-28T13:00:55.374 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:55.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f71340682f0 msgr2=0x7f713406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f71340682f0 0x7f713406a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f713c000f60 tx=0x7f713c007ed0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7148160540 msgr2=0x7f71480b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7148160540 0x7f71480b4ee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7140014040 tx=0x7f7140004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 shutdown_connections 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f71340682f0 0x7f713406a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7148160540 0x7f71480b4ee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f714806dee0 0x7f714814dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 --2- 172.21.15.153:0/3989775423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7148071040 0x7f71480b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 >> 172.21.15.153:0/3989775423 conn(0x7f71480d0000 msgr2=0x7f71480cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:55.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.375+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 shutdown_connections 2023-11-28T13:00:55.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.376+0000 7f714e92d700 1 -- 172.21.15.153:0/3989775423 wait complete. 2023-11-28T13:00:55.389 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:00:55.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.512+0000 7fe3f4f97700 1 Processor -- start 2023-11-28T13:00:55.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.512+0000 7fe3f4f97700 1 -- start start 2023-11-28T13:00:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3f4f97700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe3f015cef0 0x7fe3f00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3f4f97700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f006ba90 0x7fe3f00ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3f4f97700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f00cc5f0 con 0x7fe3f00cbdf0 2023-11-28T13:00:55.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3f4f97700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f015d470 con 0x7fe3f015cef0 2023-11-28T13:00:55.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3f4f97700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f015d750 con 0x7fe3f006ba90 2023-11-28T13:00:55.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ee59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe3f015cef0 0x7fe3f00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3edd9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe3f00cbdf0 0x7fe3f006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57714/0 (socket says 172.21.15.153:57714) 2023-11-28T13:00:55.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3edd9b700 1 -- 172.21.15.153:0/931308745 learned_addr learned my addr 172.21.15.153:0/931308745 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:55.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3eed9d700 1 --2- 172.21.15.153:0/931308745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f006ba90 0x7fe3f00ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ee59c700 1 -- 172.21.15.153:0/931308745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f006ba90 msgr2=0x7fe3f00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:55.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ee59c700 1 --2- 172.21.15.153:0/931308745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f006ba90 0x7fe3f00ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ee59c700 1 -- 172.21.15.153:0/931308745 >> v1:172.21.15.153:6789/0 conn(0x7fe3f00cbdf0 legacy=0x7fe3f006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ee59c700 1 -- 172.21.15.153:0/931308745 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f015da30 con 0x7fe3f015cef0 2023-11-28T13:00:55.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ed59a700 1 -- 172.21.15.153:0/931308745 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996795808 0 0) 0x7fe3f00cc5f0 con 0x7fe3f00cbdf0 2023-11-28T13:00:55.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ee59c700 1 --2- 172.21.15.153:0/931308745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f015cef0 0x7fe3f00af400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009870 tx=0x7fe3e0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57705e7846d4589f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:55.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.513+0000 7fe3ed59a700 1 -- 172.21.15.153:0/931308745 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0011830 con 0x7fe3f015cef0 2023-11-28T13:00:55.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3ed59a700 1 -- 172.21.15.153:0/931308745 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e0011990 con 0x7fe3f015cef0 2023-11-28T13:00:55.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3ed59a700 1 -- 172.21.15.153:0/931308745 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0011d20 con 0x7fe3f015cef0 2023-11-28T13:00:55.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 -- 172.21.15.153:0/931308745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f015cef0 msgr2=0x7fe3f00af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/931308745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f015cef0 0x7fe3f00af400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009870 tx=0x7fe3e0009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:55.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 -- 172.21.15.153:0/931308745 shutdown_connections 2023-11-28T13:00:55.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/931308745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f006ba90 0x7fe3f00ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/931308745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f015cef0 0x7fe3f00af400 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 -- 172.21.15.153:0/931308745 >> 172.21.15.153:0/931308745 conn(0x7fe3f00d0000 msgr2=0x7fe3f00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:55.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 -- 172.21.15.153:0/931308745 shutdown_connections 2023-11-28T13:00:55.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.514+0000 7fe3f4f97700 1 -- 172.21.15.153:0/931308745 wait complete. 2023-11-28T13:00:55.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.515+0000 7fe3f4f97700 1 Processor -- start 2023-11-28T13:00:55.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3f4f97700 1 -- start start 2023-11-28T13:00:55.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3f4f97700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f006ba90 0x7fe3f01f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3f4f97700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3f00cbdf0 0x7fe3f014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3f4f97700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f015cef0 0x7fe3f00cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3f4f97700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f00c7290 con 0x7fe3f006ba90 2023-11-28T13:00:55.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3f4f97700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01f4510 con 0x7fe3f00cbdf0 2023-11-28T13:00:55.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3f4f97700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01508b0 con 0x7fe3f015cef0 2023-11-28T13:00:55.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3ee59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f006ba90 0x7fe3f01f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3ee59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f006ba90 0x7fe3f01f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39556/0 (socket says 172.21.15.153:39556) 2023-11-28T13:00:55.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.516+0000 7fe3ee59c700 1 -- 172.21.15.153:0/3903194115 learned_addr learned my addr 172.21.15.153:0/3903194115 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:55.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3eed9d700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f015cef0 0x7fe3f00cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3edd9b700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3f00cbdf0 0x7fe3f014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3ee59c700 1 -- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f015cef0 msgr2=0x7fe3f00cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3ee59c700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f015cef0 0x7fe3f00cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3ee59c700 1 -- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3f00cbdf0 msgr2=0x7fe3f014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3ee59c700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3f00cbdf0 0x7fe3f014f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3ee59c700 1 -- 172.21.15.153:0/3903194115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00094a0 con 0x7fe3f006ba90 2023-11-28T13:00:55.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3eed9d700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f015cef0 0x7fe3f00cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:55.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3edd9b700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3f00cbdf0 0x7fe3f014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:55.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3ee59c700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f006ba90 0x7fe3f01f35e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00090b0 tx=0x7fe3e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:55.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3deffd700 1 -- 172.21.15.153:0/3903194115 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00036a0 con 0x7fe3f006ba90 2023-11-28T13:00:55.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3deffd700 1 -- 172.21.15.153:0/3903194115 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e001e070 con 0x7fe3f006ba90 2023-11-28T13:00:55.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f015dff0 con 0x7fe3f006ba90 2023-11-28T13:00:55.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3deffd700 1 -- 172.21.15.153:0/3903194115 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0003a80 con 0x7fe3f006ba90 2023-11-28T13:00:55.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.517+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f015dd10 con 0x7fe3f006ba90 2023-11-28T13:00:55.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.518+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f00cab90 con 0x7fe3f006ba90 2023-11-28T13:00:55.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.518+0000 7fe3deffd700 1 -- 172.21.15.153:0/3903194115 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe3e000f050 con 0x7fe3f006ba90 2023-11-28T13:00:55.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.519+0000 7fe3deffd700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe3d00682f0 0x7fe3d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:55.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.519+0000 7fe3edd9b700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe3d00682f0 0x7fe3d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:55.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.519+0000 7fe3deffd700 1 -- 172.21.15.153:0/3903194115 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3e0086b40 con 0x7fe3f006ba90 2023-11-28T13:00:55.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.519+0000 7fe3edd9b700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe3d00682f0 0x7fe3d006a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00b59d0 tx=0x7fe3e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:55.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.523+0000 7fe3deffd700 1 -- 172.21.15.153:0/3903194115 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe3e00533a0 con 0x7fe3f006ba90 2023-11-28T13:00:55.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.725+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe3f00d0e60 con 0x7fe3f006ba90 2023-11-28T13:00:55.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.726+0000 7fe3deffd700 1 -- 172.21.15.153:0/3903194115 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe3e0015080 con 0x7fe3f006ba90 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe3d00682f0 msgr2=0x7fe3d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe3d00682f0 0x7fe3d006a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00b59d0 tx=0x7fe3e4009040 comp rx=0 tx=0).stop 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f006ba90 msgr2=0x7fe3f01f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f006ba90 0x7fe3f01f35e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00090b0 tx=0x7fe3e0004750 comp rx=0 tx=0).stop 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 shutdown_connections 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe3d00682f0 0x7fe3d006a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3f006ba90 0x7fe3f01f35e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3f015cef0 0x7fe3f00cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 --2- 172.21.15.153:0/3903194115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3f00cbdf0 0x7fe3f014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:55.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 >> 172.21.15.153:0/3903194115 conn(0x7fe3f00d0000 msgr2=0x7fe3f006d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:55.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 shutdown_connections 2023-11-28T13:00:55.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:55.728+0000 7fe3f4f97700 1 -- 172.21.15.153:0/3903194115 wait complete. 2023-11-28T13:00:55.730 INFO:teuthology.orchestra.run.smithi153.stderr:listed 0 entries 2023-11-28T13:00:55.741 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T13:00:55.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.741 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T13:00:55.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:00:55.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.780 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T13:00:55.836 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:55.837 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:55.858 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:55.858 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:55.919 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:00:55.919 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:55.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:56.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.098+0000 7feb1458f700 1 Processor -- start 2023-11-28T13:00:56.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.098+0000 7feb1458f700 1 -- start start 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb1458f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb1458f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7feb0c06b060 0x7feb0c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb1458f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0cc030 con 0x7feb0c160b10 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb1458f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0ca720 con 0x7feb0c06b060 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb1458f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0c9e10 con 0x7feb0c160740 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb12b2c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7feb0c06b060 0x7feb0c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb12b2c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7feb0c06b060 0x7feb0c06b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39566/0 (socket says 172.21.15.153:39566) 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.099+0000 7feb12b2c700 1 -- 172.21.15.153:0/3129006530 learned_addr learned my addr 172.21.15.153:0/3129006530 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb1232b700 1 --2- 172.21.15.153:0/3129006530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb12b2c700 1 -- 172.21.15.153:0/3129006530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 msgr2=0x7feb0c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb12b2c700 1 --2- 172.21.15.153:0/3129006530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb12b2c700 1 -- 172.21.15.153:0/3129006530 >> v1:172.21.15.177:6789/0 conn(0x7feb0c160b10 legacy=0x7feb0c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb12b2c700 1 -- 172.21.15.153:0/3129006530 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c06efc0 con 0x7feb0c06b060 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb1232b700 1 --2- 172.21.15.153:0/3129006530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb12b2c700 1 --2- 172.21.15.153:0/3129006530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c06b4e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feb04009e50 tx=0x7feb0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc29fa9c1f63f910 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb11329700 1 -- 172.21.15.153:0/3129006530 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0400e070 con 0x7feb0c06b060 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb11329700 1 -- 172.21.15.153:0/3129006530 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0400b9c0 con 0x7feb0c06b060 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb11329700 1 -- 172.21.15.153:0/3129006530 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0400bd70 con 0x7feb0c06b060 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb1458f700 1 -- 172.21.15.153:0/3129006530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 msgr2=0x7feb0c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.100+0000 7feb1458f700 1 --2- 172.21.15.153:0/3129006530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c06b4e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feb04009e50 tx=0x7feb0400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.101+0000 7feb1458f700 1 -- 172.21.15.153:0/3129006530 shutdown_connections 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.101+0000 7feb1458f700 1 --2- 172.21.15.153:0/3129006530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c06b4e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.101+0000 7feb1458f700 1 --2- 172.21.15.153:0/3129006530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.101+0000 7feb1458f700 1 -- 172.21.15.153:0/3129006530 >> 172.21.15.153:0/3129006530 conn(0x7feb0c0b12b0 msgr2=0x7feb0c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.101+0000 7feb1458f700 1 -- 172.21.15.153:0/3129006530 shutdown_connections 2023-11-28T13:00:56.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.101+0000 7feb1458f700 1 -- 172.21.15.153:0/3129006530 wait complete. 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.102+0000 7feb1458f700 1 Processor -- start 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.102+0000 7feb1458f700 1 -- start start 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c14f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb0c160b10 0x7feb0c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c14f8b0 con 0x7feb0c06b060 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1f3040 con 0x7feb0c160b10 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1f3320 con 0x7feb0c160740 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c14f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39570/0 (socket says 172.21.15.153:39570) 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 -- 172.21.15.153:0/1588713262 learned_addr learned my addr 172.21.15.153:0/1588713262 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb12b2c700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb0c160b10 0x7feb0c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb11b2a700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 -- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 msgr2=0x7feb0c156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 -- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb0c160b10 msgr2=0x7feb0c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb0c160b10 0x7feb0c157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 -- 172.21.15.153:0/1588713262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04009b30 con 0x7feb0c06b060 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1232b700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c14f370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e810 tx=0x7feafc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb02ffd700 1 -- 172.21.15.153:0/1588713262 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc013610 con 0x7feb0c06b060 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb02ffd700 1 -- 172.21.15.153:0/1588713262 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc013770 con 0x7feb0c06b060 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb02ffd700 1 -- 172.21.15.153:0/1588713262 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc013b20 con 0x7feb0c06b060 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c0ca3b0 con 0x7feb0c06b060 2023-11-28T13:00:56.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.103+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c0ca0d0 con 0x7feb0c06b060 2023-11-28T13:00:56.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.104+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c160f60 con 0x7feb0c06b060 2023-11-28T13:00:56.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.105+0000 7feb02ffd700 1 -- 172.21.15.153:0/1588713262 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feafc023070 con 0x7feb0c06b060 2023-11-28T13:00:56.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.105+0000 7feb02ffd700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feaf00682f0 0x7feaf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.105+0000 7feb11b2a700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feaf00682f0 0x7feaf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.105+0000 7feb02ffd700 1 -- 172.21.15.153:0/1588713262 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feafc085810 con 0x7feb0c06b060 2023-11-28T13:00:56.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.106+0000 7feb11b2a700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feaf00682f0 0x7feaf006a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb0c06d650 tx=0x7feb08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.109+0000 7feb02ffd700 1 -- 172.21.15.153:0/1588713262 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feafc051fe0 con 0x7feb0c06b060 2023-11-28T13:00:56.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.292+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb0c066110 con 0x7feb0c06b060 2023-11-28T13:00:56.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.294+0000 7feb02ffd700 1 -- 172.21.15.153:0/1588713262 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+17955 (secure 0 0 0) 0x7feafc055600 con 0x7feb0c06b060 2023-11-28T13:00:56.297 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:56.297 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:18.626485+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:56.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feaf00682f0 msgr2=0x7feaf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feaf00682f0 0x7feaf006a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb0c06d650 tx=0x7feb08009040 comp rx=0 tx=0).stop 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 msgr2=0x7feb0c14f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c14f370 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e810 tx=0x7feafc00eb20 comp rx=0 tx=0).stop 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 shutdown_connections 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feaf00682f0 0x7feaf006a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c06b060 0x7feb0c14f370 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb0c160b10 0x7feb0c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 --2- 172.21.15.153:0/1588713262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb0c160740 0x7feb0c156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 >> 172.21.15.153:0/1588713262 conn(0x7feb0c0b12b0 msgr2=0x7feb0c0af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 shutdown_connections 2023-11-28T13:00:56.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.296+0000 7feb1458f700 1 -- 172.21.15.153:0/1588713262 wait complete. 2023-11-28T13:00:56.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:56.310 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:56.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.432+0000 7f8389e29700 1 Processor -- start 2023-11-28T13:00:56.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.432+0000 7f8389e29700 1 -- start start 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8389e29700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8384071040 0x7f8384071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8389e29700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f838406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8389e29700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8384071950 con 0x7f838406cdc0 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8389e29700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8384161550 con 0x7f8384071040 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8389e29700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838415d060 con 0x7f838406dee0 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8382ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f838406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f83837fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8384071040 0x7f8384071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8382ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f838406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:57188/0 (socket says 172.21.15.153:57188) 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8383fff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f838406cdc0 0x7f838406d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:34360/0 (socket says 172.21.15.153:34360) 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f8382ffd700 1 -- 172.21.15.153:0/2105958137 learned_addr learned my addr 172.21.15.153:0/2105958137 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.433+0000 7f83837fe700 1 --2- 172.21.15.153:0/2105958137 >> v2:172.21.15.177:3300/0 conn(0x7f8384071040 0x7f8384071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:49044/0 (socket says 172.21.15.153:49044) 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f8382ffd700 1 -- 172.21.15.153:0/2105958137 >> v2:172.21.15.177:3300/0 conn(0x7f8384071040 msgr2=0x7f8384071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f8382ffd700 1 --2- 172.21.15.153:0/2105958137 >> v2:172.21.15.177:3300/0 conn(0x7f8384071040 0x7f8384071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f8382ffd700 1 -- 172.21.15.153:0/2105958137 >> v1:172.21.15.177:6789/0 conn(0x7f838406cdc0 legacy=0x7f838406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f8382ffd700 1 -- 172.21.15.153:0/2105958137 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838415d340 con 0x7f838406dee0 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f8382ffd700 1 --2- 172.21.15.153:0/2105958137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f838406e330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f83740098f0 tx=0x7f8374009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f09f7ec12e9192 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f83827fc700 1 -- 172.21.15.153:0/2105958137 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374011830 con 0x7f838406dee0 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f83827fc700 1 -- 172.21.15.153:0/2105958137 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8374011990 con 0x7f838406dee0 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.434+0000 7f83827fc700 1 -- 172.21.15.153:0/2105958137 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374011d20 con 0x7f838406dee0 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 -- 172.21.15.153:0/2105958137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 msgr2=0x7f838406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 --2- 172.21.15.153:0/2105958137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f838406e330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f83740098f0 tx=0x7f8374009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 -- 172.21.15.153:0/2105958137 shutdown_connections 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 --2- 172.21.15.153:0/2105958137 >> v2:172.21.15.177:3300/0 conn(0x7f8384071040 0x7f8384071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 --2- 172.21.15.153:0/2105958137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f838406e330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 -- 172.21.15.153:0/2105958137 >> 172.21.15.153:0/2105958137 conn(0x7f83840d0000 msgr2=0x7f83840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 -- 172.21.15.153:0/2105958137 shutdown_connections 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.435+0000 7f8389e29700 1 -- 172.21.15.153:0/2105958137 wait complete. 2023-11-28T13:00:56.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.436+0000 7f8389e29700 1 Processor -- start 2023-11-28T13:00:56.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.436+0000 7f8389e29700 1 -- start start 2023-11-28T13:00:56.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8389e29700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838406cdc0 0x7f8384146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8389e29700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f83840b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8389e29700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8384071040 0x7f83840b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8389e29700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8384146a90 con 0x7f838406cdc0 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8389e29700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b58c0 con 0x7f8384071040 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8389e29700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840b5ba0 con 0x7f838406dee0 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838406cdc0 0x7f8384146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838406cdc0 0x7f8384146160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39584/0 (socket says 172.21.15.153:39584) 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 -- 172.21.15.153:0/3346993226 learned_addr learned my addr 172.21.15.153:0/3346993226 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8383fff700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8384071040 0x7f83840b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f8382ffd700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f83840b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 -- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 msgr2=0x7f83840b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f83840b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 -- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8384071040 msgr2=0x7f83840b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8384071040 0x7f83840b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 -- 172.21.15.153:0/3346993226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8374009550 con 0x7f838406cdc0 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f83837fe700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838406cdc0 0x7f8384146160 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f837800e9b0 tx=0x7f837800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.437+0000 7f836bfff700 1 -- 172.21.15.153:0/3346993226 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378009710 con 0x7f838406cdc0 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.438+0000 7f836bfff700 1 -- 172.21.15.153:0/3346993226 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837800c850 con 0x7f838406cdc0 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.438+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838415d900 con 0x7f838406cdc0 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.438+0000 7f836bfff700 1 -- 172.21.15.153:0/3346993226 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837800cbc0 con 0x7f838406cdc0 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.438+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838415d620 con 0x7f838406cdc0 2023-11-28T13:00:56.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.439+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838406e870 con 0x7f838406cdc0 2023-11-28T13:00:56.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.439+0000 7f836bfff700 1 -- 172.21.15.153:0/3346993226 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8378019030 con 0x7f838406cdc0 2023-11-28T13:00:56.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.439+0000 7f836bfff700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f83700682f0 0x7f837006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.439+0000 7f836bfff700 1 -- 172.21.15.153:0/3346993226 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8378085ca0 con 0x7f838406cdc0 2023-11-28T13:00:56.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.439+0000 7f8382ffd700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f83700682f0 0x7f837006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.440+0000 7f8382ffd700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f83700682f0 0x7f837006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8374000c00 tx=0x7f8374008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.444+0000 7f836bfff700 1 -- 172.21.15.153:0/3346993226 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f837808a070 con 0x7f838406cdc0 2023-11-28T13:00:56.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.629+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83840ad780 con 0x7f838406cdc0 2023-11-28T13:00:56.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.629+0000 7f836bfff700 1 -- 172.21.15.153:0/3346993226 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83780210a0 con 0x7f838406cdc0 2023-11-28T13:00:56.630 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:56.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.631+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f83700682f0 msgr2=0x7f837006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.631+0000 7f8389e29700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f83700682f0 0x7f837006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8374000c00 tx=0x7f8374008030 comp rx=0 tx=0).stop 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.631+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838406cdc0 msgr2=0x7f8384146160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.631+0000 7f8389e29700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838406cdc0 0x7f8384146160 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f837800e9b0 tx=0x7f837800ed70 comp rx=0 tx=0).stop 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 shutdown_connections 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f83700682f0 0x7f837006a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838406cdc0 0x7f8384146160 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8384071040 0x7f83840b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 --2- 172.21.15.153:0/3346993226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f838406dee0 0x7f83840b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 >> 172.21.15.153:0/3346993226 conn(0x7f83840d0000 msgr2=0x7f838406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 shutdown_connections 2023-11-28T13:00:56.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.632+0000 7f8389e29700 1 -- 172.21.15.153:0/3346993226 wait complete. 2023-11-28T13:00:56.645 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:56.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.767+0000 7fe5236ba700 1 Processor -- start 2023-11-28T13:00:56.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.768+0000 7fe5236ba700 1 -- start start 2023-11-28T13:00:56.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.768+0000 7fe5236ba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 0x7fe51c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.768+0000 7fe5236ba700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c0cc5f0 con 0x7fe51c15cef0 2023-11-28T13:00:56.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.768+0000 7fe5236ba700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c15d470 con 0x7fe51c06ba90 2023-11-28T13:00:56.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.768+0000 7fe5236ba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c15d750 con 0x7fe51c0cbdf0 2023-11-28T13:00:56.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.768+0000 7fe521456700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe51c15cef0 0x7fe51c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57756/0 (socket says 172.21.15.153:57756) 2023-11-28T13:00:56.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.768+0000 7fe521456700 1 -- 172.21.15.153:0/15666279 learned_addr learned my addr 172.21.15.153:0/15666279 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:56.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521c57700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 0x7fe51c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706803368 0 0) 0x7fe51c0cc5f0 con 0x7fe51c15cef0 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5080035b0 con 0x7fe51c15cef0 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 224686137 0 0) 0x7fe5080035b0 con 0x7fe51c15cef0 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 msgr2=0x7fe51c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 0x7fe51c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 >> v1:172.21.15.177:6789/0 conn(0x7fe51c06ba90 legacy=0x7fe51c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c15da30 con 0x7fe51c15cef0 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fe50c001670 con 0x7fe51c15cef0 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 >> v1:172.21.15.153:6789/0 conn(0x7fe51c15cef0 legacy=0x7fe51c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe508003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 0x7fe508004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe508004d90 0x7fe508005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c15d750 con 0x7fe508003a30 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50800ba10 con 0x7fe508004d90 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50800bba0 con 0x7fe51c0cbdf0 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521456700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe508003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe520c55700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 0x7fe508004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521c57700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe508004d90 0x7fe508005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521456700 1 -- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 msgr2=0x7fe508004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521456700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0cbdf0 0x7fe508004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521456700 1 -- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe508004d90 msgr2=0x7fe508005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521456700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe508004d90 0x7fe508005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521456700 1 -- 172.21.15.153:0/15666279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080035b0 con 0x7fe508003a30 2023-11-28T13:00:56.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521c57700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe508004d90 0x7fe508005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.769+0000 7fe521456700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe508003e00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c2f0 tx=0x7fe508005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aef7e5836e6fb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00ce30 con 0x7fe508003a30 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c005960 con 0x7fe508003a30 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe513fff700 1 -- 172.21.15.153:0/15666279 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c005cf0 con 0x7fe508003a30 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe5236ba700 1 -- 172.21.15.153:0/15666279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 msgr2=0x7fe508003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe5236ba700 1 --2- 172.21.15.153:0/15666279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe508003e00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c2f0 tx=0x7fe508005880 comp rx=0 tx=0).stop 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe521c57700 1 -- 172.21.15.153:0/15666279 reap_dead start 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe5236ba700 1 -- 172.21.15.153:0/15666279 shutdown_connections 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe5236ba700 1 -- 172.21.15.153:0/15666279 >> 172.21.15.153:0/15666279 conn(0x7fe51c0d0000 msgr2=0x7fe51c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe5236ba700 1 -- 172.21.15.153:0/15666279 shutdown_connections 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.770+0000 7fe5236ba700 1 -- 172.21.15.153:0/15666279 wait complete. 2023-11-28T13:00:56.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.771+0000 7fe5236ba700 1 Processor -- start 2023-11-28T13:00:56.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.771+0000 7fe5236ba700 1 -- start start 2023-11-28T13:00:56.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe5236ba700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe51c0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe5236ba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe51c1506c0 0x7fe51c0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe5236ba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0c9b10 0x7fe51c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe5236ba700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c9f80 con 0x7fe508003a30 2023-11-28T13:00:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe5236ba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c7610 con 0x7fe51c1506c0 2023-11-28T13:00:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe5236ba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c7940 con 0x7fe51c0c9b10 2023-11-28T13:00:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe521456700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe51c0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe521456700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe51c0c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39602/0 (socket says 172.21.15.153:39602) 2023-11-28T13:00:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.772+0000 7fe521456700 1 -- 172.21.15.153:0/2652870439 learned_addr learned my addr 172.21.15.153:0/2652870439 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521c57700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0c9b10 0x7fe51c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe520c55700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe51c1506c0 0x7fe51c0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521456700 1 -- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0c9b10 msgr2=0x7fe51c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521456700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0c9b10 0x7fe51c06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521456700 1 -- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe51c1506c0 msgr2=0x7fe51c0c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521456700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe51c1506c0 0x7fe51c0c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521456700 1 -- 172.21.15.153:0/2652870439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c15dff0 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe520c55700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe51c1506c0 0x7fe51c0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521c57700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0c9b10 0x7fe51c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe521456700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe51c0c70d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe508005800 tx=0x7fe50c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe511ffb700 1 -- 172.21.15.153:0/2652870439 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c001660 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe511ffb700 1 -- 172.21.15.153:0/2652870439 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c004a80 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c15dd10 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe511ffb700 1 -- 172.21.15.153:0/2652870439 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004e10 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.773+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c0af400 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.774+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c0c8b10 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.774+0000 7fe511ffb700 1 -- 172.21.15.153:0/2652870439 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe50c02e070 con 0x7fe508003a30 2023-11-28T13:00:56.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.775+0000 7fe511ffb700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5000682f0 0x7fe50006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.775+0000 7fe511ffb700 1 -- 172.21.15.153:0/2652870439 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe50c0874b0 con 0x7fe508003a30 2023-11-28T13:00:56.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.775+0000 7fe520c55700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5000682f0 0x7fe50006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.775+0000 7fe520c55700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5000682f0 0x7fe50006a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe51c1f3390 tx=0x7fe518009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.779+0000 7fe511ffb700 1 -- 172.21.15.153:0/2652870439 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe50c053dd0 con 0x7fe508003a30 2023-11-28T13:00:56.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.960+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe51c0d0e60 con 0x7fe508003a30 2023-11-28T13:00:56.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.960+0000 7fe511ffb700 1 -- 172.21.15.153:0/2652870439 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe50c007da0 con 0x7fe508003a30 2023-11-28T13:00:56.962 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:56.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5000682f0 msgr2=0x7fe50006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5000682f0 0x7fe50006a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe51c1f3390 tx=0x7fe518009040 comp rx=0 tx=0).stop 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 msgr2=0x7fe51c0c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe51c0c70d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe508005800 tx=0x7fe50c001a00 comp rx=0 tx=0).stop 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 shutdown_connections 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe5000682f0 0x7fe50006a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe508003a30 0x7fe51c0c70d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe51c0c9b10 0x7fe51c06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.962+0000 7fe5236ba700 1 --2- 172.21.15.153:0/2652870439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe51c1506c0 0x7fe51c0c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.963+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 >> 172.21.15.153:0/2652870439 conn(0x7fe51c0d0000 msgr2=0x7fe51c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.963+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 shutdown_connections 2023-11-28T13:00:56.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:56.963+0000 7fe5236ba700 1 -- 172.21.15.153:0/2652870439 wait complete. 2023-11-28T13:00:56.975 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:57.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.098+0000 7f5318bb9700 1 Processor -- start 2023-11-28T13:00:57.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.098+0000 7f5318bb9700 1 -- start start 2023-11-28T13:00:57.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f5318bb9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f5318bb9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f53140ae9d0 0x7f531406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f5318bb9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531406ee10 con 0x7f531415e540 2023-11-28T13:00:57.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f5318bb9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531415bf10 con 0x7f53140ae9d0 2023-11-28T13:00:57.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f5318bb9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53140aee40 con 0x7f531406e500 2023-11-28T13:00:57.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f531259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f531259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:57220/0 (socket says 172.21.15.153:57220) 2023-11-28T13:00:57.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f531259c700 1 -- 172.21.15.153:0/4042700380 learned_addr learned my addr 172.21.15.153:0/4042700380 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:57.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.099+0000 7f5311d9b700 1 --2- 172.21.15.153:0/4042700380 >> v2:172.21.15.177:3300/0 conn(0x7f53140ae9d0 0x7f531406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531259c700 1 -- 172.21.15.153:0/4042700380 >> v2:172.21.15.177:3300/0 conn(0x7f53140ae9d0 msgr2=0x7f531406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531259c700 1 --2- 172.21.15.153:0/4042700380 >> v2:172.21.15.177:3300/0 conn(0x7f53140ae9d0 0x7f531406e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531259c700 1 -- 172.21.15.153:0/4042700380 >> v1:172.21.15.177:6789/0 conn(0x7f531415e540 legacy=0x7f531415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531259c700 1 -- 172.21.15.153:0/4042700380 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531406c930 con 0x7f531406e500 2023-11-28T13:00:57.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531259c700 1 --2- 172.21.15.153:0/4042700380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531415b9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f53040098f0 tx=0x7f5304009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9247a1e8f0ba301 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531159a700 1 -- 172.21.15.153:0/4042700380 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304011830 con 0x7f531406e500 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531159a700 1 -- 172.21.15.153:0/4042700380 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5304011990 con 0x7f531406e500 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f531159a700 1 -- 172.21.15.153:0/4042700380 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304011d20 con 0x7f531406e500 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f5318bb9700 1 -- 172.21.15.153:0/4042700380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 msgr2=0x7f531415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f5318bb9700 1 --2- 172.21.15.153:0/4042700380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531415b9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f53040098f0 tx=0x7f5304009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f5318bb9700 1 -- 172.21.15.153:0/4042700380 shutdown_connections 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f5318bb9700 1 --2- 172.21.15.153:0/4042700380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531415b9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f5318bb9700 1 --2- 172.21.15.153:0/4042700380 >> v2:172.21.15.177:3300/0 conn(0x7f53140ae9d0 0x7f531406e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.100+0000 7f5318bb9700 1 -- 172.21.15.153:0/4042700380 >> 172.21.15.153:0/4042700380 conn(0x7f53140b12b0 msgr2=0x7f53140b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.101+0000 7f5318bb9700 1 -- 172.21.15.153:0/4042700380 shutdown_connections 2023-11-28T13:00:57.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.101+0000 7f5318bb9700 1 -- 172.21.15.153:0/4042700380 wait complete. 2023-11-28T13:00:57.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.102+0000 7f5318bb9700 1 Processor -- start 2023-11-28T13:00:57.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.102+0000 7f5318bb9700 1 -- start start 2023-11-28T13:00:57.110 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53140ae9d0 0x7f53141e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f531415e540 0x7f53140b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5314156190 con 0x7f53140ae9d0 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5314156ff0 con 0x7f531415e540 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53141563e0 con 0x7f531406e500 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53140ae9d0 0x7f53141e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f531259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53140ae9d0 0x7f53141e4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39614/0 (socket says 172.21.15.153:39614) 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f531259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531414e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:57222/0 (socket says 172.21.15.153:57222) 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5312d9d700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f531415e540 0x7f53140b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 -- 172.21.15.153:0/264227858 learned_addr learned my addr 172.21.15.153:0/264227858 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 -- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 msgr2=0x7f531414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 -- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f531415e540 msgr2=0x7f53140b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f531415e540 0x7f53140b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 -- 172.21.15.153:0/264227858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304009550 con 0x7f53140ae9d0 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5312d9d700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f531415e540 0x7f53140b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f531259c700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5311d9b700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53140ae9d0 0x7f53141e4d20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f530800e9b0 tx=0x7f530800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5302ffd700 1 -- 172.21.15.153:0/264227858 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308009710 con 0x7f53140ae9d0 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5302ffd700 1 -- 172.21.15.153:0/264227858 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530800ee50 con 0x7f53140ae9d0 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53140664c0 con 0x7f53140ae9d0 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5302ffd700 1 -- 172.21.15.153:0/264227858 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530800ca50 con 0x7f53140ae9d0 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.103+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531406cc10 con 0x7f53140ae9d0 2023-11-28T13:00:57.113 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.104+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531406cef0 con 0x7f53140ae9d0 2023-11-28T13:00:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.105+0000 7f5302ffd700 1 -- 172.21.15.153:0/264227858 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5308019030 con 0x7f53140ae9d0 2023-11-28T13:00:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.105+0000 7f5302ffd700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52f40682f0 0x7f52f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.105+0000 7f5302ffd700 1 -- 172.21.15.153:0/264227858 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5308085b10 con 0x7f53140ae9d0 2023-11-28T13:00:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.105+0000 7f531259c700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52f40682f0 0x7f52f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.106+0000 7f531259c700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52f40682f0 0x7f52f406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5304014040 tx=0x7f5304008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.114 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.109+0000 7f5302ffd700 1 -- 172.21.15.153:0/264227858 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5308052370 con 0x7f53140ae9d0 2023-11-28T13:00:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.292+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5314065fc0 con 0x7f53140ae9d0 2023-11-28T13:00:57.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.292+0000 7f5302ffd700 1 -- 172.21.15.153:0/264227858 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5308055990 con 0x7f53140ae9d0 2023-11-28T13:00:57.293 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:57.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.294+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52f40682f0 msgr2=0x7f52f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.294+0000 7f5318bb9700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52f40682f0 0x7f52f406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5304014040 tx=0x7f5304008030 comp rx=0 tx=0).stop 2023-11-28T13:00:57.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.294+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53140ae9d0 msgr2=0x7f53141e4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.294+0000 7f5318bb9700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53140ae9d0 0x7f53141e4d20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f530800e9b0 tx=0x7f530800ecc0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 shutdown_connections 2023-11-28T13:00:57.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f531415e540 0x7f53140b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f531406e500 0x7f531414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52f40682f0 0x7f52f406a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 --2- 172.21.15.153:0/264227858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53140ae9d0 0x7f53141e4d20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 >> 172.21.15.153:0/264227858 conn(0x7f53140b12b0 msgr2=0x7f531406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 shutdown_connections 2023-11-28T13:00:57.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.295+0000 7f5318bb9700 1 -- 172.21.15.153:0/264227858 wait complete. 2023-11-28T13:00:57.308 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:57.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.432+0000 7f50f2edb700 1 Processor -- start 2023-11-28T13:00:57.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.432+0000 7f50f2edb700 1 -- start start 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f2edb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f50ec15cef0 0x7f50ec0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f2edb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f2edb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f50ec06b8c0 0x7f50ec071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f2edb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec06bd40 con 0x7f50ec15cef0 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f2edb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec0cc5f0 con 0x7f50ec06b8c0 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f2edb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15d470 con 0x7f50ec0cbdf0 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f50ec15cef0 0x7f50ec0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f50ec15cef0 0x7f50ec0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39620/0 (socket says 172.21.15.153:39620) 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 -- 172.21.15.153:0/3809220374 learned_addr learned my addr 172.21.15.153:0/3809220374 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:57.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50ebfff700 1 --2- 172.21.15.153:0/3809220374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f1478700 1 --2- 172.21.15.153:0/3809220374 >> v2:172.21.15.177:3300/0 conn(0x7f50ec06b8c0 0x7f50ec071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 -- 172.21.15.153:0/3809220374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 msgr2=0x7f50ec06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 --2- 172.21.15.153:0/3809220374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 -- 172.21.15.153:0/3809220374 >> v2:172.21.15.177:3300/0 conn(0x7f50ec06b8c0 msgr2=0x7f50ec071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 --2- 172.21.15.153:0/3809220374 >> v2:172.21.15.177:3300/0 conn(0x7f50ec06b8c0 0x7f50ec071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 -- 172.21.15.153:0/3809220374 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec15d750 con 0x7f50ec15cef0 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50ebfff700 1 --2- 172.21.15.153:0/3809220374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f1478700 1 --2- 172.21.15.153:0/3809220374 >> v2:172.21.15.177:3300/0 conn(0x7f50ec06b8c0 0x7f50ec071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50f0c77700 1 --2- 172.21.15.153:0/3809220374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec15cef0 0x7f50ec0af400 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50ec073a70 tx=0x7f50e000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b838c8f64f4304e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.433+0000 7f50eb7fe700 1 -- 172.21.15.153:0/3809220374 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000dd30 con 0x7f50ec15cef0 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50eb7fe700 1 -- 172.21.15.153:0/3809220374 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e000de90 con 0x7f50ec15cef0 2023-11-28T13:00:57.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50eb7fe700 1 -- 172.21.15.153:0/3809220374 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0010690 con 0x7f50ec15cef0 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 -- 172.21.15.153:0/3809220374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec15cef0 msgr2=0x7f50ec0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 --2- 172.21.15.153:0/3809220374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec15cef0 0x7f50ec0af400 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f50ec073a70 tx=0x7f50e000be70 comp rx=0 tx=0).stop 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 -- 172.21.15.153:0/3809220374 shutdown_connections 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 --2- 172.21.15.153:0/3809220374 >> v2:172.21.15.177:3300/0 conn(0x7f50ec06b8c0 0x7f50ec071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 --2- 172.21.15.153:0/3809220374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec15cef0 0x7f50ec0af400 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 --2- 172.21.15.153:0/3809220374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 -- 172.21.15.153:0/3809220374 >> 172.21.15.153:0/3809220374 conn(0x7f50ec0d0000 msgr2=0x7f50ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 -- 172.21.15.153:0/3809220374 shutdown_connections 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.434+0000 7f50f2edb700 1 -- 172.21.15.153:0/3809220374 wait complete. 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.435+0000 7f50f2edb700 1 Processor -- start 2023-11-28T13:00:57.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.435+0000 7f50f2edb700 1 -- start start 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f2edb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec06b8c0 0x7f50ec15af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f2edb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec09a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f2edb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec15cef0 0x7f50ec0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f2edb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec14a0c0 con 0x7f50ec06b8c0 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f2edb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec14af20 con 0x7f50ec15cef0 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f2edb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec149e70 con 0x7f50ec0cbdf0 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f0c77700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec06b8c0 0x7f50ec15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f0c77700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec06b8c0 0x7f50ec15af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39626/0 (socket says 172.21.15.153:39626) 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.436+0000 7f50f0c77700 1 -- 172.21.15.153:0/2020548267 learned_addr learned my addr 172.21.15.153:0/2020548267 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f1478700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec15cef0 0x7f50ec0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50ebfff700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec09a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f0c77700 1 -- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 msgr2=0x7f50ec09a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f0c77700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec09a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f0c77700 1 -- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec15cef0 msgr2=0x7f50ec0c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f0c77700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec15cef0 0x7f50ec0c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f0c77700 1 -- 172.21.15.153:0/2020548267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e000b7f0 con 0x7f50ec06b8c0 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f1478700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec15cef0 0x7f50ec0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50ebfff700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec09a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f0c77700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec06b8c0 0x7f50ec15af50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50ec06c7b0 tx=0x7f50e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50e97fa700 1 -- 172.21.15.153:0/2020548267 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e001f070 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50e97fa700 1 -- 172.21.15.153:0/2020548267 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0003b20 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec15dd10 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50e97fa700 1 -- 172.21.15.153:0/2020548267 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0010470 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.437+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec15da30 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.438+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec15dff0 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.439+0000 7f50e97fa700 1 -- 172.21.15.153:0/2020548267 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f50e002d460 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.439+0000 7f50e97fa700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f50d00682f0 0x7f50d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.439+0000 7f50e97fa700 1 -- 172.21.15.153:0/2020548267 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50e0089ac0 con 0x7f50ec06b8c0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.440+0000 7f50ebfff700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f50d00682f0 0x7f50d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.440+0000 7f50ebfff700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f50d00682f0 0x7f50d006a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009a00 tx=0x7f50d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.443+0000 7f50e97fa700 1 -- 172.21.15.153:0/2020548267 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f50e0056250 con 0x7f50ec06b8c0 2023-11-28T13:00:57.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.629+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50ec14b060 con 0x7f50ec06b8c0 2023-11-28T13:00:57.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.629+0000 7f50e97fa700 1 -- 172.21.15.153:0/2020548267 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50ec14b060 con 0x7f50ec06b8c0 2023-11-28T13:00:57.630 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:00:57.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f50d00682f0 msgr2=0x7f50d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f50d00682f0 0x7f50d006a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009a00 tx=0x7f50d8008040 comp rx=0 tx=0).stop 2023-11-28T13:00:57.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec06b8c0 msgr2=0x7f50ec15af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec06b8c0 0x7f50ec15af50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50ec06c7b0 tx=0x7f50e0004750 comp rx=0 tx=0).stop 2023-11-28T13:00:57.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 shutdown_connections 2023-11-28T13:00:57.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f50d00682f0 0x7f50d006a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f50ec06b8c0 0x7f50ec15af50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec15cef0 0x7f50ec0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 --2- 172.21.15.153:0/2020548267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec0cbdf0 0x7f50ec09a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 >> 172.21.15.153:0/2020548267 conn(0x7f50ec0d0000 msgr2=0x7f50ec06d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 shutdown_connections 2023-11-28T13:00:57.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.631+0000 7f50f2edb700 1 -- 172.21.15.153:0/2020548267 wait complete. 2023-11-28T13:00:57.645 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:00:57.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.766+0000 7f19c97d8700 1 Processor -- start 2023-11-28T13:00:57.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.766+0000 7f19c97d8700 1 -- start start 2023-11-28T13:00:57.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c97d8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19c40cd7c0 0x7f19c40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c97d8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c40d1950 con 0x7f19c4161340 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c97d8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c40ce140 con 0x7f19c415c9b0 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c97d8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40b5800 con 0x7f19c40cd7c0 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c27fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f19c4161340 0x7f19c40d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57796/0 (socket says 172.21.15.153:57796) 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c27fc700 1 -- 172.21.15.153:0/1601673415 learned_addr learned my addr 172.21.15.153:0/1601673415 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c37fe700 1 --2- 172.21.15.153:0/1601673415 >> v2:172.21.15.177:3300/0 conn(0x7f19c40cd7c0 0x7f19c40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623221359 0 0) 0x7f19c40d1950 con 0x7f19c4161340 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19ac0035b0 con 0x7f19c4161340 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2740931165 0 0) 0x7f19ac0035b0 con 0x7f19c4161340 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 >> v2:172.21.15.177:3300/0 conn(0x7f19c40cd7c0 msgr2=0x7f19c40cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 --2- 172.21.15.153:0/1601673415 >> v2:172.21.15.177:3300/0 conn(0x7f19c40cd7c0 0x7f19c40cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 >> v1:172.21.15.177:6789/0 conn(0x7f19c415c9b0 legacy=0x7f19c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40d24f0 con 0x7f19c4161340 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f19b4001670 con 0x7f19c4161340 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 >> v1:172.21.15.153:6789/0 conn(0x7f19c4161340 legacy=0x7f19c40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19ac003a00 0x7f19ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac004850 0x7f19ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.767+0000 7f19c1ffb700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ac005200 0x7f19ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40b5800 con 0x7f19ac003a00 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac00c110 con 0x7f19ac005200 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac00c2d0 con 0x7f19ac004850 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19ac003a00 0x7f19ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c37fe700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ac005200 0x7f19ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c27fc700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac004850 0x7f19ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c2ffd700 1 -- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac004850 msgr2=0x7f19ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac004850 0x7f19ac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c2ffd700 1 -- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ac005200 msgr2=0x7f19ac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ac005200 0x7f19ac00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c2ffd700 1 -- 172.21.15.153:0/1601673415 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac0035b0 con 0x7f19ac003a00 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c37fe700 1 -- 172.21.15.153:0/1601673415 reap_dead start 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c27fc700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac004850 0x7f19ac004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19ac003a00 0x7f19ac003e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19b80017c0 tx=0x7f19ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73868be33eb403b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80052d0 con 0x7f19ac003a00 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b8005450 con 0x7f19ac003a00 2023-11-28T13:00:57.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.768+0000 7f19c1ffb700 1 -- 172.21.15.153:0/1601673415 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b80038c0 con 0x7f19ac003a00 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.769+0000 7f19c97d8700 1 -- 172.21.15.153:0/1601673415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19ac003a00 msgr2=0x7f19ac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.769+0000 7f19c97d8700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19ac003a00 0x7f19ac003e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19b80017c0 tx=0x7f19ac005880 comp rx=0 tx=0).stop 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.769+0000 7f19c97d8700 1 -- 172.21.15.153:0/1601673415 shutdown_connections 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.769+0000 7f19c97d8700 1 --2- 172.21.15.153:0/1601673415 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19ac003a00 0x7f19ac003e10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.769+0000 7f19c97d8700 1 -- 172.21.15.153:0/1601673415 >> 172.21.15.153:0/1601673415 conn(0x7f19c4072600 msgr2=0x7f19c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.769+0000 7f19c97d8700 1 -- 172.21.15.153:0/1601673415 shutdown_connections 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.769+0000 7f19c97d8700 1 -- 172.21.15.153:0/1601673415 wait complete. 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.770+0000 7f19c97d8700 1 Processor -- start 2023-11-28T13:00:57.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.770+0000 7f19c97d8700 1 -- start start 2023-11-28T13:00:57.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c97d8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac003a00 0x7f19c40c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c97d8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19c40c6e60 0x7f19c41f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c97d8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c415e860 0x7f19c415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c97d8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40ca0c0 con 0x7f19c415e860 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c97d8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c415f210 con 0x7f19c40c6e60 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c97d8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40c9b00 con 0x7f19ac003a00 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c415e860 0x7f19c415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c415e860 0x7f19c415ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39648/0 (socket says 172.21.15.153:39648) 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 -- 172.21.15.153:0/78788692 learned_addr learned my addr 172.21.15.153:0/78788692 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c27fc700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19c40c6e60 0x7f19c41f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac003a00 0x7f19c40c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 -- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac003a00 msgr2=0x7f19c40c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac003a00 0x7f19c40c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 -- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19c40c6e60 msgr2=0x7f19c41f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19c40c6e60 0x7f19c41f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 -- 172.21.15.153:0/78788692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40d1300 con 0x7f19c415e860 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac003a00 0x7f19c40c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c37fe700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c415e860 0x7f19c415ecd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19c40d05d0 tx=0x7f19bc011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19ab7fe700 1 -- 172.21.15.153:0/78788692 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00e040 con 0x7f19c415e860 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19ab7fe700 1 -- 172.21.15.153:0/78788692 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00f040 con 0x7f19c415e860 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.771+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c40b5520 con 0x7f19c415e860 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.772+0000 7f19ab7fe700 1 -- 172.21.15.153:0/78788692 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00fd80 con 0x7f19c415e860 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.772+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c40d15e0 con 0x7f19c415e860 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.772+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c40745c0 con 0x7f19c415e860 2023-11-28T13:00:57.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.773+0000 7f19ab7fe700 1 -- 172.21.15.153:0/78788692 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19bc015430 con 0x7f19c415e860 2023-11-28T13:00:57.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.773+0000 7f19ab7fe700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19b00682f0 0x7f19b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.773+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19b00682f0 0x7f19b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.773+0000 7f19ab7fe700 1 -- 172.21.15.153:0/78788692 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19bc086380 con 0x7f19c415e860 2023-11-28T13:00:57.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.774+0000 7f19c2ffd700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19b00682f0 0x7f19b006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19b8015040 tx=0x7f19b8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.778+0000 7f19ab7fe700 1 -- 172.21.15.153:0/78788692 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19bc052b80 con 0x7f19c415e860 2023-11-28T13:00:57.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.974+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f19c40d08a0 con 0x7f19c415e860 2023-11-28T13:00:57.975 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.974+0000 7f19ab7fe700 1 -- 172.21.15.153:0/78788692 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f19bc0561a0 con 0x7f19c415e860 2023-11-28T13:00:57.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.976+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19b00682f0 msgr2=0x7f19b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.976+0000 7f19c97d8700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19b00682f0 0x7f19b006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19b8015040 tx=0x7f19b8009030 comp rx=0 tx=0).stop 2023-11-28T13:00:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.976+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c415e860 msgr2=0x7f19c415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.976+0000 7f19c97d8700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c415e860 0x7f19c415ecd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19c40d05d0 tx=0x7f19bc011f80 comp rx=0 tx=0).stop 2023-11-28T13:00:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 shutdown_connections 2023-11-28T13:00:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19b00682f0 0x7f19b006a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19c415e860 0x7f19c415ecd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ac003a00 0x7f19c40c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 --2- 172.21.15.153:0/78788692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19c40c6e60 0x7f19c41f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 >> 172.21.15.153:0/78788692 conn(0x7f19c4072600 msgr2=0x7f19c4160460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 shutdown_connections 2023-11-28T13:00:57.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:57.977+0000 7f19c97d8700 1 -- 172.21.15.153:0/78788692 wait complete. 2023-11-28T13:00:57.980 INFO:teuthology.orchestra.run.smithi153.stderr:listed 0 entries 2023-11-28T13:00:57.990 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T13:00:57.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:57.990 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T13:00:58.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T13:00:58.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:58.038 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T13:00:58.057 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:00:58.057 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:58.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.180+0000 7f498e790700 1 Processor -- start 2023-11-28T13:00:58.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.180+0000 7f498e790700 1 -- start start 2023-11-28T13:00:58.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f498e790700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 0x7f4988167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f498e790700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498815d060 con 0x7f498806dee0 2023-11-28T13:00:58.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f498e790700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498815d340 con 0x7f4988071040 2023-11-28T13:00:58.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f498e790700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4988161550 con 0x7f49881658c0 2023-11-28T13:00:58.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f49877fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f498806dee0 0x7f498806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57822/0 (socket says 172.21.15.153:57822) 2023-11-28T13:00:58.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f49877fe700 1 -- 172.21.15.153:0/2924116483 learned_addr learned my addr 172.21.15.153:0/2924116483 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:58.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f498cd2d700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 0x7f4988167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039729396 0 0) 0x7f498815d060 con 0x7f498806dee0 2023-11-28T13:00:58.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49700035b0 con 0x7f498806dee0 2023-11-28T13:00:58.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530409778 0 0) 0x7f498815d340 con 0x7f4988071040 2023-11-28T13:00:58.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f498815d060 con 0x7f4988071040 2023-11-28T13:00:58.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1747342316 0 0) 0x7f49700035b0 con 0x7f498806dee0 2023-11-28T13:00:58.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 msgr2=0x7f4988167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 0x7f4988167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 >> v1:172.21.15.177:6789/0 conn(0x7f4988071040 legacy=0x7f4988071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498815d620 con 0x7f498806dee0 2023-11-28T13:00:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f497c001670 con 0x7f498806dee0 2023-11-28T13:00:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 >> v1:172.21.15.153:6789/0 conn(0x7f498806dee0 legacy=0x7f498806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4970003bc0 0x7f4970003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 0x7f4970004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970004d90 0x7f4970005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.181+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4988161550 con 0x7f4970003bc0 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497000bba0 con 0x7f4970004d90 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497000bd60 con 0x7f49881658c0 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4987fff700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4970003bc0 0x7f4970003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f498cd2d700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970004d90 0x7f4970005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f49877fe700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 0x7f4970004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4987fff700 1 -- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 msgr2=0x7f4970004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4987fff700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49881658c0 0x7f4970004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4987fff700 1 -- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970004d90 msgr2=0x7f4970005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4987fff700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970004d90 0x7f4970005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4987fff700 1 -- 172.21.15.153:0/2924116483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700035b0 con 0x7f4970003bc0 2023-11-28T13:00:58.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4987fff700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4970003bc0 0x7f4970003fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4978001980 tx=0x7f4970005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239d0fe62a89f250 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978001470 con 0x7f4970003bc0 2023-11-28T13:00:58.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49780052d0 con 0x7f4970003bc0 2023-11-28T13:00:58.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.182+0000 7f4986ffd700 1 -- 172.21.15.153:0/2924116483 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978003680 con 0x7f4970003bc0 2023-11-28T13:00:58.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.183+0000 7f498e790700 1 -- 172.21.15.153:0/2924116483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4970003bc0 msgr2=0x7f4970003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.183+0000 7f498e790700 1 --2- 172.21.15.153:0/2924116483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4970003bc0 0x7f4970003fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4978001980 tx=0x7f4970005880 comp rx=0 tx=0).stop 2023-11-28T13:00:58.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.183+0000 7f498cd2d700 1 -- 172.21.15.153:0/2924116483 reap_dead start 2023-11-28T13:00:58.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.183+0000 7f498e790700 1 -- 172.21.15.153:0/2924116483 shutdown_connections 2023-11-28T13:00:58.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.183+0000 7f498e790700 1 -- 172.21.15.153:0/2924116483 >> 172.21.15.153:0/2924116483 conn(0x7f49880d0000 msgr2=0x7f49880d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.183+0000 7f498e790700 1 -- 172.21.15.153:0/2924116483 shutdown_connections 2023-11-28T13:00:58.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.183+0000 7f498e790700 1 -- 172.21.15.153:0/2924116483 wait complete. 2023-11-28T13:00:58.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.184+0000 7f498e790700 1 Processor -- start 2023-11-28T13:00:58.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.184+0000 7f498e790700 1 -- start start 2023-11-28T13:00:58.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f498e790700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970003bc0 0x7f4988159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f498e790700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f498814e300 0x7f49880ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f498e790700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498814dac0 0x7f498815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f498e790700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49881561d0 con 0x7f498814e300 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f498e790700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4988158d20 con 0x7f4970003bc0 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f498e790700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498814e8c0 con 0x7f498814dac0 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f498814e300 0x7f49880ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f498814e300 0x7f49880ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39670/0 (socket says 172.21.15.153:39670) 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 -- 172.21.15.153:0/2112662517 learned_addr learned my addr 172.21.15.153:0/2112662517 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f4987fff700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970003bc0 0x7f4988159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f498cd2d700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498814dac0 0x7f498815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 -- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498814dac0 msgr2=0x7f498815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498814dac0 0x7f498815e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 -- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970003bc0 msgr2=0x7f4988159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970003bc0 0x7f4988159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 -- 172.21.15.153:0/2112662517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498815ccd0 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f49877fe700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f498814e300 0x7f49880ae860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f497c007340 tx=0x7f497c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.185+0000 7f4984ff9700 1 -- 172.21.15.153:0/2112662517 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00f660 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.186+0000 7f4984ff9700 1 -- 172.21.15.153:0/2112662517 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c00f7e0 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.186+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498815d900 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.186+0000 7f4984ff9700 1 -- 172.21.15.153:0/2112662517 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00fb90 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.186+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498806a270 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.186+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49880c3690 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.187+0000 7f4984ff9700 1 -- 172.21.15.153:0/2112662517 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f497c00fcf0 con 0x7f498814e300 2023-11-28T13:00:58.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.187+0000 7f4984ff9700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49740682f0 0x7f497406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.187+0000 7f4984ff9700 1 -- 172.21.15.153:0/2112662517 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f497c085c70 con 0x7f498814e300 2023-11-28T13:00:58.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.187+0000 7f4987fff700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49740682f0 0x7f497406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.188+0000 7f4987fff700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49740682f0 0x7f497406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4978000c00 tx=0x7f497800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.191+0000 7f4984ff9700 1 -- 172.21.15.153:0/2112662517 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f497c052580 con 0x7f498814e300 2023-11-28T13:00:58.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.415+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4988160010 con 0x7f498814e300 2023-11-28T13:00:58.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.416+0000 7f4984ff9700 1 -- 172.21.15.153:0/2112662517 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f497c055ba0 con 0x7f498814e300 2023-11-28T13:00:58.416 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:58.417 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2499459281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2499459281},{"type":"v1","addr":"172.21.15.177:6834","nonce":2499459281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2708083457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2708083457},{"type":"v1","addr":"172.21.15.153:6837","nonce":2708083457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2730194389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2730194389},{"type":"v1","addr":"172.21.15.153:6836","nonce":2730194389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:17.624622+0000","modified":"2023-11-28T13:00:20.642491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/923823631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":923823631},{"type":"v1","addr":"172.21.15.177:6835","nonce":923823631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49740682f0 msgr2=0x7f497406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49740682f0 0x7f497406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4978000c00 tx=0x7f497800a000 comp rx=0 tx=0).stop 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f498814e300 msgr2=0x7f49880ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f498814e300 0x7f49880ae860 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f497c007340 tx=0x7f497c007650 comp rx=0 tx=0).stop 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 shutdown_connections 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f498814dac0 0x7f498815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4970003bc0 0x7f4988159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49740682f0 0x7f497406a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 --2- 172.21.15.153:0/2112662517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f498814e300 0x7f49880ae860 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 >> 172.21.15.153:0/2112662517 conn(0x7f49880d0000 msgr2=0x7f498806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 shutdown_connections 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.418+0000 7f498e790700 1 -- 172.21.15.153:0/2112662517 wait complete. 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 8 2023-11-28T13:00:58.431 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.550+0000 7fba4c6f3700 1 Processor -- start 2023-11-28T13:00:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.550+0000 7fba4c6f3700 1 -- start start 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.551+0000 7fba4c6f3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fba44071040 0x7fba44071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.551+0000 7fba4c6f3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba4406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.551+0000 7fba4c6f3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba4406cdc0 0x7fba4406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.551+0000 7fba4c6f3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba44071950 con 0x7fba4406cdc0 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.551+0000 7fba4c6f3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4415d060 con 0x7fba44071040 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.551+0000 7fba4c6f3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4415d340 con 0x7fba4406dee0 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba4406cdc0 0x7fba4406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fba4406cdc0 0x7fba4406d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39680/0 (socket says 172.21.15.153:39680) 2023-11-28T13:00:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 -- 172.21.15.153:0/426764443 learned_addr learned my addr 172.21.15.153:0/426764443 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4a48f700 1 --2- 172.21.15.153:0/426764443 >> v2:172.21.15.177:3300/0 conn(0x7fba44071040 0x7fba44071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba49c8e700 1 --2- 172.21.15.153:0/426764443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba4406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 -- 172.21.15.153:0/426764443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 msgr2=0x7fba4406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 --2- 172.21.15.153:0/426764443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba4406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 -- 172.21.15.153:0/426764443 >> v2:172.21.15.177:3300/0 conn(0x7fba44071040 msgr2=0x7fba44071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 --2- 172.21.15.153:0/426764443 >> v2:172.21.15.177:3300/0 conn(0x7fba44071040 0x7fba44071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 -- 172.21.15.153:0/426764443 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4415d620 con 0x7fba4406cdc0 2023-11-28T13:00:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.552+0000 7fba4ac90700 1 --2- 172.21.15.153:0/426764443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba4406d240 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fba440d0f50 tx=0x7fba3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a97840b9860eae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.553+0000 7fba4948d700 1 -- 172.21.15.153:0/426764443 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00e070 con 0x7fba4406cdc0 2023-11-28T13:00:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.553+0000 7fba4948d700 1 -- 172.21.15.153:0/426764443 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c00b910 con 0x7fba4406cdc0 2023-11-28T13:00:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.553+0000 7fba4948d700 1 -- 172.21.15.153:0/426764443 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00bcc0 con 0x7fba4406cdc0 2023-11-28T13:00:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.554+0000 7fba4c6f3700 1 -- 172.21.15.153:0/426764443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 msgr2=0x7fba4406d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.554+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/426764443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba4406d240 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fba440d0f50 tx=0x7fba3c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:58.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.555+0000 7fba4c6f3700 1 -- 172.21.15.153:0/426764443 shutdown_connections 2023-11-28T13:00:58.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.555+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/426764443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba4406d240 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.555+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/426764443 >> v2:172.21.15.177:3300/0 conn(0x7fba44071040 0x7fba44071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.555+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/426764443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba4406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.555+0000 7fba4c6f3700 1 -- 172.21.15.153:0/426764443 >> 172.21.15.153:0/426764443 conn(0x7fba440d0000 msgr2=0x7fba440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.555+0000 7fba4c6f3700 1 -- 172.21.15.153:0/426764443 shutdown_connections 2023-11-28T13:00:58.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.555+0000 7fba4c6f3700 1 -- 172.21.15.153:0/426764443 wait complete. 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.557+0000 7fba4c6f3700 1 Processor -- start 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.557+0000 7fba4c6f3700 1 -- start start 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4c6f3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba441f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4c6f3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba440c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4c6f3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba44071040 0x7fba440c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4c6f3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba44158a80 con 0x7fba4406cdc0 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4c6f3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba441f72d0 con 0x7fba44071040 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4c6f3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba440c4f80 con 0x7fba4406dee0 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4a48f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba441f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4a48f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba441f6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39682/0 (socket says 172.21.15.153:39682) 2023-11-28T13:00:58.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4a48f700 1 -- 172.21.15.153:0/1667090095 learned_addr learned my addr 172.21.15.153:0/1667090095 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:58.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba49c8e700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba440c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.558+0000 7fba4ac90700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba44071040 0x7fba440c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4a48f700 1 -- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 msgr2=0x7fba440c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4a48f700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba440c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4a48f700 1 -- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba44071040 msgr2=0x7fba440c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4a48f700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba44071040 0x7fba440c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4a48f700 1 -- 172.21.15.153:0/1667090095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c009b30 con 0x7fba4406cdc0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4a48f700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba441f6d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba4000e8d0 tx=0x7fba4000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba3affd700 1 -- 172.21.15.153:0/1667090095 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40009590 con 0x7fba4406cdc0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba3affd700 1 -- 172.21.15.153:0/1667090095 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba40013680 con 0x7fba4406cdc0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4406e870 con 0x7fba4406cdc0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba3affd700 1 -- 172.21.15.153:0/1667090095 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400139f0 con 0x7fba4406cdc0 2023-11-28T13:00:58.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.559+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4415d900 con 0x7fba4406cdc0 2023-11-28T13:00:58.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.560+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4415ccd0 con 0x7fba4406cdc0 2023-11-28T13:00:58.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.561+0000 7fba3affd700 1 -- 172.21.15.153:0/1667090095 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fba40019030 con 0x7fba4406cdc0 2023-11-28T13:00:58.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.561+0000 7fba3affd700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba280682f0 0x7fba2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.561+0000 7fba3affd700 1 -- 172.21.15.153:0/1667090095 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fba40085a70 con 0x7fba4406cdc0 2023-11-28T13:00:58.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.561+0000 7fba49c8e700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba280682f0 0x7fba2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.561+0000 7fba49c8e700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba280682f0 0x7fba2806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba4409e060 tx=0x7fba34009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.565+0000 7fba3affd700 1 -- 172.21.15.153:0/1667090095 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fba40052290 con 0x7fba4406cdc0 2023-11-28T13:00:58.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.749+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba440720a0 con 0x7fba4406cdc0 2023-11-28T13:00:58.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.750+0000 7fba3affd700 1 -- 172.21.15.153:0/1667090095 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18051 (secure 0 0 0) 0x7fba400558b0 con 0x7fba4406cdc0 2023-11-28T13:00:58.751 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:00:58.751 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":26,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:58.550244+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:58.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba280682f0 msgr2=0x7fba2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba280682f0 0x7fba2806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba4409e060 tx=0x7fba34009210 comp rx=0 tx=0).stop 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 msgr2=0x7fba441f6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba441f6d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba4000e8d0 tx=0x7fba4000ebe0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 shutdown_connections 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fba280682f0 0x7fba2806a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fba4406cdc0 0x7fba441f6d90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba44071040 0x7fba440c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 --2- 172.21.15.153:0/1667090095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba4406dee0 0x7fba440c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.752+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 >> 172.21.15.153:0/1667090095 conn(0x7fba440d0000 msgr2=0x7fba440cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.753+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 shutdown_connections 2023-11-28T13:00:58.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.753+0000 7fba4c6f3700 1 -- 172.21.15.153:0/1667090095 wait complete. 2023-11-28T13:00:58.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:15.280771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:16.578107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:58.766 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:58.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.889+0000 7f91753ce700 1 Processor -- start 2023-11-28T13:00:58.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.889+0000 7f91753ce700 1 -- start start 2023-11-28T13:00:58.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f91753ce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917006ba90 0x7f91700cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f91753ce700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f91700ccc40 0x7f91700cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f91753ce700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91700cf5e0 con 0x7f917015ca50 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f91753ce700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91700cd470 con 0x7f91700ccc40 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f91753ce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91700cecb0 con 0x7f917006ba90 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f916e7fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917006ba90 0x7f91700cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f916f7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f91700ccc40 0x7f91700cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f916effd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f917015ca50 0x7f917006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:34468/0 (socket says 172.21.15.153:34468) 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f916effd700 1 -- 172.21.15.153:0/2890630584 learned_addr learned my addr 172.21.15.153:0/2890630584 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f916dffb700 1 -- 172.21.15.153:0/2890630584 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260170874 0 0) 0x7f91700cf5e0 con 0x7f917015ca50 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.890+0000 7f916dffb700 1 -- 172.21.15.153:0/2890630584 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91580035b0 con 0x7f917015ca50 2023-11-28T13:00:58.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916e7fc700 1 -- 172.21.15.153:0/2890630584 >> v2:172.21.15.177:3300/0 conn(0x7f91700ccc40 msgr2=0x7f91700cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916e7fc700 1 --2- 172.21.15.153:0/2890630584 >> v2:172.21.15.177:3300/0 conn(0x7f91700ccc40 0x7f91700cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916e7fc700 1 -- 172.21.15.153:0/2890630584 >> v1:172.21.15.177:6789/0 conn(0x7f917015ca50 legacy=0x7f917006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916e7fc700 1 -- 172.21.15.153:0/2890630584 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700cef90 con 0x7f917006ba90 2023-11-28T13:00:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916e7fc700 1 --2- 172.21.15.153:0/2890630584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917006ba90 0x7f91700cc700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9160009840 tx=0x7f9160009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400a3ea8c1079e7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916dffb700 1 -- 172.21.15.153:0/2890630584 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011830 con 0x7f917006ba90 2023-11-28T13:00:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916dffb700 1 -- 172.21.15.153:0/2890630584 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9160011990 con 0x7f917006ba90 2023-11-28T13:00:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f916dffb700 1 -- 172.21.15.153:0/2890630584 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011d20 con 0x7f917006ba90 2023-11-28T13:00:58.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f91753ce700 1 -- 172.21.15.153:0/2890630584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917006ba90 msgr2=0x7f91700cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f91753ce700 1 --2- 172.21.15.153:0/2890630584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917006ba90 0x7f91700cc700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9160009840 tx=0x7f9160009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:58.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f91753ce700 1 -- 172.21.15.153:0/2890630584 shutdown_connections 2023-11-28T13:00:58.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f91753ce700 1 --2- 172.21.15.153:0/2890630584 >> v2:172.21.15.177:3300/0 conn(0x7f91700ccc40 0x7f91700cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f91753ce700 1 --2- 172.21.15.153:0/2890630584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917006ba90 0x7f91700cc700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.891+0000 7f91753ce700 1 -- 172.21.15.153:0/2890630584 >> 172.21.15.153:0/2890630584 conn(0x7f917009a230 msgr2=0x7f917008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.892+0000 7f91753ce700 1 -- 172.21.15.153:0/2890630584 shutdown_connections 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.892+0000 7f91753ce700 1 -- 172.21.15.153:0/2890630584 wait complete. 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.893+0000 7f91753ce700 1 Processor -- start 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.893+0000 7f91753ce700 1 -- start start 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91753ce700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f917006ba90 0x7f91701f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91753ce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91700ccc40 0x7f91700af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91753ce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917015ca50 0x7f91700b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91753ce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91700b1d30 con 0x7f91700ccc40 2023-11-28T13:00:58.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91753ce700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9170159dd0 con 0x7f917006ba90 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91753ce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917015a0b0 con 0x7f917015ca50 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91700ccc40 0x7f91700af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91700ccc40 0x7f91700af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39698/0 (socket says 172.21.15.153:39698) 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 -- 172.21.15.153:0/3180208115 learned_addr learned my addr 172.21.15.153:0/3180208115 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916effd700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f917006ba90 0x7f91701f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916f7fe700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917015ca50 0x7f91700b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 -- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917015ca50 msgr2=0x7f91700b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917015ca50 0x7f91700b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 -- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f917006ba90 msgr2=0x7f91701f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f917006ba90 0x7f91701f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 -- 172.21.15.153:0/3180208115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91600094a0 con 0x7f91700ccc40 2023-11-28T13:00:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f916e7fc700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91700ccc40 0x7f91700af680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91600090b0 tx=0x7f9160004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91577fe700 1 -- 172.21.15.153:0/3180208115 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91600036a0 con 0x7f91700ccc40 2023-11-28T13:00:58.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91577fe700 1 -- 172.21.15.153:0/3180208115 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916001f070 con 0x7f91700ccc40 2023-11-28T13:00:58.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91700cd0e0 con 0x7f91700ccc40 2023-11-28T13:00:58.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.894+0000 7f91577fe700 1 -- 172.21.15.153:0/3180208115 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160003a40 con 0x7f91700ccc40 2023-11-28T13:00:58.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.895+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91700cf270 con 0x7f91700ccc40 2023-11-28T13:00:58.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.896+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917015ce20 con 0x7f91700ccc40 2023-11-28T13:00:58.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.896+0000 7f91577fe700 1 -- 172.21.15.153:0/3180208115 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f916000f050 con 0x7f91700ccc40 2023-11-28T13:00:58.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.896+0000 7f91577fe700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f915c0682f0 0x7f915c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.896+0000 7f91577fe700 1 -- 172.21.15.153:0/3180208115 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9160021070 con 0x7f91700ccc40 2023-11-28T13:00:58.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.896+0000 7f916effd700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f915c0682f0 0x7f915c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.897+0000 7f916effd700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f915c0682f0 0x7f915c06a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9164007fd0 tx=0x7f9164007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:58.900+0000 7f91577fe700 1 -- 172.21.15.153:0/3180208115 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9160053c10 con 0x7f91700ccc40 2023-11-28T13:00:59.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.082+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91700745c0 con 0x7f91700ccc40 2023-11-28T13:00:59.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.082+0000 7f91577fe700 1 -- 172.21.15.153:0/3180208115 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9160057230 con 0x7f91700ccc40 2023-11-28T13:00:59.083 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:00:59.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f915c0682f0 msgr2=0x7f915c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f915c0682f0 0x7f915c06a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9164007fd0 tx=0x7f9164007dc0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91700ccc40 msgr2=0x7f91700af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91700ccc40 0x7f91700af680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91600090b0 tx=0x7f9160004750 comp rx=0 tx=0).stop 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 shutdown_connections 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f917015ca50 0x7f91700b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f917006ba90 0x7f91701f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f915c0682f0 0x7f915c06a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 --2- 172.21.15.153:0/3180208115 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91700ccc40 0x7f91700af680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.084+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 >> 172.21.15.153:0/3180208115 conn(0x7f917009a230 msgr2=0x7f91700d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.087 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.085+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 shutdown_connections 2023-11-28T13:00:59.087 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.085+0000 7f91753ce700 1 -- 172.21.15.153:0/3180208115 wait complete. 2023-11-28T13:00:59.097 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:59.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.220+0000 7f1ead392700 1 Processor -- start 2023-11-28T13:00:59.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.220+0000 7f1ead392700 1 -- start start 2023-11-28T13:00:59.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ead392700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1ea80cd7c0 0x7f1ea80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ead392700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea80d1950 con 0x7f1ea8161340 2023-11-28T13:00:59.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ead392700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea80ce140 con 0x7f1ea815c9b0 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ead392700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea80b5800 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ea67fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1ea8161340 0x7f1ea80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57872/0 (socket says 172.21.15.153:57872) 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ea77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1ea80cd7c0 0x7f1ea80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ea67fc700 1 -- 172.21.15.153:0/1818468961 learned_addr learned my addr 172.21.15.153:0/1818468961 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ea77fe700 1 -- 172.21.15.153:0/1818468961 >> v1:172.21.15.177:6789/0 conn(0x7f1ea815c9b0 legacy=0x7f1ea80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ea77fe700 1 -- 172.21.15.153:0/1818468961 >> v1:172.21.15.153:6789/0 conn(0x7f1ea8161340 legacy=0x7f1ea80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.221+0000 7f1ea77fe700 1 -- 172.21.15.153:0/1818468961 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea80d24f0 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ea77fe700 1 --2- 172.21.15.153:0/1818468961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80cdc00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ffdde7c8875c8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ea5ffb700 1 -- 172.21.15.153:0/1818468961 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000e070 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ea5ffb700 1 -- 172.21.15.153:0/1818468961 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea000bbd0 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ea5ffb700 1 -- 172.21.15.153:0/1818468961 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0012430 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ead392700 1 -- 172.21.15.153:0/1818468961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 msgr2=0x7f1ea80cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ead392700 1 --2- 172.21.15.153:0/1818468961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80cdc00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:59.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ead392700 1 -- 172.21.15.153:0/1818468961 shutdown_connections 2023-11-28T13:00:59.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ead392700 1 --2- 172.21.15.153:0/1818468961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80cdc00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ead392700 1 -- 172.21.15.153:0/1818468961 >> 172.21.15.153:0/1818468961 conn(0x7f1ea8072600 msgr2=0x7f1ea80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ead392700 1 -- 172.21.15.153:0/1818468961 shutdown_connections 2023-11-28T13:00:59.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.222+0000 7f1ead392700 1 -- 172.21.15.153:0/1818468961 wait complete. 2023-11-28T13:00:59.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.224+0000 7f1ead392700 1 Processor -- start 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.224+0000 7f1ead392700 1 -- start start 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ead392700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ead392700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ea815c9b0 0x7f1ea80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ead392700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ea8161340 0x7f1ea814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ead392700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80cb320 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ead392700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80c7360 con 0x7f1ea8161340 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ead392700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80c7640 con 0x7f1ea815c9b0 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39712/0 (socket says 172.21.15.153:39712) 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 -- 172.21.15.153:0/1812460054 learned_addr learned my addr 172.21.15.153:0/1812460054 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 -- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ea815c9b0 msgr2=0x7f1ea80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea77fe700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ea8161340 0x7f1ea814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea67fc700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ea815c9b0 0x7f1ea80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ea815c9b0 0x7f1ea80cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 -- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ea8161340 msgr2=0x7f1ea814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ea8161340 0x7f1ea814a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.225+0000 7f1ea6ffd700 1 -- 172.21.15.153:0/1812460054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0009a70 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.226+0000 7f1ea6ffd700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80c6ca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c4e0 tx=0x7f1e9c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.226+0000 7f1e8f7fe700 1 -- 172.21.15.153:0/1812460054 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0052d0 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.226+0000 7f1e8f7fe700 1 -- 172.21.15.153:0/1812460054 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c005430 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.226+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea80d1300 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.226+0000 7f1e8f7fe700 1 -- 172.21.15.153:0/1812460054 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0038c0 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.226+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea80b5520 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.227+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea80d15e0 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.227+0000 7f1e8f7fe700 1 -- 172.21.15.153:0/1812460054 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1e9c019030 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.228+0000 7f1e8f7fe700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e940682f0 0x7f1e9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.228+0000 7f1e8f7fe700 1 -- 172.21.15.153:0/1812460054 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1e9c085030 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.228+0000 7f1ea67fc700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e940682f0 0x7f1e9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.228+0000 7f1ea67fc700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e940682f0 0x7f1e9406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8160f50 tx=0x7f1e98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.232+0000 7f1e8f7fe700 1 -- 172.21.15.153:0/1812460054 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1e9c051850 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.413+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ea80745c0 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.414+0000 7f1e8f7fe700 1 -- 172.21.15.153:0/1812460054 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e9c054e70 con 0x7f1ea80cd7c0 2023-11-28T13:00:59.415 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:00:59.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e940682f0 msgr2=0x7f1e9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e940682f0 0x7f1e9406a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8160f50 tx=0x7f1e98009040 comp rx=0 tx=0).stop 2023-11-28T13:00:59.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 msgr2=0x7f1ea80c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80c6ca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c4e0 tx=0x7f1e9c00c8a0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 shutdown_connections 2023-11-28T13:00:59.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e940682f0 0x7f1e9406a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ea80cd7c0 0x7f1ea80c6ca0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ea8161340 0x7f1ea814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 --2- 172.21.15.153:0/1812460054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ea815c9b0 0x7f1ea80cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 >> 172.21.15.153:0/1812460054 conn(0x7f1ea8072600 msgr2=0x7f1ea815f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 shutdown_connections 2023-11-28T13:00:59.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.416+0000 7f1ead392700 1 -- 172.21.15.153:0/1812460054 wait complete. 2023-11-28T13:00:59.429 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:59.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.555+0000 7f8adcb0b700 1 Processor -- start 2023-11-28T13:00:59.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.555+0000 7f8adcb0b700 1 -- start start 2023-11-28T13:00:59.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8adcb0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad806ba90 0x7f8ad80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8adcb0b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8ad80cbdf0 0x7f8ad80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8adcb0b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad80cc5f0 con 0x7f8ad815cef0 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8adcb0b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad815d470 con 0x7f8ad80cbdf0 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8adcb0b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad815d750 con 0x7f8ad806ba90 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8ad6d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8ad80cbdf0 0x7f8ad80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8ad6d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8ad80cbdf0 0x7f8ad80ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39724/0 (socket says 172.21.15.153:39724) 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8ad659c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8ad815cef0 0x7f8ad806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57884/0 (socket says 172.21.15.153:57884) 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8ad6d9d700 1 -- 172.21.15.153:0/153055969 learned_addr learned my addr 172.21.15.153:0/153055969 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.556+0000 7f8ad5d9b700 1 --2- 172.21.15.153:0/153055969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad806ba90 0x7f8ad80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.557+0000 7f8ad6d9d700 1 -- 172.21.15.153:0/153055969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad806ba90 msgr2=0x7f8ad80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:59.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.557+0000 7f8ad6d9d700 1 --2- 172.21.15.153:0/153055969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad806ba90 0x7f8ad80af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.557+0000 7f8ad6d9d700 1 -- 172.21.15.153:0/153055969 >> v1:172.21.15.153:6789/0 conn(0x7f8ad815cef0 legacy=0x7f8ad806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.557+0000 7f8ad6d9d700 1 -- 172.21.15.153:0/153055969 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad815da30 con 0x7f8ad80cbdf0 2023-11-28T13:00:59.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.557+0000 7f8ad559a700 1 -- 172.21.15.153:0/153055969 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527401670 0 0) 0x7f8ad80cc5f0 con 0x7f8ad815cef0 2023-11-28T13:00:59.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.557+0000 7f8ad6d9d700 1 --2- 172.21.15.153:0/153055969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad80cbdf0 0x7f8ad80ca650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000d150 tx=0x7f8ad000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85619c22011edc30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.559+0000 7f8ad559a700 1 -- 172.21.15.153:0/153055969 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0010070 con 0x7f8ad80cbdf0 2023-11-28T13:00:59.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.559+0000 7f8ad559a700 1 -- 172.21.15.153:0/153055969 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad000dd10 con 0x7f8ad80cbdf0 2023-11-28T13:00:59.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.559+0000 7f8ad559a700 1 -- 172.21.15.153:0/153055969 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000b310 con 0x7f8ad80cbdf0 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 -- 172.21.15.153:0/153055969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad80cbdf0 msgr2=0x7f8ad80ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/153055969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad80cbdf0 0x7f8ad80ca650 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000d150 tx=0x7f8ad000d460 comp rx=0 tx=0).stop 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 -- 172.21.15.153:0/153055969 shutdown_connections 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/153055969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad80cbdf0 0x7f8ad80ca650 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/153055969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad806ba90 0x7f8ad80af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 -- 172.21.15.153:0/153055969 >> 172.21.15.153:0/153055969 conn(0x7f8ad80d0000 msgr2=0x7f8ad80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 -- 172.21.15.153:0/153055969 shutdown_connections 2023-11-28T13:00:59.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.560+0000 7f8adcb0b700 1 -- 172.21.15.153:0/153055969 wait complete. 2023-11-28T13:00:59.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.561+0000 7f8adcb0b700 1 Processor -- start 2023-11-28T13:00:59.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.561+0000 7f8adcb0b700 1 -- start start 2023-11-28T13:00:59.563 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8adcb0b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad806ba90 0x7f8ad80c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8adcb0b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ad80cbdf0 0x7f8ad81edad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8adcb0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad815cef0 0x7f8ad80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8adcb0b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81ee010 con 0x7f8ad806ba90 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8adcb0b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80c7640 con 0x7f8ad80cbdf0 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8adcb0b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80c7920 con 0x7f8ad815cef0 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8ad659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad806ba90 0x7f8ad80c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8ad659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad806ba90 0x7f8ad80c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39726/0 (socket says 172.21.15.153:39726) 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8ad659c700 1 -- 172.21.15.153:0/4229834379 learned_addr learned my addr 172.21.15.153:0/4229834379 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8ad6d9d700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad815cef0 0x7f8ad80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.562+0000 7f8ad5d9b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ad80cbdf0 0x7f8ad81edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ad659c700 1 -- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad815cef0 msgr2=0x7f8ad80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ad659c700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad815cef0 0x7f8ad80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.564 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ad659c700 1 -- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ad80cbdf0 msgr2=0x7f8ad81edad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ad659c700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ad80cbdf0 0x7f8ad81edad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ad659c700 1 -- 172.21.15.153:0/4229834379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad000bd40 con 0x7f8ad806ba90 2023-11-28T13:00:59.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ad659c700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad806ba90 0x7f8ad80c9ee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c550 tx=0x7f8acc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ac6ffd700 1 -- 172.21.15.153:0/4229834379 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc004c80 con 0x7f8ad806ba90 2023-11-28T13:00:59.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ac6ffd700 1 -- 172.21.15.153:0/4229834379 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc004e00 con 0x7f8ad806ba90 2023-11-28T13:00:59.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad815dff0 con 0x7f8ad806ba90 2023-11-28T13:00:59.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.563+0000 7f8ac6ffd700 1 -- 172.21.15.153:0/4229834379 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc007310 con 0x7f8ad806ba90 2023-11-28T13:00:59.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.564+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad815dd10 con 0x7f8ad806ba90 2023-11-28T13:00:59.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.565+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad80cab90 con 0x7f8ad806ba90 2023-11-28T13:00:59.566 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.565+0000 7f8ac6ffd700 1 -- 172.21.15.153:0/4229834379 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8acc01a020 con 0x7f8ad806ba90 2023-11-28T13:00:59.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.566+0000 7f8ac6ffd700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ab80682f0 0x7f8ab806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.566+0000 7f8ac6ffd700 1 -- 172.21.15.153:0/4229834379 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8acc0856f0 con 0x7f8ad806ba90 2023-11-28T13:00:59.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.566+0000 7f8ad5d9b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ab80682f0 0x7f8ab806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.567 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.567+0000 7f8ad5d9b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ab80682f0 0x7f8ab806a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ad815c8e0 tx=0x7f8ac8009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.570+0000 7f8ac6ffd700 1 -- 172.21.15.153:0/4229834379 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8acc051f10 con 0x7f8ad806ba90 2023-11-28T13:00:59.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.753+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ad80d0e60 con 0x7f8ad806ba90 2023-11-28T13:00:59.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.753+0000 7f8ac6ffd700 1 -- 172.21.15.153:0/4229834379 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8acc055530 con 0x7f8ad806ba90 2023-11-28T13:00:59.754 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ab80682f0 msgr2=0x7f8ab806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ab80682f0 0x7f8ab806a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ad815c8e0 tx=0x7f8ac8009210 comp rx=0 tx=0).stop 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad806ba90 msgr2=0x7f8ad80c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad806ba90 0x7f8ad80c9ee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c550 tx=0x7f8acc00c860 comp rx=0 tx=0).stop 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 shutdown_connections 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8ab80682f0 0x7f8ab806a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ad806ba90 0x7f8ad80c9ee0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ad815cef0 0x7f8ad80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 --2- 172.21.15.153:0/4229834379 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ad80cbdf0 0x7f8ad81edad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 >> 172.21.15.153:0/4229834379 conn(0x7f8ad80d0000 msgr2=0x7f8ad806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 shutdown_connections 2023-11-28T13:00:59.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.756+0000 7f8adcb0b700 1 -- 172.21.15.153:0/4229834379 wait complete. 2023-11-28T13:00:59.769 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.893+0000 7ff6a544d700 1 Processor -- start 2023-11-28T13:00:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.893+0000 7ff6a544d700 1 -- start start 2023-11-28T13:00:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff6a544d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6a015c9b0 0x7ff6a00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff6a544d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a00cd7c0 0x7ff6a00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff6a544d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a00d1950 con 0x7ff6a0161340 2023-11-28T13:00:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff6a544d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a00b5800 con 0x7ff6a015c9b0 2023-11-28T13:00:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff6a544d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00ce140 con 0x7ff6a00cd7c0 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6a015c9b0 0x7ff6a00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff6a015c9b0 0x7ff6a00b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39734/0 (socket says 172.21.15.153:39734) 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69effd700 1 -- 172.21.15.153:0/1636470344 learned_addr learned my addr 172.21.15.153:0/1636470344 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69f7fe700 1 --2- 172.21.15.153:0/1636470344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a00cd7c0 0x7ff6a00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69effd700 1 -- 172.21.15.153:0/1636470344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a00cd7c0 msgr2=0x7ff6a00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69effd700 1 --2- 172.21.15.153:0/1636470344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a00cd7c0 0x7ff6a00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69effd700 1 -- 172.21.15.153:0/1636470344 >> v1:172.21.15.177:6789/0 conn(0x7ff6a0161340 legacy=0x7ff6a00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69effd700 1 -- 172.21.15.153:0/1636470344 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00d24f0 con 0x7ff6a015c9b0 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.894+0000 7ff69f7fe700 1 --2- 172.21.15.153:0/1636470344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a00cd7c0 0x7ff6a00cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff69effd700 1 --2- 172.21.15.153:0/1636470344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a015c9b0 0x7ff6a00b4bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6940098f0 tx=0x7ff694009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff357c6c244a423 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff69dffb700 1 -- 172.21.15.153:0/1636470344 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400b8a0 con 0x7ff6a015c9b0 2023-11-28T13:00:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff69dffb700 1 -- 172.21.15.153:0/1636470344 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69400ba00 con 0x7ff6a015c9b0 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff69dffb700 1 -- 172.21.15.153:0/1636470344 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69400bd90 con 0x7ff6a015c9b0 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 -- 172.21.15.153:0/1636470344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a015c9b0 msgr2=0x7ff6a00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 --2- 172.21.15.153:0/1636470344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a015c9b0 0x7ff6a00b4bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6940098f0 tx=0x7ff694009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 -- 172.21.15.153:0/1636470344 shutdown_connections 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 --2- 172.21.15.153:0/1636470344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a00cd7c0 0x7ff6a00cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 --2- 172.21.15.153:0/1636470344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a015c9b0 0x7ff6a00b4bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 -- 172.21.15.153:0/1636470344 >> 172.21.15.153:0/1636470344 conn(0x7ff6a0072600 msgr2=0x7ff6a00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 -- 172.21.15.153:0/1636470344 shutdown_connections 2023-11-28T13:00:59.899 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.895+0000 7ff6a544d700 1 -- 172.21.15.153:0/1636470344 wait complete. 2023-11-28T13:00:59.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.897+0000 7ff6a544d700 1 Processor -- start 2023-11-28T13:00:59.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.897+0000 7ff6a544d700 1 -- start start 2023-11-28T13:00:59.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff6a544d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a00cd7c0 0x7ff6a0153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff6a544d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a015c9b0 0x7ff6a00c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff6a544d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0161340 0x7ff6a015e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff6a544d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c8440 con 0x7ff6a0161340 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff6a544d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c8720 con 0x7ff6a00cd7c0 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff6a544d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00c8a00 con 0x7ff6a015c9b0 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0161340 0x7ff6a015e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0161340 0x7ff6a015e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39742/0 (socket says 172.21.15.153:39742) 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 -- 172.21.15.153:0/2766632560 learned_addr learned my addr 172.21.15.153:0/2766632560 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69effd700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a00cd7c0 0x7ff6a0153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69e7fc700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a015c9b0 0x7ff6a00c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 -- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a015c9b0 msgr2=0x7ff6a00c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a015c9b0 0x7ff6a00c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 -- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a00cd7c0 msgr2=0x7ff6a0153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a00cd7c0 0x7ff6a0153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.898+0000 7ff69f7fe700 1 -- 172.21.15.153:0/2766632560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff694009550 con 0x7ff6a0161340 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.899+0000 7ff69f7fe700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0161340 0x7ff6a015e5a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff69800cd50 tx=0x7ff69800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.899+0000 7ff6877fe700 1 -- 172.21.15.153:0/2766632560 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800cea0 con 0x7ff6a0161340 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.899+0000 7ff6877fe700 1 -- 172.21.15.153:0/2766632560 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800f480 con 0x7ff6a0161340 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.899+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a00d1300 con 0x7ff6a0161340 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.899+0000 7ff6877fe700 1 -- 172.21.15.153:0/2766632560 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800f830 con 0x7ff6a0161340 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.899+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a00b5520 con 0x7ff6a0161340 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.900+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a00d15e0 con 0x7ff6a0161340 2023-11-28T13:00:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.901+0000 7ff6877fe700 1 -- 172.21.15.153:0/2766632560 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff698009030 con 0x7ff6a0161340 2023-11-28T13:00:59.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.901+0000 7ff6877fe700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff68c0682f0 0x7ff68c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.901+0000 7ff6877fe700 1 -- 172.21.15.153:0/2766632560 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff6980867c0 con 0x7ff6a0161340 2023-11-28T13:00:59.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.901+0000 7ff69effd700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff68c0682f0 0x7ff68c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.902+0000 7ff69effd700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff68c0682f0 0x7ff68c06a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff694014040 tx=0x7ff694008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:00:59.905+0000 7ff6877fe700 1 -- 172.21.15.153:0/2766632560 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff698053080 con 0x7ff6a0161340 2023-11-28T13:01:00.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.085+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6a00745c0 con 0x7ff6a0161340 2023-11-28T13:01:00.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.085+0000 7ff6877fe700 1 -- 172.21.15.153:0/2766632560 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6980566a0 con 0x7ff6a0161340 2023-11-28T13:01:00.086 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff68c0682f0 msgr2=0x7ff68c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff68c0682f0 0x7ff68c06a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff694014040 tx=0x7ff694008030 comp rx=0 tx=0).stop 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0161340 msgr2=0x7ff6a015e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0161340 0x7ff6a015e5a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff69800cd50 tx=0x7ff69800a560 comp rx=0 tx=0).stop 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 shutdown_connections 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff68c0682f0 0x7ff68c06a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a0161340 0x7ff6a015e5a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a00cd7c0 0x7ff6a0153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 --2- 172.21.15.153:0/2766632560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a015c9b0 0x7ff6a00c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 >> 172.21.15.153:0/2766632560 conn(0x7ff6a0072600 msgr2=0x7ff6a00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 shutdown_connections 2023-11-28T13:01:00.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.088+0000 7ff6a544d700 1 -- 172.21.15.153:0/2766632560 wait complete. 2023-11-28T13:01:00.101 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.220+0000 7f65fd1b2700 1 Processor -- start 2023-11-28T13:01:00.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.220+0000 7f65fd1b2700 1 -- start start 2023-11-28T13:01:00.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.220+0000 7f65fd1b2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f65f8071040 0x7f65f8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65fd1b2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65fd1b2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f8071950 con 0x7f65f806cdc0 2023-11-28T13:01:00.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65fd1b2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f8161550 con 0x7f65f8071040 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65fd1b2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f815d060 con 0x7f65f806dee0 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f6d9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f65f8071040 0x7f65f8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f759e700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f65f806cdc0 0x7f65f806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:34526/0 (socket says 172.21.15.153:34526) 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f759e700 1 -- 172.21.15.153:0/3927974441 learned_addr learned my addr 172.21.15.153:0/3927974441 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f6d9d700 1 -- 172.21.15.153:0/3927974441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 msgr2=0x7f65f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f6d9d700 1 --2- 172.21.15.153:0/3927974441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f6d9d700 1 -- 172.21.15.153:0/3927974441 >> v1:172.21.15.177:6789/0 conn(0x7f65f806cdc0 legacy=0x7f65f806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f6d9d700 1 -- 172.21.15.153:0/3927974441 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f815d340 con 0x7f65f8071040 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.221+0000 7f65f6d9d700 1 --2- 172.21.15.153:0/3927974441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f8071040 0x7f65f8071410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009840 tx=0x7f65e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae682615b5b4a95 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65f5d9b700 1 -- 172.21.15.153:0/3927974441 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8011830 con 0x7f65f8071040 2023-11-28T13:01:00.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65f5d9b700 1 -- 172.21.15.153:0/3927974441 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e8011990 con 0x7f65f8071040 2023-11-28T13:01:00.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65f5d9b700 1 -- 172.21.15.153:0/3927974441 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8011d20 con 0x7f65f8071040 2023-11-28T13:01:00.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 -- 172.21.15.153:0/3927974441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f8071040 msgr2=0x7f65f8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/3927974441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f8071040 0x7f65f8071410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009840 tx=0x7f65e8009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 -- 172.21.15.153:0/3927974441 shutdown_connections 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/3927974441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f8071040 0x7f65f8071410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/3927974441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 -- 172.21.15.153:0/3927974441 >> 172.21.15.153:0/3927974441 conn(0x7f65f80d0000 msgr2=0x7f65f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 -- 172.21.15.153:0/3927974441 shutdown_connections 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.222+0000 7f65fd1b2700 1 -- 172.21.15.153:0/3927974441 wait complete. 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.224+0000 7f65fd1b2700 1 Processor -- start 2023-11-28T13:01:00.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.224+0000 7f65fd1b2700 1 -- start start 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.224+0000 7f65fd1b2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f806cdc0 0x7f65f815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.224+0000 7f65fd1b2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f80c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.224+0000 7f65fd1b2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f8071040 0x7f65f80c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.224+0000 7f65fd1b2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80b3d30 con 0x7f65f8071040 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.224+0000 7f65fd1b2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c6a40 con 0x7f65f806cdc0 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65fd1b2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80c6d20 con 0x7f65f806dee0 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f8071040 0x7f65f80c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f8071040 0x7f65f80c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39754/0 (socket says 172.21.15.153:39754) 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 -- 172.21.15.153:0/1571057597 learned_addr learned my addr 172.21.15.153:0/1571057597 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f6d9d700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f806cdc0 0x7f65f815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f659c700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f80c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 -- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 msgr2=0x7f65f80c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f80c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 -- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f806cdc0 msgr2=0x7f65f815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f806cdc0 0x7f65f815faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 -- 172.21.15.153:0/1571057597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e80094a0 con 0x7f65f8071040 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f6d9d700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f806cdc0 0x7f65f815faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65f759e700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f8071040 0x7f65f80c6200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f65f000ca80 tx=0x7f65f000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65df7fe700 1 -- 172.21.15.153:0/1571057597 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f000a9e0 con 0x7f65f8071040 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65df7fe700 1 -- 172.21.15.153:0/1571057597 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f000ab40 con 0x7f65f8071040 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f815d900 con 0x7f65f8071040 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65df7fe700 1 -- 172.21.15.153:0/1571057597 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0012400 con 0x7f65f8071040 2023-11-28T13:01:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.225+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f815d620 con 0x7f65f8071040 2023-11-28T13:01:00.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.226+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f806e870 con 0x7f65f8071040 2023-11-28T13:01:00.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.227+0000 7f65df7fe700 1 -- 172.21.15.153:0/1571057597 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f65f0019030 con 0x7f65f8071040 2023-11-28T13:01:00.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.227+0000 7f65df7fe700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65e40682f0 0x7f65e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.227+0000 7f65df7fe700 1 -- 172.21.15.153:0/1571057597 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f65f0085600 con 0x7f65f8071040 2023-11-28T13:01:00.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.227+0000 7f65f6d9d700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65e40682f0 0x7f65e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.228+0000 7f65f6d9d700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65e40682f0 0x7f65e406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009fd0 tx=0x7f65e8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.231+0000 7f65df7fe700 1 -- 172.21.15.153:0/1571057597 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f65f004b5e0 con 0x7f65f8071040 2023-11-28T13:01:00.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.454+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65f80b3a10 con 0x7f65f8071040 2023-11-28T13:01:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.455+0000 7f65df7fe700 1 -- 172.21.15.153:0/1571057597 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f65f0017090 con 0x7f65f8071040 2023-11-28T13:01:00.456 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:00.456 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2499459281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2499459281},{"type":"v1","addr":"172.21.15.177:6834","nonce":2499459281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2708083457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2708083457},{"type":"v1","addr":"172.21.15.153:6837","nonce":2708083457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2730194389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2730194389},{"type":"v1","addr":"172.21.15.153:6836","nonce":2730194389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:17.624622+0000","modified":"2023-11-28T13:00:20.642491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/923823631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":923823631},{"type":"v1","addr":"172.21.15.177:6835","nonce":923823631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:01:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65e40682f0 msgr2=0x7f65e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65e40682f0 0x7f65e406a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009fd0 tx=0x7f65e8008030 comp rx=0 tx=0).stop 2023-11-28T13:01:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f8071040 msgr2=0x7f65f80c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f8071040 0x7f65f80c6200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f65f000ca80 tx=0x7f65f000cd90 comp rx=0 tx=0).stop 2023-11-28T13:01:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 shutdown_connections 2023-11-28T13:01:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65e40682f0 0x7f65e406a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65f8071040 0x7f65f80c6200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65f806cdc0 0x7f65f815faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 --2- 172.21.15.153:0/1571057597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65f806dee0 0x7f65f80c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 >> 172.21.15.153:0/1571057597 conn(0x7f65f80d0000 msgr2=0x7f65f80cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 shutdown_connections 2023-11-28T13:01:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.457+0000 7f65fd1b2700 1 -- 172.21.15.153:0/1571057597 wait complete. 2023-11-28T13:01:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 8 2023-11-28T13:01:00.470 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:00.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.591+0000 7f192b9c6700 1 Processor -- start 2023-11-28T13:01:00.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.591+0000 7f192b9c6700 1 -- start start 2023-11-28T13:01:00.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f192b9c6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 0x7f19240af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f192b9c6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19240cbdf0 0x7f192406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f192b9c6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f192406b8c0 0x7f1924071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f192b9c6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192406bd40 con 0x7f19240cbdf0 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f192b9c6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240cc5f0 con 0x7f192406b8c0 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f192b9c6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192415d470 con 0x7f192415cef0 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f1928f61700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19240cbdf0 0x7f192406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f1928f61700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19240cbdf0 0x7f192406b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39758/0 (socket says 172.21.15.153:39758) 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.592+0000 7f1928f61700 1 -- 172.21.15.153:0/3957683288 learned_addr learned my addr 172.21.15.153:0/3957683288 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f1928f61700 1 -- 172.21.15.153:0/3957683288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 msgr2=0x7f19240af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f1928f61700 1 --2- 172.21.15.153:0/3957683288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 0x7f19240af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f1929f63700 1 --2- 172.21.15.153:0/3957683288 >> v2:172.21.15.177:3300/0 conn(0x7f192406b8c0 0x7f1924071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f1928f61700 1 -- 172.21.15.153:0/3957683288 >> v2:172.21.15.177:3300/0 conn(0x7f192406b8c0 msgr2=0x7f1924071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:00.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f1928f61700 1 --2- 172.21.15.153:0/3957683288 >> v2:172.21.15.177:3300/0 conn(0x7f192406b8c0 0x7f1924071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f1928f61700 1 -- 172.21.15.153:0/3957683288 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192415d750 con 0x7f19240cbdf0 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f1928f61700 1 --2- 172.21.15.153:0/3957683288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f192406b380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1924073a70 tx=0x7f1920009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a982a896dc44a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f191bfff700 1 -- 172.21.15.153:0/3957683288 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192000b870 con 0x7f19240cbdf0 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f191bfff700 1 -- 172.21.15.153:0/3957683288 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192000b9d0 con 0x7f19240cbdf0 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f191bfff700 1 -- 172.21.15.153:0/3957683288 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192000bd60 con 0x7f19240cbdf0 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f192b9c6700 1 -- 172.21.15.153:0/3957683288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 msgr2=0x7f192406b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3957683288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f192406b380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1924073a70 tx=0x7f1920009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f192b9c6700 1 -- 172.21.15.153:0/3957683288 shutdown_connections 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3957683288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 0x7f19240af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3957683288 >> v2:172.21.15.177:3300/0 conn(0x7f192406b8c0 0x7f1924071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.593+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3957683288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f192406b380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.594+0000 7f192b9c6700 1 -- 172.21.15.153:0/3957683288 >> 172.21.15.153:0/3957683288 conn(0x7f19240d0000 msgr2=0x7f19240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.594+0000 7f192b9c6700 1 -- 172.21.15.153:0/3957683288 shutdown_connections 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.594+0000 7f192b9c6700 1 -- 172.21.15.153:0/3957683288 wait complete. 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.595+0000 7f192b9c6700 1 Processor -- start 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.595+0000 7f192b9c6700 1 -- start start 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f192b9c6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192406b8c0 0x7f19240b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f192b9c6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f1924153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f192b9c6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 0x7f19241e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f192b9c6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b5d00 con 0x7f19240cbdf0 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f192b9c6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c9060 con 0x7f192406b8c0 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f192b9c6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c8450 con 0x7f192415cef0 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f1924153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f1924153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39766/0 (socket says 172.21.15.153:39766) 2023-11-28T13:01:00.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 -- 172.21.15.153:0/3572295965 learned_addr learned my addr 172.21.15.153:0/3572295965 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1929f63700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 0x7f19241e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1929762700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192406b8c0 0x7f19240b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 -- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 msgr2=0x7f19241e31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 0x7f19241e31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 -- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192406b8c0 msgr2=0x7f19240b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192406b8c0 0x7f19240b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.596+0000 7f1928f61700 1 -- 172.21.15.153:0/3572295965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1920009550 con 0x7f19240cbdf0 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.597+0000 7f1928f61700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f1924153a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f192406c7b0 tx=0x7f1920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.597+0000 7f1919ffb700 1 -- 172.21.15.153:0/3572295965 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920004310 con 0x7f19240cbdf0 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.597+0000 7f1919ffb700 1 -- 172.21.15.153:0/3572295965 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192001f070 con 0x7f19240cbdf0 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.597+0000 7f1919ffb700 1 -- 172.21.15.153:0/3572295965 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19200036a0 con 0x7f19240cbdf0 2023-11-28T13:01:00.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.597+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192415dd10 con 0x7f19240cbdf0 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.597+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192415da30 con 0x7f19240cbdf0 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.598+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192415dff0 con 0x7f19240cbdf0 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.598+0000 7f1919ffb700 1 -- 172.21.15.153:0/3572295965 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f192002a020 con 0x7f19240cbdf0 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.599+0000 7f1919ffb700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19080682f0 0x7f190806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.599+0000 7f1919ffb700 1 -- 172.21.15.153:0/3572295965 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f19200873e0 con 0x7f19240cbdf0 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.599+0000 7f1929762700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19080682f0 0x7f190806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.599+0000 7f1929762700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19080682f0 0x7f190806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1914006fd0 tx=0x7f1914008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.603+0000 7f1919ffb700 1 -- 172.21.15.153:0/3572295965 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1920053b10 con 0x7f19240cbdf0 2023-11-28T13:01:00.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.788+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19240c91a0 con 0x7f19240cbdf0 2023-11-28T13:01:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.790+0000 7f1919ffb700 1 -- 172.21.15.153:0/3572295965 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18051 (secure 0 0 0) 0x7f1920057130 con 0x7f19240cbdf0 2023-11-28T13:01:00.790 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:00.791 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":27,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:59.556817+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19080682f0 msgr2=0x7f190806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19080682f0 0x7f190806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1914006fd0 tx=0x7f1914008040 comp rx=0 tx=0).stop 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 msgr2=0x7f1924153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f1924153a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f192406c7b0 tx=0x7f1920004750 comp rx=0 tx=0).stop 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 shutdown_connections 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f192415cef0 0x7f19241e31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f192406b8c0 0x7f19240b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19080682f0 0x7f190806a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 --2- 172.21.15.153:0/3572295965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19240cbdf0 0x7f1924153a90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 >> 172.21.15.153:0/3572295965 conn(0x7f19240d0000 msgr2=0x7f192415add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 shutdown_connections 2023-11-28T13:01:00.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.792+0000 7f192b9c6700 1 -- 172.21.15.153:0/3572295965 wait complete. 2023-11-28T13:01:00.805 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:01:00.806 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:00.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.924+0000 7f0db0de2700 1 Processor -- start 2023-11-28T13:01:00.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.924+0000 7f0db0de2700 1 -- start start 2023-11-28T13:01:00.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0db0de2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0dac15c9b0 0x7f0dac0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0db0de2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0dac161340 0x7f0dac0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0db0de2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac0d1950 con 0x7f0dac0d2320 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0db0de2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dac0b5800 con 0x7f0dac161340 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0db0de2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dac0b5520 con 0x7f0dac15c9b0 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0da9d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0dac161340 0x7f0dac0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0da9d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0dac161340 0x7f0dac0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39772/0 (socket says 172.21.15.153:39772) 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0da9d9b700 1 -- 172.21.15.153:0/800549666 learned_addr learned my addr 172.21.15.153:0/800549666 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.925+0000 7f0daa59c700 1 --2- 172.21.15.153:0/800549666 >> v2:172.21.15.177:3300/0 conn(0x7f0dac15c9b0 0x7f0dac0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da9d9b700 1 -- 172.21.15.153:0/800549666 >> v2:172.21.15.177:3300/0 conn(0x7f0dac15c9b0 msgr2=0x7f0dac0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:00.928 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da9d9b700 1 --2- 172.21.15.153:0/800549666 >> v2:172.21.15.177:3300/0 conn(0x7f0dac15c9b0 0x7f0dac0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da9d9b700 1 -- 172.21.15.153:0/800549666 >> v1:172.21.15.177:6789/0 conn(0x7f0dac0d2320 legacy=0x7f0dac0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da9d9b700 1 -- 172.21.15.153:0/800549666 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac0d1300 con 0x7f0dac161340 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0daa59c700 1 --2- 172.21.15.153:0/800549666 >> v2:172.21.15.177:3300/0 conn(0x7f0dac15c9b0 0x7f0dac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da9d9b700 1 --2- 172.21.15.153:0/800549666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac161340 0x7f0dac0d1de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0dac067460 tx=0x7f0d9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95961b5f0d6abda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da959a700 1 -- 172.21.15.153:0/800549666 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00b870 con 0x7f0dac161340 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da959a700 1 -- 172.21.15.153:0/800549666 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c00b9d0 con 0x7f0dac161340 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0da959a700 1 -- 172.21.15.153:0/800549666 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00bd60 con 0x7f0dac161340 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0db0de2700 1 -- 172.21.15.153:0/800549666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac161340 msgr2=0x7f0dac0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.926+0000 7f0db0de2700 1 --2- 172.21.15.153:0/800549666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac161340 0x7f0dac0d1de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0dac067460 tx=0x7f0d9c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.927+0000 7f0db0de2700 1 -- 172.21.15.153:0/800549666 shutdown_connections 2023-11-28T13:01:00.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.927+0000 7f0db0de2700 1 --2- 172.21.15.153:0/800549666 >> v2:172.21.15.177:3300/0 conn(0x7f0dac15c9b0 0x7f0dac0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.927+0000 7f0db0de2700 1 --2- 172.21.15.153:0/800549666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac161340 0x7f0dac0d1de0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.927+0000 7f0db0de2700 1 -- 172.21.15.153:0/800549666 >> 172.21.15.153:0/800549666 conn(0x7f0dac072600 msgr2=0x7f0dac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.927+0000 7f0db0de2700 1 -- 172.21.15.153:0/800549666 shutdown_connections 2023-11-28T13:01:00.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.927+0000 7f0db0de2700 1 -- 172.21.15.153:0/800549666 wait complete. 2023-11-28T13:01:00.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.928+0000 7f0db0de2700 1 Processor -- start 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.928+0000 7f0db0de2700 1 -- start start 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac0d2320 0x7f0dac0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0dac15c9b0 0x7f0dac1f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dac161340 0x7f0dac152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac1f5d50 con 0x7f0dac0d2320 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0c8fe0 con 0x7f0dac161340 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac153060 con 0x7f0dac15c9b0 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac0d2320 0x7f0dac0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac0d2320 0x7f0dac0c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39778/0 (socket says 172.21.15.153:39778) 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 -- 172.21.15.153:0/1717655332 learned_addr learned my addr 172.21.15.153:0/1717655332 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daad9d700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dac161340 0x7f0dac152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0da9d9b700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0dac15c9b0 0x7f0dac1f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 -- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0dac15c9b0 msgr2=0x7f0dac1f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0dac15c9b0 0x7f0dac1f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 -- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dac161340 msgr2=0x7f0dac152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dac161340 0x7f0dac152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 -- 172.21.15.153:0/1717655332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c009550 con 0x7f0dac0d2320 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0da9d9b700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0dac15c9b0 0x7f0dac1f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:00.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0daa59c700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac0d2320 0x7f0dac0c8650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0da000e810 tx=0x7f0da000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0d9affd700 1 -- 172.21.15.153:0/1717655332 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0013610 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0d9affd700 1 -- 172.21.15.153:0/1717655332 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da0013770 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac0b5130 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0d9affd700 1 -- 172.21.15.153:0/1717655332 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0013b20 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.929+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac0d15e0 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.930+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac0b52c0 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.931+0000 7f0d9affd700 1 -- 172.21.15.153:0/1717655332 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0da0023070 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.931+0000 7f0d9affd700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0d8c0682f0 0x7f0d8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.931+0000 7f0da9d9b700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0d8c0682f0 0x7f0d8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.931+0000 7f0d9affd700 1 -- 172.21.15.153:0/1717655332 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0da00858e0 con 0x7f0dac0d2320 2023-11-28T13:01:00.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.932+0000 7f0da9d9b700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0d8c0682f0 0x7f0d8c06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c000c00 tx=0x7f0d9c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:00.936+0000 7f0d9affd700 1 -- 172.21.15.153:0/1717655332 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0da0052100 con 0x7f0dac0d2320 2023-11-28T13:01:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.158+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dac0745c0 con 0x7f0dac0d2320 2023-11-28T13:01:01.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.159+0000 7f0d9affd700 1 -- 172.21.15.153:0/1717655332 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7f0da0055720 con 0x7f0dac0d2320 2023-11-28T13:01:01.160 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:01.160 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2499459281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2499459281},{"type":"v1","addr":"172.21.15.177:6834","nonce":2499459281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2708083457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2708083457},{"type":"v1","addr":"172.21.15.153:6837","nonce":2708083457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2730194389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2730194389},{"type":"v1","addr":"172.21.15.153:6836","nonce":2730194389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:17.624622+0000","modified":"2023-11-28T13:00:20.642491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/923823631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":923823631},{"type":"v1","addr":"172.21.15.177:6835","nonce":923823631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0d8c0682f0 msgr2=0x7f0d8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0d8c0682f0 0x7f0d8c06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c000c00 tx=0x7f0d9c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac0d2320 msgr2=0x7f0dac0c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac0d2320 0x7f0dac0c8650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0da000e810 tx=0x7f0da000eb20 comp rx=0 tx=0).stop 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 shutdown_connections 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0d8c0682f0 0x7f0d8c06a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0dac0d2320 0x7f0dac0c8650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0dac161340 0x7f0dac152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 --2- 172.21.15.153:0/1717655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0dac15c9b0 0x7f0dac1f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 >> 172.21.15.153:0/1717655332 conn(0x7f0dac072600 msgr2=0x7f0dac0b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 shutdown_connections 2023-11-28T13:01:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.161+0000 7f0db0de2700 1 -- 172.21.15.153:0/1717655332 wait complete. 2023-11-28T13:01:01.163 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 8 2023-11-28T13:01:01.174 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:01.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.293+0000 7fa015e4a700 1 Processor -- start 2023-11-28T13:01:01.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.293+0000 7fa015e4a700 1 -- start start 2023-11-28T13:01:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.293+0000 7fa015e4a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa01006ba90 0x7fa0100af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa015e4a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0100cc5f0 con 0x7fa0100cbdf0 2023-11-28T13:01:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa01006ba90 0x7fa0100af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa015e4a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01015d470 con 0x7fa01015cef0 2023-11-28T13:01:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa0100cbdf0 0x7fa0100ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57954/0 (socket says 172.21.15.153:57954) 2023-11-28T13:01:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa015e4a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01015d750 con 0x7fa01006ba90 2023-11-28T13:01:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00ffff700 1 -- 172.21.15.153:0/3746246090 learned_addr learned my addr 172.21.15.153:0/3746246090 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00effd700 1 -- 172.21.15.153:0/3746246090 >> v1:172.21.15.177:6789/0 conn(0x7fa01015cef0 legacy=0x7fa01006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00effd700 1 -- 172.21.15.153:0/3746246090 >> v1:172.21.15.153:6789/0 conn(0x7fa0100cbdf0 legacy=0x7fa0100ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00effd700 1 -- 172.21.15.153:0/3746246090 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01015da30 con 0x7fa01006ba90 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00e7fc700 1 -- 172.21.15.153:0/3746246090 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193631193 0 0) 0x7fa0100cc5f0 con 0x7fa0100cbdf0 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00effd700 1 --2- 172.21.15.153:0/3746246090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa01006ba90 0x7fa0100af400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa004009840 tx=0x7fa004009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ba70da0da79bd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00e7fc700 1 -- 172.21.15.153:0/3746246090 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011830 con 0x7fa01006ba90 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00e7fc700 1 -- 172.21.15.153:0/3746246090 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004011990 con 0x7fa01006ba90 2023-11-28T13:01:01.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.294+0000 7fa00e7fc700 1 -- 172.21.15.153:0/3746246090 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011d20 con 0x7fa01006ba90 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.295+0000 7fa015e4a700 1 -- 172.21.15.153:0/3746246090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa01006ba90 msgr2=0x7fa0100af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.295+0000 7fa015e4a700 1 --2- 172.21.15.153:0/3746246090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa01006ba90 0x7fa0100af400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa004009840 tx=0x7fa004009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.295+0000 7fa015e4a700 1 -- 172.21.15.153:0/3746246090 shutdown_connections 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.295+0000 7fa015e4a700 1 --2- 172.21.15.153:0/3746246090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa01006ba90 0x7fa0100af400 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.295+0000 7fa015e4a700 1 -- 172.21.15.153:0/3746246090 >> 172.21.15.153:0/3746246090 conn(0x7fa0100d0000 msgr2=0x7fa0100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.295+0000 7fa015e4a700 1 -- 172.21.15.153:0/3746246090 shutdown_connections 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.295+0000 7fa015e4a700 1 -- 172.21.15.153:0/3746246090 wait complete. 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.296+0000 7fa015e4a700 1 Processor -- start 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.296+0000 7fa015e4a700 1 -- start start 2023-11-28T13:01:01.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa015e4a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa01006ba90 0x7fa0100c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa015e4a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0100cbdf0 0x7fa0101f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa015e4a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa01015cef0 0x7fa0100b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa015e4a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100b5d30 con 0x7fa0100cbdf0 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa015e4a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa010158e60 con 0x7fa01015cef0 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa015e4a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100ca090 con 0x7fa01006ba90 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0100cbdf0 0x7fa0101f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0100cbdf0 0x7fa0101f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39794/0 (socket says 172.21.15.153:39794) 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 -- 172.21.15.153:0/741145572 learned_addr learned my addr 172.21.15.153:0/741145572 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00ffff700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa01015cef0 0x7fa0100b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00f7fe700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa01006ba90 0x7fa0100c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 -- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa01006ba90 msgr2=0x7fa0100c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa01006ba90 0x7fa0100c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 -- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa01015cef0 msgr2=0x7fa0100b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa01015cef0 0x7fa0100b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 -- 172.21.15.153:0/741145572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040094a0 con 0x7fa0100cbdf0 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7fa00effd700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0100cbdf0 0x7fa0101f0e10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa004004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7f9ff7fff700 1 -- 172.21.15.153:0/741145572 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00401a900 con 0x7fa0100cbdf0 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7f9ff7fff700 1 -- 172.21.15.153:0/741145572 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00401aa60 con 0x7fa0100cbdf0 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.297+0000 7f9ff7fff700 1 -- 172.21.15.153:0/741145572 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00401add0 con 0x7fa0100cbdf0 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.298+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01015dff0 con 0x7fa0100cbdf0 2023-11-28T13:01:01.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.298+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01015dd10 con 0x7fa0100cbdf0 2023-11-28T13:01:01.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.299+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0100cad60 con 0x7fa0100cbdf0 2023-11-28T13:01:01.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.299+0000 7f9ff7fff700 1 -- 172.21.15.153:0/741145572 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa00402b030 con 0x7fa0100cbdf0 2023-11-28T13:01:01.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.299+0000 7f9ff7fff700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ffc068320 0x7f9ffc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.299+0000 7f9ff7fff700 1 -- 172.21.15.153:0/741145572 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa004087060 con 0x7fa0100cbdf0 2023-11-28T13:01:01.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.299+0000 7fa00f7fe700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ffc068320 0x7f9ffc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.300+0000 7fa00f7fe700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ffc068320 0x7f9ffc06a7d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa000000f60 tx=0x7fa000009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.303+0000 7f9ff7fff700 1 -- 172.21.15.153:0/741145572 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa004053930 con 0x7fa0100cbdf0 2023-11-28T13:01:01.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.526+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa01006a8d0 con 0x7fa0100cbdf0 2023-11-28T13:01:01.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.527+0000 7f9ff7fff700 1 -- 172.21.15.153:0/741145572 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4651 (secure 0 0 0) 0x7fa004015090 con 0x7fa0100cbdf0 2023-11-28T13:01:01.528 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2499459281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2499459281},{"type":"v1","addr":"172.21.15.177:6834","nonce":2499459281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/2708083457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":2708083457},{"type":"v1","addr":"172.21.15.153:6837","nonce":2708083457}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/2730194389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":2730194389},{"type":"v1","addr":"172.21.15.153:6836","nonce":2730194389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:17.624622+0000","modified":"2023-11-28T13:00:20.642491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/923823631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":923823631},{"type":"v1","addr":"172.21.15.177:6835","nonce":923823631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:01:01.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ffc068320 msgr2=0x7f9ffc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ffc068320 0x7f9ffc06a7d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa000000f60 tx=0x7fa000009040 comp rx=0 tx=0).stop 2023-11-28T13:01:01.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0100cbdf0 msgr2=0x7fa0101f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0100cbdf0 0x7fa0101f0e10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa004004750 comp rx=0 tx=0).stop 2023-11-28T13:01:01.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 shutdown_connections 2023-11-28T13:01:01.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa01015cef0 0x7fa0100b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa01006ba90 0x7fa0100c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ffc068320 0x7f9ffc06a7d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 --2- 172.21.15.153:0/741145572 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0100cbdf0 0x7fa0101f0e10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 >> 172.21.15.153:0/741145572 conn(0x7fa0100d0000 msgr2=0x7fa0100ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 shutdown_connections 2023-11-28T13:01:01.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.529+0000 7fa015e4a700 1 -- 172.21.15.153:0/741145572 wait complete. 2023-11-28T13:01:01.532 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 8 2023-11-28T13:01:01.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:01.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.664+0000 7f23b4c72700 1 Processor -- start 2023-11-28T13:01:01.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.664+0000 7f23b4c72700 1 -- start start 2023-11-28T13:01:01.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23b4c72700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159860 0x7f23b00c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23b4c72700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f23b00b0d80 0x7f23b00c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23b4c72700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b00ccc00 con 0x7f23b0159490 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23b4c72700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b00ccec0 con 0x7f23b00b0d80 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23b4c72700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00cd1a0 con 0x7f23b0159860 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23ae59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f23b0159490 0x7f23b00b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57964/0 (socket says 172.21.15.153:57964) 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23ae59c700 1 -- 172.21.15.153:0/3828052598 learned_addr learned my addr 172.21.15.153:0/3828052598 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23add9b700 1 --2- 172.21.15.153:0/3828052598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159860 0x7f23b00c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23aed9d700 1 --2- 172.21.15.153:0/3828052598 >> v2:172.21.15.177:3300/0 conn(0x7f23b00b0d80 0x7f23b00c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23ad59a700 1 -- 172.21.15.153:0/3828052598 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94263981 0 0) 0x7f23b00ccc00 con 0x7f23b0159490 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.665+0000 7f23ad59a700 1 -- 172.21.15.153:0/3828052598 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23980035b0 con 0x7f23b0159490 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23add9b700 1 -- 172.21.15.153:0/3828052598 >> v2:172.21.15.177:3300/0 conn(0x7f23b00b0d80 msgr2=0x7f23b00c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23add9b700 1 --2- 172.21.15.153:0/3828052598 >> v2:172.21.15.177:3300/0 conn(0x7f23b00b0d80 0x7f23b00c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23add9b700 1 -- 172.21.15.153:0/3828052598 >> v1:172.21.15.153:6789/0 conn(0x7f23b0159490 legacy=0x7f23b00b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23ad59a700 1 -- 172.21.15.153:0/3828052598 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3743064976 0 0) 0x7f23980035b0 con 0x7f23b0159490 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23add9b700 1 -- 172.21.15.153:0/3828052598 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b00cd480 con 0x7f23b0159860 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23aed9d700 1 --2- 172.21.15.153:0/3828052598 >> v2:172.21.15.177:3300/0 conn(0x7f23b00b0d80 0x7f23b00c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23add9b700 1 --2- 172.21.15.153:0/3828052598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159860 0x7f23b00c94c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009840 tx=0x7f23a0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e3df9e674e2a53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23ad59a700 1 -- 172.21.15.153:0/3828052598 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a0011830 con 0x7f23b0159860 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23ad59a700 1 -- 172.21.15.153:0/3828052598 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a0011990 con 0x7f23b0159860 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23ad59a700 1 -- 172.21.15.153:0/3828052598 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a0011d20 con 0x7f23b0159860 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23b4c72700 1 -- 172.21.15.153:0/3828052598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159860 msgr2=0x7f23b00c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.666+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3828052598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159860 0x7f23b00c94c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009840 tx=0x7f23a0009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.667+0000 7f23b4c72700 1 -- 172.21.15.153:0/3828052598 shutdown_connections 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.667+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3828052598 >> v2:172.21.15.177:3300/0 conn(0x7f23b00b0d80 0x7f23b00c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.667+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3828052598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159860 0x7f23b00c94c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.667+0000 7f23b4c72700 1 -- 172.21.15.153:0/3828052598 >> 172.21.15.153:0/3828052598 conn(0x7f23b00cb670 msgr2=0x7f23b00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.667+0000 7f23b4c72700 1 -- 172.21.15.153:0/3828052598 shutdown_connections 2023-11-28T13:01:01.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.667+0000 7f23b4c72700 1 -- 172.21.15.153:0/3828052598 wait complete. 2023-11-28T13:01:01.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.668+0000 7f23b4c72700 1 Processor -- start 2023-11-28T13:01:01.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.668+0000 7f23b4c72700 1 -- start start 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23b4c72700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23b00b0d80 0x7f23b01512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23b4c72700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159490 0x7f23b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23b4c72700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b01ed740 0x7f23b0071440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23b4c72700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0070bf0 con 0x7f23b00b0d80 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23b4c72700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0152570 con 0x7f23b01ed740 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23b4c72700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00c1a90 con 0x7f23b0159490 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23b00b0d80 0x7f23b01512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23b00b0d80 0x7f23b01512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39806/0 (socket says 172.21.15.153:39806) 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 -- 172.21.15.153:0/3583941308 learned_addr learned my addr 172.21.15.153:0/3583941308 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23add9b700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159490 0x7f23b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23aed9d700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b01ed740 0x7f23b0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 -- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159490 msgr2=0x7f23b00706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159490 0x7f23b00706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 -- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b01ed740 msgr2=0x7f23b0071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b01ed740 0x7f23b0071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 -- 172.21.15.153:0/3583941308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a8009a70 con 0x7f23b00b0d80 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23add9b700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159490 0x7f23b00706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.669+0000 7f23ae59c700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23b00b0d80 0x7f23b01512a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c780 tx=0x7f23a400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.670+0000 7f239effd700 1 -- 172.21.15.153:0/3583941308 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4005d70 con 0x7f23b00b0d80 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.670+0000 7f239effd700 1 -- 172.21.15.153:0/3583941308 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40070d0 con 0x7f23b00b0d80 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.670+0000 7f239effd700 1 -- 172.21.15.153:0/3583941308 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4007460 con 0x7f23b00b0d80 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.670+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a00094a0 con 0x7f23b00b0d80 2023-11-28T13:01:01.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.670+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b00c9f40 con 0x7f23b00b0d80 2023-11-28T13:01:01.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.671+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b00cd760 con 0x7f23b00b0d80 2023-11-28T13:01:01.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.671+0000 7f239effd700 1 -- 172.21.15.153:0/3583941308 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f23a4019030 con 0x7f23b00b0d80 2023-11-28T13:01:01.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.671+0000 7f239effd700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23900682f0 0x7f239006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.671+0000 7f239effd700 1 -- 172.21.15.153:0/3583941308 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f23a40851d0 con 0x7f23b00b0d80 2023-11-28T13:01:01.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.671+0000 7f23add9b700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23900682f0 0x7f239006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.672+0000 7f23add9b700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23900682f0 0x7f239006a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009fd0 tx=0x7f23a001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.676+0000 7f239effd700 1 -- 172.21.15.153:0/3583941308 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f23a40519f0 con 0x7f23b00b0d80 2023-11-28T13:01:01.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.859+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23b00cdc80 con 0x7f23b00b0d80 2023-11-28T13:01:01.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.860+0000 7f239effd700 1 -- 172.21.15.153:0/3583941308 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18051 (secure 0 0 0) 0x7f23a4055010 con 0x7f23b00b0d80 2023-11-28T13:01:01.861 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:01.861 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":27,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:00:59.556817+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:15.280771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:16.578107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23900682f0 msgr2=0x7f239006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23900682f0 0x7f239006a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009fd0 tx=0x7f23a001c040 comp rx=0 tx=0).stop 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23b00b0d80 msgr2=0x7f23b01512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23b00b0d80 0x7f23b01512a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c780 tx=0x7f23a400cb40 comp rx=0 tx=0).stop 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 shutdown_connections 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23900682f0 0x7f239006a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23b00b0d80 0x7f23b01512a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23b01ed740 0x7f23b0071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 --2- 172.21.15.153:0/3583941308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23b0159490 0x7f23b00706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 >> 172.21.15.153:0/3583941308 conn(0x7f23b00cb670 msgr2=0x7f23b00afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 shutdown_connections 2023-11-28T13:01:01.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.862+0000 7f23b4c72700 1 -- 172.21.15.153:0/3583941308 wait complete. 2023-11-28T13:01:01.876 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:01:01.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.996+0000 7f8858b05700 1 Processor -- start 2023-11-28T13:01:01.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.996+0000 7f8858b05700 1 -- start start 2023-11-28T13:01:02.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f8858b05700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88540cbdf0 0x7f88540af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f8858b05700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88540cc5f0 con 0x7f885406ba90 2023-11-28T13:01:02.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f8858b05700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885415d470 con 0x7f885415cef0 2023-11-28T13:01:02.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f8858b05700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885415d750 con 0x7f88540cbdf0 2023-11-28T13:01:02.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f8851d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f885406ba90 0x7f88540ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:57980/0 (socket says 172.21.15.153:57980) 2023-11-28T13:01:02.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f8851d9b700 1 -- 172.21.15.153:0/4188590073 learned_addr learned my addr 172.21.15.153:0/4188590073 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:02.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f8852d9d700 1 --2- 172.21.15.153:0/4188590073 >> v2:172.21.15.177:3300/0 conn(0x7f88540cbdf0 0x7f88540af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547555938 0 0) 0x7f88540cc5f0 con 0x7f885406ba90 2023-11-28T13:01:02.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.997+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f883c0035b0 con 0x7f885406ba90 2023-11-28T13:01:02.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855957660 0 0) 0x7f883c0035b0 con 0x7f885406ba90 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 >> v2:172.21.15.177:3300/0 conn(0x7f88540cbdf0 msgr2=0x7f88540af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 --2- 172.21.15.153:0/4188590073 >> v2:172.21.15.177:3300/0 conn(0x7f88540cbdf0 0x7f88540af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 >> v1:172.21.15.177:6789/0 conn(0x7f885415cef0 legacy=0x7f885406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885415da30 con 0x7f885406ba90 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f8844001670 con 0x7f885406ba90 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 >> v1:172.21.15.153:6789/0 conn(0x7f885406ba90 legacy=0x7f88540ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c003a00 0x7f883c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f883c004850 0x7f883c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f883c005200 0x7f883c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885415d750 con 0x7f883c005200 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c00c080 con 0x7f883c004850 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c00c240 con 0x7f883c003a00 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f883c005200 0x7f883c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8851d9b700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f883c004850 0x7f883c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885259c700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c003a00 0x7f883c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 -- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c003a00 msgr2=0x7f883c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c003a00 0x7f883c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 -- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f883c004850 msgr2=0x7f883c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f883c004850 0x7f883c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 -- 172.21.15.153:0/4188590073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0035b0 con 0x7f883c005200 2023-11-28T13:01:02.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885259c700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c003a00 0x7f883c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 -- 172.21.15.153:0/4188590073 reap_dead start 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f8852d9d700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f883c005200 0x7f883c00ba10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f884c009fc0 tx=0x7f883c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d545738c3037b75e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c003a20 con 0x7f883c005200 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c00f040 con 0x7f883c005200 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.998+0000 7f885159a700 1 -- 172.21.15.153:0/4188590073 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c003de0 con 0x7f883c005200 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.999+0000 7f8858b05700 1 -- 172.21.15.153:0/4188590073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f883c005200 msgr2=0x7f883c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.999+0000 7f8858b05700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f883c005200 0x7f883c00ba10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f884c009fc0 tx=0x7f883c005880 comp rx=0 tx=0).stop 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.999+0000 7f8858b05700 1 -- 172.21.15.153:0/4188590073 shutdown_connections 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.999+0000 7f8858b05700 1 --2- 172.21.15.153:0/4188590073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f883c005200 0x7f883c00ba10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.999+0000 7f8858b05700 1 -- 172.21.15.153:0/4188590073 >> 172.21.15.153:0/4188590073 conn(0x7f88540d0000 msgr2=0x7f88540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.999+0000 7f8858b05700 1 -- 172.21.15.153:0/4188590073 shutdown_connections 2023-11-28T13:01:02.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:01.999+0000 7f8858b05700 1 -- 172.21.15.153:0/4188590073 wait complete. 2023-11-28T13:01:02.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.000+0000 7f8858b05700 1 Processor -- start 2023-11-28T13:01:02.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8858b05700 1 -- start start 2023-11-28T13:01:02.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8858b05700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c005200 0x7f885414e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8858b05700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88540c90c0 0x7f88541f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8858b05700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88540c70d0 0x7f88540c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8858b05700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540c7a90 con 0x7f88540c70d0 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8858b05700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885414ea30 con 0x7f88540c90c0 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8858b05700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540ca440 con 0x7f883c005200 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8852d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88540c70d0 0x7f88540c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8852d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88540c70d0 0x7f88540c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39830/0 (socket says 172.21.15.153:39830) 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.001+0000 7f8852d9d700 1 -- 172.21.15.153:0/2974837764 learned_addr learned my addr 172.21.15.153:0/2974837764 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f885259c700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c005200 0x7f885414e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8851d9b700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88540c90c0 0x7f88541f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8852d9d700 1 -- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c005200 msgr2=0x7f885414e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8852d9d700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c005200 0x7f885414e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8852d9d700 1 -- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88540c90c0 msgr2=0x7f88541f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8852d9d700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88540c90c0 0x7f88541f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8852d9d700 1 -- 172.21.15.153:0/2974837764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885415dff0 con 0x7f88540c70d0 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f885259c700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c005200 0x7f885414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8851d9b700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88540c90c0 0x7f88541f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8852d9d700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88540c70d0 0x7f88540c7550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8854070e20 tx=0x7f884c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8842ffd700 1 -- 172.21.15.153:0/2974837764 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c003e90 con 0x7f88540c70d0 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8842ffd700 1 -- 172.21.15.153:0/2974837764 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c004230 con 0x7f88540c70d0 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8842ffd700 1 -- 172.21.15.153:0/2974837764 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0045e0 con 0x7f88540c70d0 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885415dd10 con 0x7f88540c70d0 2023-11-28T13:01:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.002+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88540af400 con 0x7f88540c70d0 2023-11-28T13:01:02.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.003+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88540b3f10 con 0x7f88540c70d0 2023-11-28T13:01:02.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.003+0000 7f8842ffd700 1 -- 172.21.15.153:0/2974837764 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f884c004740 con 0x7f88540c70d0 2023-11-28T13:01:02.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.004+0000 7f8842ffd700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88340682f0 0x7f883406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.004+0000 7f8842ffd700 1 -- 172.21.15.153:0/2974837764 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f884c087080 con 0x7f88540c70d0 2023-11-28T13:01:02.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.004+0000 7f885259c700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88340682f0 0x7f883406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.004+0000 7f885259c700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88340682f0 0x7f883406a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88480055b0 tx=0x7f8848009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.008+0000 7f8842ffd700 1 -- 172.21.15.153:0/2974837764 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88540b3f10 con 0x7f88540c70d0 2023-11-28T13:01:02.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.233+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f88540b3f10 con 0x7f88540c70d0 2023-11-28T13:01:02.568 INFO:tasks.ceph.mon.a.smithi153.stderr:2023-11-28T13:01:02.567+0000 7feb34d4f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:01:02.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.578+0000 7f8842ffd700 1 -- 172.21.15.153:0/2974837764 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f88540b3f10 con 0x7f88540c70d0 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88340682f0 msgr2=0x7f883406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88340682f0 0x7f883406a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88480055b0 tx=0x7f8848009040 comp rx=0 tx=0).stop 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88540c70d0 msgr2=0x7f88540c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88540c70d0 0x7f88540c7550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8854070e20 tx=0x7f884c009750 comp rx=0 tx=0).stop 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 shutdown_connections 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88340682f0 0x7f883406a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88540c70d0 0x7f88540c7550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f883c005200 0x7f885414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 --2- 172.21.15.153:0/2974837764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88540c90c0 0x7f88541f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 >> 172.21.15.153:0/2974837764 conn(0x7f88540d0000 msgr2=0x7f885406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 shutdown_connections 2023-11-28T13:01:02.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.580+0000 7f8858b05700 1 -- 172.21.15.153:0/2974837764 wait complete. 2023-11-28T13:01:02.582 INFO:teuthology.orchestra.run.smithi153.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:01:02.594 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:01:02.619 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T13:01:02.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.717+0000 7f0558205700 1 Processor -- start 2023-11-28T13:01:02.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.718+0000 7f0558205700 1 -- start start 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.718+0000 7f0558205700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f055006dee0 0x7f055006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f0558205700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055015d060 con 0x7f0550160540 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f0558205700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055015d340 con 0x7f0550071040 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f0558205700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055015d620 con 0x7f055006dee0 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f05557a0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f055006dee0 0x7f055006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f05567a2700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0550160540 0x7f0550160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58002/0 (socket says 172.21.15.153:58002) 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f05567a2700 1 -- 172.21.15.153:0/2611741813 learned_addr learned my addr 172.21.15.153:0/2611741813 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f05557a0700 1 -- 172.21.15.153:0/2611741813 >> v1:172.21.15.177:6789/0 conn(0x7f0550071040 legacy=0x7f0550071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f05557a0700 1 -- 172.21.15.153:0/2611741813 >> v1:172.21.15.153:6789/0 conn(0x7f0550160540 legacy=0x7f0550160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f05557a0700 1 -- 172.21.15.153:0/2611741813 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055015d900 con 0x7f055006dee0 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f05557a0700 1 --2- 172.21.15.153:0/2611741813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f055006dee0 0x7f055006e330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f054c009840 tx=0x7f054c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a39081823a2f843 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f0554f9f700 1 -- 172.21.15.153:0/2611741813 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c011830 con 0x7f055006dee0 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f0554f9f700 1 -- 172.21.15.153:0/2611741813 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c011990 con 0x7f055006dee0 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.719+0000 7f0554f9f700 1 -- 172.21.15.153:0/2611741813 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c011d20 con 0x7f055006dee0 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.720+0000 7f0558205700 1 -- 172.21.15.153:0/2611741813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f055006dee0 msgr2=0x7f055006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.720+0000 7f0558205700 1 --2- 172.21.15.153:0/2611741813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f055006dee0 0x7f055006e330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f054c009840 tx=0x7f054c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.720+0000 7f0558205700 1 -- 172.21.15.153:0/2611741813 shutdown_connections 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.720+0000 7f0558205700 1 --2- 172.21.15.153:0/2611741813 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f055006dee0 0x7f055006e330 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.720+0000 7f0558205700 1 -- 172.21.15.153:0/2611741813 >> 172.21.15.153:0/2611741813 conn(0x7f05500d0000 msgr2=0x7f05500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.720+0000 7f0558205700 1 -- 172.21.15.153:0/2611741813 shutdown_connections 2023-11-28T13:01:02.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.720+0000 7f0558205700 1 -- 172.21.15.153:0/2611741813 wait complete. 2023-11-28T13:01:02.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.721+0000 7f0558205700 1 Processor -- start 2023-11-28T13:01:02.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.721+0000 7f0558205700 1 -- start start 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0558205700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055006dee0 0x7f05500c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0558205700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0550071040 0x7f05500b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0558205700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0550160540 0x7f05501481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0558205700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05500c6760 con 0x7f0550160540 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0558205700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05500b5530 con 0x7f0550071040 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0558205700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05500b5810 con 0x7f055006dee0 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0550160540 0x7f05501481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0550160540 0x7f05501481f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39844/0 (socket says 172.21.15.153:39844) 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 -- 172.21.15.153:0/3941253667 learned_addr learned my addr 172.21.15.153:0/3941253667 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0555fa1700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055006dee0 0x7f05500c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05557a0700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0550071040 0x7f05500b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 -- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055006dee0 msgr2=0x7f05500c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055006dee0 0x7f05500c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 -- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0550071040 msgr2=0x7f05500b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0550071040 0x7f05500b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 -- 172.21.15.153:0/3941253667 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0094a0 con 0x7f0550160540 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05567a2700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0550160540 0x7f05501481f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f054800ca80 tx=0x7f054800ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05467fc700 1 -- 172.21.15.153:0/3941253667 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054800aaa0 con 0x7f0550160540 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f05467fc700 1 -- 172.21.15.153:0/3941253667 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054800ac00 con 0x7f0550160540 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.722+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055015ccd0 con 0x7f0550160540 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.723+0000 7f05467fc700 1 -- 172.21.15.153:0/3941253667 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548012400 con 0x7f0550160540 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.723+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055006e870 con 0x7f0550160540 2023-11-28T13:01:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.723+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055006a270 con 0x7f0550160540 2023-11-28T13:01:02.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.724+0000 7f05467fc700 1 -- 172.21.15.153:0/3941253667 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0548019030 con 0x7f0550160540 2023-11-28T13:01:02.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.724+0000 7f05467fc700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05340682f0 0x7f053406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.724+0000 7f0555fa1700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05340682f0 0x7f053406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.724+0000 7f05467fc700 1 -- 172.21.15.153:0/3941253667 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f0548085740 con 0x7f0550160540 2023-11-28T13:01:02.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.725+0000 7f0555fa1700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05340682f0 0x7f053406a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0540000f60 tx=0x7f0540009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.729+0000 7f05467fc700 1 -- 172.21.15.153:0/3941253667 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f054804b5e0 con 0x7f0550160540 2023-11-28T13:01:02.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:02.957+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05500b5d30 con 0x7f0550160540 2023-11-28T13:01:03.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.583+0000 7f05467fc700 1 -- 172.21.15.153:0/3941253667 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05500b5d30 con 0x7f0550160540 2023-11-28T13:01:03.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05340682f0 msgr2=0x7f053406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05340682f0 0x7f053406a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0540000f60 tx=0x7f0540009040 comp rx=0 tx=0).stop 2023-11-28T13:01:03.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0550160540 msgr2=0x7f05501481f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0550160540 0x7f05501481f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f054800ca80 tx=0x7f054800ce40 comp rx=0 tx=0).stop 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 shutdown_connections 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f05340682f0 0x7f053406a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0550160540 0x7f05501481f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055006dee0 0x7f05500c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 --2- 172.21.15.153:0/3941253667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0550071040 0x7f05500b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 >> 172.21.15.153:0/3941253667 conn(0x7f05500d0000 msgr2=0x7f05500cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 shutdown_connections 2023-11-28T13:01:03.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.585+0000 7f0558205700 1 -- 172.21.15.153:0/3941253667 wait complete. 2023-11-28T13:01:03.600 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:01:03.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.728+0000 7f913c873700 1 Processor -- start 2023-11-28T13:01:03.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.728+0000 7f913c873700 1 -- start start 2023-11-28T13:01:03.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913c873700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f913806a980 0x7f9138070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913c873700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91380af430 0x7f913806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913c873700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91380713c0 con 0x7f91380af060 2023-11-28T13:01:03.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913c873700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913806dee0 con 0x7f913806a980 2023-11-28T13:01:03.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913c873700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91380716a0 con 0x7f91380af430 2023-11-28T13:01:03.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f9135d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f913806a980 0x7f9138070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913659c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f91380af060 0x7f913806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58012/0 (socket says 172.21.15.153:58012) 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913659c700 1 -- 172.21.15.153:0/3552781532 learned_addr learned my addr 172.21.15.153:0/3552781532 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f9135d9b700 1 -- 172.21.15.153:0/3552781532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91380af430 msgr2=0x7f913806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f9135d9b700 1 --2- 172.21.15.153:0/3552781532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91380af430 0x7f913806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f9135d9b700 1 -- 172.21.15.153:0/3552781532 >> v1:172.21.15.153:6789/0 conn(0x7f91380af060 legacy=0x7f913806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913559a700 1 -- 172.21.15.153:0/3552781532 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131783492 0 0) 0x7f91380713c0 con 0x7f91380af060 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f9135d9b700 1 -- 172.21.15.153:0/3552781532 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138071980 con 0x7f913806a980 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f9135d9b700 1 --2- 172.21.15.153:0/3552781532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913806a980 0x7f9138070d50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f912c009840 tx=0x7f912c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1c6f96b953fa17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913559a700 1 -- 172.21.15.153:0/3552781532 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c011830 con 0x7f913806a980 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913559a700 1 -- 172.21.15.153:0/3552781532 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c011990 con 0x7f913806a980 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.729+0000 7f913559a700 1 -- 172.21.15.153:0/3552781532 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c011d20 con 0x7f913806a980 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 -- 172.21.15.153:0/3552781532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913806a980 msgr2=0x7f9138070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 --2- 172.21.15.153:0/3552781532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913806a980 0x7f9138070d50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f912c009840 tx=0x7f912c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:03.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 -- 172.21.15.153:0/3552781532 shutdown_connections 2023-11-28T13:01:03.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 --2- 172.21.15.153:0/3552781532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91380af430 0x7f913806d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 --2- 172.21.15.153:0/3552781532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913806a980 0x7f9138070d50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 -- 172.21.15.153:0/3552781532 >> 172.21.15.153:0/3552781532 conn(0x7f913806b380 msgr2=0x7f913806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 -- 172.21.15.153:0/3552781532 shutdown_connections 2023-11-28T13:01:03.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.730+0000 7f913c873700 1 -- 172.21.15.153:0/3552781532 wait complete. 2023-11-28T13:01:03.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.731+0000 7f913c873700 1 Processor -- start 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f913c873700 1 -- start start 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f913c873700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913806a980 0x7f913814bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f913c873700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91380af060 0x7f91380c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f913c873700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91380af430 0x7f91380cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f913c873700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913815d090 con 0x7f91380af060 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f913c873700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913815d370 con 0x7f91380af430 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f913c873700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913815d650 con 0x7f913806a980 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f9135d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91380af060 0x7f91380c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f9135d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91380af060 0x7f91380c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39856/0 (socket says 172.21.15.153:39856) 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.732+0000 7f9135d9b700 1 -- 172.21.15.153:0/2476053936 learned_addr learned my addr 172.21.15.153:0/2476053936 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f913659c700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913806a980 0x7f913814bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9136d9d700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91380af430 0x7f91380cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9135d9b700 1 -- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913806a980 msgr2=0x7f913814bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9135d9b700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913806a980 0x7f913814bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9135d9b700 1 -- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91380af430 msgr2=0x7f91380cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9135d9b700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91380af430 0x7f91380cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9135d9b700 1 -- 172.21.15.153:0/2476053936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0094a0 con 0x7f91380af060 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9135d9b700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91380af060 0x7f91380c82a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f912c014040 tx=0x7f912c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9126ffd700 1 -- 172.21.15.153:0/2476053936 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c0045f0 con 0x7f91380af060 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9126ffd700 1 -- 172.21.15.153:0/2476053936 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c01e070 con 0x7f91380af060 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f9126ffd700 1 -- 172.21.15.153:0/2476053936 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c003b00 con 0x7f91380af060 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9138071f40 con 0x7f91380af060 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.733+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9138071c60 con 0x7f91380af060 2023-11-28T13:01:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.734+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91380ca660 con 0x7f91380af060 2023-11-28T13:01:03.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.734+0000 7f9126ffd700 1 -- 172.21.15.153:0/2476053936 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f912c00f050 con 0x7f91380af060 2023-11-28T13:01:03.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.735+0000 7f9126ffd700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91180682f0 0x7f911806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.735+0000 7f9126ffd700 1 -- 172.21.15.153:0/2476053936 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f912c086ed0 con 0x7f91380af060 2023-11-28T13:01:03.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.735+0000 7f913659c700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91180682f0 0x7f911806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.735+0000 7f913659c700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91180682f0 0x7f911806a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9128000f60 tx=0x7f9128009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.739 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.739+0000 7f9126ffd700 1 -- 172.21.15.153:0/2476053936 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f912c0536a0 con 0x7f91380af060 2023-11-28T13:01:03.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:03.924+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9138074670 con 0x7f91380af060 2023-11-28T13:01:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.591+0000 7f9126ffd700 1 -- 172.21.15.153:0/2476053936 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f912c020020 con 0x7f91380af060 2023-11-28T13:01:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91180682f0 msgr2=0x7f911806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91180682f0 0x7f911806a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9128000f60 tx=0x7f9128009040 comp rx=0 tx=0).stop 2023-11-28T13:01:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91380af060 msgr2=0x7f91380c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91380af060 0x7f91380c82a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f912c014040 tx=0x7f912c004750 comp rx=0 tx=0).stop 2023-11-28T13:01:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 shutdown_connections 2023-11-28T13:01:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91380af430 0x7f91380cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913806a980 0x7f913814bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91180682f0 0x7f911806a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 --2- 172.21.15.153:0/2476053936 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91380af060 0x7f91380c82a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.595+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 >> 172.21.15.153:0/2476053936 conn(0x7f913806b380 msgr2=0x7f91380d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:04.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.596+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 shutdown_connections 2023-11-28T13:01:04.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.597+0000 7f913c873700 1 -- 172.21.15.153:0/2476053936 wait complete. 2023-11-28T13:01:04.603 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:01:04.614 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:01:04.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.741+0000 7f733958b700 1 Processor -- start 2023-11-28T13:01:04.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.741+0000 7f733958b700 1 -- start start 2023-11-28T13:01:04.744 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f733958b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f733415a760 0x7f7334161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.744 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f733958b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73340cded0 0x7f73340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.744 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f733958b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733415e0a0 con 0x7f733415dc30 2023-11-28T13:01:04.744 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f733958b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733415ab30 con 0x7f733415a760 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f733958b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7334160d70 con 0x7f73340cded0 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f73337fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73340cded0 0x7f73340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f7332ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f733415a760 0x7f7334161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f73337fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73340cded0 0x7f73340b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:57474/0 (socket says 172.21.15.153:57474) 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f73337fe700 1 -- 172.21.15.153:0/1231593124 learned_addr learned my addr 172.21.15.153:0/1231593124 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f73337fe700 1 -- 172.21.15.153:0/1231593124 >> v2:172.21.15.177:3300/0 conn(0x7f733415a760 msgr2=0x7f7334161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f73337fe700 1 --2- 172.21.15.153:0/1231593124 >> v2:172.21.15.177:3300/0 conn(0x7f733415a760 0x7f7334161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f73337fe700 1 -- 172.21.15.153:0/1231593124 >> v1:172.21.15.177:6789/0 conn(0x7f733415dc30 legacy=0x7f73340cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f73337fe700 1 -- 172.21.15.153:0/1231593124 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340b5130 con 0x7f73340cded0 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.742+0000 7f7332ffd700 1 --2- 172.21.15.153:0/1231593124 >> v2:172.21.15.177:3300/0 conn(0x7f733415a760 0x7f7334161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f73337fe700 1 --2- 172.21.15.153:0/1231593124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73340cded0 0x7f73340b4bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f732c009e50 tx=0x7f732c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd43e57a941167f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f7331ffb700 1 -- 172.21.15.153:0/1231593124 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00e070 con 0x7f73340cded0 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f7331ffb700 1 -- 172.21.15.153:0/1231593124 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c00b910 con 0x7f73340cded0 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f7331ffb700 1 -- 172.21.15.153:0/1231593124 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c00bcc0 con 0x7f73340cded0 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 -- 172.21.15.153:0/1231593124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73340cded0 msgr2=0x7f73340b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 --2- 172.21.15.153:0/1231593124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73340cded0 0x7f73340b4bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f732c009e50 tx=0x7f732c00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 -- 172.21.15.153:0/1231593124 shutdown_connections 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 --2- 172.21.15.153:0/1231593124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73340cded0 0x7f73340b4bf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 --2- 172.21.15.153:0/1231593124 >> v2:172.21.15.177:3300/0 conn(0x7f733415a760 0x7f7334161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 -- 172.21.15.153:0/1231593124 >> 172.21.15.153:0/1231593124 conn(0x7f7334072620 msgr2=0x7f73340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 -- 172.21.15.153:0/1231593124 shutdown_connections 2023-11-28T13:01:04.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.743+0000 7f733958b700 1 -- 172.21.15.153:0/1231593124 wait complete. 2023-11-28T13:01:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 Processor -- start 2023-11-28T13:01:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 -- start start 2023-11-28T13:01:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73340cded0 0x7f7334159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f733415a760 0x7f73340b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f733415dc30 0x7f73340d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73340c7590 con 0x7f73340cded0 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73340c6c80 con 0x7f733415dc30 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.745+0000 7f733958b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73340b2730 con 0x7f733415a760 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73340cded0 0x7f7334159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73340cded0 0x7f7334159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39866/0 (socket says 172.21.15.153:39866) 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 -- 172.21.15.153:0/1111542248 learned_addr learned my addr 172.21.15.153:0/1111542248 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f73327fc700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f733415a760 0x7f73340b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f73337fe700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f733415dc30 0x7f73340d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 -- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f733415a760 msgr2=0x7f73340b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f733415a760 0x7f73340b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 -- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f733415dc30 msgr2=0x7f73340d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f733415dc30 0x7f73340d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 -- 172.21.15.153:0/1111542248 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73280094a0 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f7332ffd700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73340cded0 0x7f7334159220 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f732800ebe0 tx=0x7f732800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f731b7fe700 1 -- 172.21.15.153:0/1111542248 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732800c850 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f731b7fe700 1 -- 172.21.15.153:0/1111542248 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732800c9b0 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c009b30 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f731b7fe700 1 -- 172.21.15.153:0/1111542248 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732800cd40 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.746+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73340b5670 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.747+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73340b5390 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.748+0000 7f731b7fe700 1 -- 172.21.15.153:0/1111542248 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7328019030 con 0x7f73340cded0 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.748+0000 7f731b7fe700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73200682f0 0x7f732006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.748+0000 7f731b7fe700 1 -- 172.21.15.153:0/1111542248 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f7328085b70 con 0x7f73340cded0 2023-11-28T13:01:04.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.748+0000 7f73327fc700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73200682f0 0x7f732006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.748+0000 7f73327fc700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73200682f0 0x7f732006a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f733415fde0 tx=0x7f7324009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.752+0000 7f731b7fe700 1 -- 172.21.15.153:0/1111542248 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7328052470 con 0x7f73340cded0 2023-11-28T13:01:04.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:04.937+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7334160720 con 0x7f73340cded0 2023-11-28T13:01:05.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.596+0000 7f731b7fe700 1 -- 172.21.15.153:0/1111542248 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f73280210d0 con 0x7f73340cded0 2023-11-28T13:01:05.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73200682f0 msgr2=0x7f732006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73200682f0 0x7f732006a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f733415fde0 tx=0x7f7324009040 comp rx=0 tx=0).stop 2023-11-28T13:01:05.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73340cded0 msgr2=0x7f7334159220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73340cded0 0x7f7334159220 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f732800ebe0 tx=0x7f732800eef0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 shutdown_connections 2023-11-28T13:01:05.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73200682f0 0x7f732006a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73340cded0 0x7f7334159220 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f732800ebe0 tx=0x7f732800eef0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f733415dc30 0x7f73340d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 --2- 172.21.15.153:0/1111542248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f733415a760 0x7f73340b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.599+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 >> 172.21.15.153:0/1111542248 conn(0x7f7334072620 msgr2=0x7f733415e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:05.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.600+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 shutdown_connections 2023-11-28T13:01:05.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.600+0000 7f733958b700 1 -- 172.21.15.153:0/1111542248 wait complete. 2023-11-28T13:01:05.603 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' removed 2023-11-28T13:01:05.617 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:01:05.617 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:01:05.617 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:01:05.617 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:05.617 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:01:05.618 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:01:05.618 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:01:05.618 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:05.618 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:01:05.618 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:01:05.618 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:01:05.619 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:05.619 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:01:05.619 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:01:05.619 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:01:05.619 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:05.628 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:01:05.628 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:01:05.629 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:01:05.629 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:01:05.631 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:01:05.631 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:01:05.632 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:01:05.632 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:01:05.671 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:01:05.671 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:05.672 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:01:05.672 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:05.673 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:01:05.674 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:05.674 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:01:05.674 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:05.675 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:01:05.763 INFO:tasks.ceph.mds.c.smithi153.stdout:starting mds.c at 2023-11-28T13:01:05.763 INFO:tasks.ceph.mds.a.smithi153.stdout:starting mds.a at 2023-11-28T13:01:05.763 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T13:01:05.766 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T13:01:05.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.816+0000 7f8ac307a700 1 Processor -- start 2023-11-28T13:01:05.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.816+0000 7f8ac307a700 1 -- start start 2023-11-28T13:01:05.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac307a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8abc15ca50 0x7f8abc06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac307a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 0x7f8abc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac307a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0cf5e0 con 0x7f8abc06b8c0 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac307a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0cd290 con 0x7f8abc15ca50 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac307a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc06bd60 con 0x7f8abc0cce10 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abbfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f8abc06b8c0 0x7f8abc0cc700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58062/0 (socket says 172.21.15.153:58062) 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abbfff700 1 -- 172.21.15.153:0/3785385689 learned_addr learned my addr 172.21.15.153:0/3785385689 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac0e16700 1 --2- 172.21.15.153:0/3785385689 >> v2:172.21.15.177:3300/0 conn(0x7f8abc15ca50 0x7f8abc06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac1617700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 0x7f8abc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218006851 0 0) 0x7f8abc0cf5e0 con 0x7f8abc06b8c0 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aac0035b0 con 0x7f8abc06b8c0 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 596513827 0 0) 0x7f8aac0035b0 con 0x7f8abc06b8c0 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 msgr2=0x7f8abc0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 0x7f8abc0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 >> v2:172.21.15.177:3300/0 conn(0x7f8abc15ca50 msgr2=0x7f8abc06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 --2- 172.21.15.153:0/3785385689 >> v2:172.21.15.177:3300/0 conn(0x7f8abc15ca50 0x7f8abc06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cecb0 con 0x7f8abc06b8c0 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f8aa8001670 con 0x7f8abc06b8c0 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8ac1617700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 0x7f8abc0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 >> v1:172.21.15.153:6789/0 conn(0x7f8abc06b8c0 legacy=0x7f8abc0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 0x7f8aac003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.817+0000 7f8abb7fe700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8aac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abb7fe700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8aac004d90 0x7f8aac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0cd290 con 0x7f8aac003f40 2023-11-28T13:01:05.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac00ba10 con 0x7f8aac004d90 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac00bb50 con 0x7f8abc0cce10 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8ac0e16700 1 --2- 172.21.15.153:0/3785385689 >> v2:172.21.15.177:3300/0 conn(0x7f8abc15ca50 0x7f8abc06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abbfff700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8aac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8ac1617700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8aac004d90 0x7f8aac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8ac0e16700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 0x7f8aac003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abbfff700 1 -- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 msgr2=0x7f8aac003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abbfff700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0cce10 0x7f8aac003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abbfff700 1 -- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8aac004d90 msgr2=0x7f8aac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abbfff700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8aac004d90 0x7f8aac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abbfff700 1 -- 172.21.15.153:0/3785385689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009a70 con 0x7f8aac003f40 2023-11-28T13:01:05.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abbfff700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8aac004850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c320 tx=0x7f8aac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921aeae481fd34f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800ce60 con 0x7f8aac003f40 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa8005960 con 0x7f8aac003f40 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.818+0000 7f8abb7fe700 1 -- 172.21.15.153:0/3785385689 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8005cf0 con 0x7f8aac003f40 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.819+0000 7f8ac307a700 1 -- 172.21.15.153:0/3785385689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 msgr2=0x7f8aac004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.819+0000 7f8ac307a700 1 --2- 172.21.15.153:0/3785385689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8aac004850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c320 tx=0x7f8aac005880 comp rx=0 tx=0).stop 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.819+0000 7f8ac1617700 1 -- 172.21.15.153:0/3785385689 reap_dead start 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.819+0000 7f8ac307a700 1 -- 172.21.15.153:0/3785385689 shutdown_connections 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.819+0000 7f8ac307a700 1 -- 172.21.15.153:0/3785385689 >> 172.21.15.153:0/3785385689 conn(0x7f8abc09a230 msgr2=0x7f8abc08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.819+0000 7f8ac307a700 1 -- 172.21.15.153:0/3785385689 shutdown_connections 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.819+0000 7f8ac307a700 1 -- 172.21.15.153:0/3785385689 wait complete. 2023-11-28T13:01:05.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.820+0000 7f8ac307a700 1 Processor -- start 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.820+0000 7f8ac307a700 1 -- start start 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac307a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8abc1554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac307a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc15e700 0x7f8abc0b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac307a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15eb70 0x7f8abc0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac307a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1601d0 con 0x7f8aac003f40 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac307a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15f130 con 0x7f8abc15eb70 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac307a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc15f410 con 0x7f8abc15e700 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8abc1554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8abc1554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39910/0 (socket says 172.21.15.153:39910) 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 -- 172.21.15.153:0/1413263724 learned_addr learned my addr 172.21.15.153:0/1413263724 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8abbfff700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc15e700 0x7f8abc0b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac1617700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15eb70 0x7f8abc0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 -- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc15e700 msgr2=0x7f8abc0b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc15e700 0x7f8abc0b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 -- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15eb70 msgr2=0x7f8abc0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15eb70 0x7f8abc0b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 -- 172.21.15.153:0/1413263724 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cf270 con 0x7f8aac003f40 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac1617700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15eb70 0x7f8abc0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8abbfff700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc15e700 0x7f8abc0b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ac0e16700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8abc1554b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0011cd0 tx=0x7f8ab000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.821+0000 7f8ab97fa700 1 -- 172.21.15.153:0/1413263724 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000fb30 con 0x7f8aac003f40 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.822+0000 7f8ab97fa700 1 -- 172.21.15.153:0/1413263724 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab000fc90 con 0x7f8aac003f40 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.822+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc0cef90 con 0x7f8aac003f40 2023-11-28T13:01:05.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.822+0000 7f8ab97fa700 1 -- 172.21.15.153:0/1413263724 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0004030 con 0x7f8aac003f40 2023-11-28T13:01:05.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.822+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc15ce20 con 0x7f8aac003f40 2023-11-28T13:01:05.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.823+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc0745c0 con 0x7f8aac003f40 2023-11-28T13:01:05.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.823+0000 7f8ab97fa700 1 -- 172.21.15.153:0/1413263724 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ab0024070 con 0x7f8aac003f40 2023-11-28T13:01:05.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.823+0000 7f8ab97fa700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8aa0068320 0x7f8aa006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.824+0000 7f8abbfff700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8aa0068320 0x7f8aa006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.824+0000 7f8ab97fa700 1 -- 172.21.15.153:0/1413263724 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8ab0085f10 con 0x7f8aac003f40 2023-11-28T13:01:05.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.824+0000 7f8abbfff700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8aa0068320 0x7f8aa006a7d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8000c00 tx=0x7f8aa800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:05.828+0000 7f8ab97fa700 1 -- 172.21.15.153:0/1413263724 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ab0052a70 con 0x7f8aac003f40 2023-11-28T13:01:06.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.024+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8abc06f510 con 0x7f8aac003f40 2023-11-28T13:01:06.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.807+0000 7f8ab97fa700 1 -- 172.21.15.153:0/1413263724 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8ab0022020 con 0x7f8aac003f40 2023-11-28T13:01:06.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.809+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8aa0068320 msgr2=0x7f8aa006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.809+0000 7f8ac307a700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8aa0068320 0x7f8aa006a7d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8000c00 tx=0x7f8aa800a000 comp rx=0 tx=0).stop 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.809+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 msgr2=0x7f8abc1554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.809+0000 7f8ac307a700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8abc1554b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0011cd0 tx=0x7f8ab000f520 comp rx=0 tx=0).stop 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.809+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 shutdown_connections 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.810+0000 7f8ac307a700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8aa0068320 0x7f8aa006a7d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.810+0000 7f8ac307a700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8aac003f40 0x7f8abc1554b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.810+0000 7f8ac307a700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15eb70 0x7f8abc0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.810+0000 7f8ac307a700 1 --2- 172.21.15.153:0/1413263724 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc15e700 0x7f8abc0b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.810+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 >> 172.21.15.153:0/1413263724 conn(0x7f8abc09a230 msgr2=0x7f8abc159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.810+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 shutdown_connections 2023-11-28T13:01:06.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.810+0000 7f8ac307a700 1 -- 172.21.15.153:0/1413263724 wait complete. 2023-11-28T13:01:06.812 INFO:teuthology.orchestra.run.smithi153.stderr: removed all blocklist entries 2023-11-28T13:01:06.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:01:06.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.950+0000 7f60c7f25700 1 Processor -- start 2023-11-28T13:01:06.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.950+0000 7f60c7f25700 1 -- start start 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c7f25700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 0x7f60c00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c7f25700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f60c006ba90 0x7f60c00ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c7f25700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c00cc5f0 con 0x7f60c00cbdf0 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c7f25700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c015d470 con 0x7f60c006ba90 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c7f25700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c015d750 con 0x7f60c015cef0 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c64c2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f60c006ba90 0x7f60c00ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c54c0700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f60c00cbdf0 0x7f60c006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58086/0 (socket says 172.21.15.153:58086) 2023-11-28T13:01:06.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c54c0700 1 -- 172.21.15.153:0/4070446792 learned_addr learned my addr 172.21.15.153:0/4070446792 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c64c2700 1 -- 172.21.15.153:0/4070446792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 msgr2=0x7f60c00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c64c2700 1 --2- 172.21.15.153:0/4070446792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 0x7f60c00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c64c2700 1 -- 172.21.15.153:0/4070446792 >> v1:172.21.15.153:6789/0 conn(0x7f60c00cbdf0 legacy=0x7f60c006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c4cbf700 1 -- 172.21.15.153:0/4070446792 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413031866 0 0) 0x7f60c00cc5f0 con 0x7f60c00cbdf0 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c64c2700 1 -- 172.21.15.153:0/4070446792 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c015da30 con 0x7f60c006ba90 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c64c2700 1 --2- 172.21.15.153:0/4070446792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c006ba90 0x7f60c00ca650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60b800bfa0 tx=0x7f60b800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a8ccdee5a3c7da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c4cbf700 1 -- 172.21.15.153:0/4070446792 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8010070 con 0x7f60c006ba90 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c4cbf700 1 -- 172.21.15.153:0/4070446792 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b800b2e0 con 0x7f60c006ba90 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.951+0000 7f60c4cbf700 1 -- 172.21.15.153:0/4070446792 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800dcf0 con 0x7f60c006ba90 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 -- 172.21.15.153:0/4070446792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c006ba90 msgr2=0x7f60c00ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 --2- 172.21.15.153:0/4070446792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c006ba90 0x7f60c00ca650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60b800bfa0 tx=0x7f60b800d330 comp rx=0 tx=0).stop 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 -- 172.21.15.153:0/4070446792 shutdown_connections 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 --2- 172.21.15.153:0/4070446792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 0x7f60c00af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 --2- 172.21.15.153:0/4070446792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c006ba90 0x7f60c00ca650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 -- 172.21.15.153:0/4070446792 >> 172.21.15.153:0/4070446792 conn(0x7f60c00d0000 msgr2=0x7f60c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 -- 172.21.15.153:0/4070446792 shutdown_connections 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.952+0000 7f60c7f25700 1 -- 172.21.15.153:0/4070446792 wait complete. 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.953+0000 7f60c7f25700 1 Processor -- start 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.953+0000 7f60c7f25700 1 -- start start 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c7f25700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60c006ba90 0x7f60c00c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c7f25700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c00cbdf0 0x7f60c00b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c7f25700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 0x7f60c0153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c7f25700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00c76b0 con 0x7f60c00cbdf0 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c7f25700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00c7990 con 0x7f60c006ba90 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c7f25700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00c7c70 con 0x7f60c015cef0 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c00cbdf0 0x7f60c00b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c00cbdf0 0x7f60c00b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39930/0 (socket says 172.21.15.153:39930) 2023-11-28T13:01:06.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 -- 172.21.15.153:0/3082156273 learned_addr learned my addr 172.21.15.153:0/3082156273 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:06.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c64c2700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 0x7f60c0153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c5cc1700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60c006ba90 0x7f60c00c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 -- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 msgr2=0x7f60c0153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 0x7f60c0153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 -- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60c006ba90 msgr2=0x7f60c00c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60c006ba90 0x7f60c00c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 -- 172.21.15.153:0/3082156273 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b800bc80 con 0x7f60c00cbdf0 2023-11-28T13:01:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.954+0000 7f60c54c0700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c00cbdf0 0x7f60c00b51a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f60b00011e0 tx=0x7f60b000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.955+0000 7f60b67fc700 1 -- 172.21.15.153:0/3082156273 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0004c80 con 0x7f60c00cbdf0 2023-11-28T13:01:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.955+0000 7f60b67fc700 1 -- 172.21.15.153:0/3082156273 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b0004e00 con 0x7f60c00cbdf0 2023-11-28T13:01:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.955+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c015dff0 con 0x7f60c00cbdf0 2023-11-28T13:01:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.955+0000 7f60b67fc700 1 -- 172.21.15.153:0/3082156273 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0007310 con 0x7f60c00cbdf0 2023-11-28T13:01:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.955+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c015dd10 con 0x7f60c00cbdf0 2023-11-28T13:01:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.956+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c00cab90 con 0x7f60c00cbdf0 2023-11-28T13:01:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.956+0000 7f60b67fc700 1 -- 172.21.15.153:0/3082156273 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f60b001a020 con 0x7f60c00cbdf0 2023-11-28T13:01:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.956+0000 7f60b67fc700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60a40682f0 0x7f60a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.956+0000 7f60b67fc700 1 -- 172.21.15.153:0/3082156273 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f60b0085590 con 0x7f60c00cbdf0 2023-11-28T13:01:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.956+0000 7f60c5cc1700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60a40682f0 0x7f60a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.957+0000 7f60c5cc1700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60a40682f0 0x7f60a406a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60bc006fd0 tx=0x7f60bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:06.960+0000 7f60b67fc700 1 -- 172.21.15.153:0/3082156273 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f60b0052140 con 0x7f60c00cbdf0 2023-11-28T13:01:07.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.215+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f60c00d0e60 con 0x7f60c00cbdf0 2023-11-28T13:01:07.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.215+0000 7f60b67fc700 1 -- 172.21.15.153:0/3082156273 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f60b0055760 con 0x7f60c00cbdf0 2023-11-28T13:01:07.216 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:07.216 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:01:07.216 INFO:teuthology.orchestra.run.smithi153.stdout: "auth_dump": [ 2023-11-28T13:01:07.216 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.216 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.a", 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlYBAHIBAA4jaMav4CAatojxXY6AkGPw==", 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.b", 2023-11-28T13:01:07.217 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVl9vBaLBAAD84m+iSSScY6ETZRBpBUEg==", 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.c", 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlG5hJJxAAtFV9HSiLJClNn3nmB7ORSg==", 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.218 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.d", 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlvAC6MxAAX0N4lN+NKADdjjjROw+Zbw==", 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.219 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.0", 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAq5GVlatZbJhAA2RQenckwVd7P0kBpzW08Lg==", 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.220 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.1", 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAt5GVlf73BJRAA51B4aDKKKVO+yo0+jQ20Yw==", 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.221 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.2", 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAw5GVl3M9qJRAAa2Jhr7Cp6vN/fRKJpMzCOg==", 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.3", 2023-11-28T13:01:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAz5GVlWVgHJRAAlP6Gc1QZkzufchXRrkLkbA==", 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.4", 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA55GVlLb5hJRAAMMymIqHNw8oeVW8s3BrtIg==", 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.223 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.5", 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA85GVlwsLjJBAAb/1bWARS8ILpFdBGemlcoA==", 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.224 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.6", 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA/5GVlwXx5JBAAXOS8JqPaXyQ7sj+2t9tsPQ==", 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.7", 2023-11-28T13:01:07.225 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBC5GVl5q+qJBAAMc7ZUInNCMJmf0lC22s1QQ==", 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.0", 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlR2IBOBAAJBEAQ3PnTN6KaU44zoHW7Q==", 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.226 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.1", 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAn5GVlTvYJAxAA68QZff0NcXXeMV7aGZoBfQ==", 2023-11-28T13:01:07.227 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.2", 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlsT5YOxAAZ6ep21AlkI2znaE9wyskxA==", 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.228 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.3", 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAn5GVlxW63CBAApiPmiN4tN3S2dZH/bCKS/Q==", 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:01:07.229 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.admin", 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlE1RYARAA5UHDpYgXbOEuWdAhlYnmbA==", 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.230 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.bootstrap-mds", 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBI5GVlmkaEIRAAvRD4908y9vk52HT0Y4ocuw==", 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBI5GVl3U6EIRAAwcjCbmItMoYQoRV4LG9DpA==", 2023-11-28T13:01:07.231 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.bootstrap-osd", 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBI5GVlMFaEIRAA9sV53i5BmhRFWTRrVmfNHA==", 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.232 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBI5GVlI1+EIRAAsDym9/T17uAEbdhIV0IROw==", 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBI5GVlk2aEIRAAAb9dhUCcz4lbxFQDBToArA==", 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBI5GVlKW6EIRAA5o22aMCvxkE+82cOwLuzLw==", 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mgr.x", 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlKLmuHBAANErUDgd4g3Yysqk/sPAbrA==", 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.234 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile mgr", 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mgr.y", 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVld7eyFhAAafZalFXR/4jRd5fUct7T2Q==", 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile mgr", 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.235 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:07.236 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2023-11-28T13:01:07.236 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:01:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60a40682f0 msgr2=0x7f60a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60a40682f0 0x7f60a406a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60bc006fd0 tx=0x7f60bc008040 comp rx=0 tx=0).stop 2023-11-28T13:01:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c00cbdf0 msgr2=0x7f60c00b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c00cbdf0 0x7f60c00b51a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f60b00011e0 tx=0x7f60b000cae0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 shutdown_connections 2023-11-28T13:01:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60c015cef0 0x7f60c0153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60c006ba90 0x7f60c00c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60a40682f0 0x7f60a406a7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 --2- 172.21.15.153:0/3082156273 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60c00cbdf0 0x7f60c00b51a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 >> 172.21.15.153:0/3082156273 conn(0x7f60c00d0000 msgr2=0x7f60c00caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 shutdown_connections 2023-11-28T13:01:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.218+0000 7f60c7f25700 1 -- 172.21.15.153:0/3082156273 wait complete. 2023-11-28T13:01:07.238 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T13:01:07.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.359+0000 7f8325ad7700 1 Processor -- start 2023-11-28T13:01:07.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.359+0000 7f8325ad7700 1 -- start start 2023-11-28T13:01:07.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f8325ad7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8320071040 0x7f8320071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f8325ad7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f832006dee0 0x7f832006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f8325ad7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f832006cdc0 0x7f832006d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f8325ad7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8320071950 con 0x7f832006cdc0 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f8325ad7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832015d060 con 0x7f832006dee0 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f8325ad7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832015d340 con 0x7f8320071040 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f832006cdc0 0x7f832006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f832006cdc0 0x7f832006d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39940/0 (socket says 172.21.15.153:39940) 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 -- 172.21.15.153:0/2189526520 learned_addr learned my addr 172.21.15.153:0/2189526520 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831f7fe700 1 --2- 172.21.15.153:0/2189526520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8320071040 0x7f8320071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831effd700 1 --2- 172.21.15.153:0/2189526520 >> v2:172.21.15.177:3300/0 conn(0x7f832006dee0 0x7f832006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 -- 172.21.15.153:0/2189526520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8320071040 msgr2=0x7f8320071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 --2- 172.21.15.153:0/2189526520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8320071040 0x7f8320071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 -- 172.21.15.153:0/2189526520 >> v2:172.21.15.177:3300/0 conn(0x7f832006dee0 msgr2=0x7f832006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 --2- 172.21.15.153:0/2189526520 >> v2:172.21.15.177:3300/0 conn(0x7f832006dee0 0x7f832006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 -- 172.21.15.153:0/2189526520 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832015d620 con 0x7f832006cdc0 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831effd700 1 --2- 172.21.15.153:0/2189526520 >> v2:172.21.15.177:3300/0 conn(0x7f832006dee0 0x7f832006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.360+0000 7f831ffff700 1 --2- 172.21.15.153:0/2189526520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f832006d240 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f832009e140 tx=0x7f831800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9233ff92e17426 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f831e7fc700 1 -- 172.21.15.153:0/2189526520 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800e070 con 0x7f832006cdc0 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f831e7fc700 1 -- 172.21.15.153:0/2189526520 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831800b910 con 0x7f832006cdc0 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f831e7fc700 1 -- 172.21.15.153:0/2189526520 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800bcc0 con 0x7f832006cdc0 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 -- 172.21.15.153:0/2189526520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 msgr2=0x7f832006d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 --2- 172.21.15.153:0/2189526520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f832006d240 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f832009e140 tx=0x7f831800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 -- 172.21.15.153:0/2189526520 shutdown_connections 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 --2- 172.21.15.153:0/2189526520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8320071040 0x7f8320071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 --2- 172.21.15.153:0/2189526520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f832006d240 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 --2- 172.21.15.153:0/2189526520 >> v2:172.21.15.177:3300/0 conn(0x7f832006dee0 0x7f832006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 -- 172.21.15.153:0/2189526520 >> 172.21.15.153:0/2189526520 conn(0x7f83200d0000 msgr2=0x7f83200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 -- 172.21.15.153:0/2189526520 shutdown_connections 2023-11-28T13:01:07.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.361+0000 7f8325ad7700 1 -- 172.21.15.153:0/2189526520 wait complete. 2023-11-28T13:01:07.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.362+0000 7f8325ad7700 1 Processor -- start 2023-11-28T13:01:07.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f8325ad7700 1 -- start start 2023-11-28T13:01:07.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f8325ad7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f8320144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f8325ad7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832006dee0 0x7f83200c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f8325ad7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8320071040 0x7f832015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f8325ad7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201454e0 con 0x7f832006cdc0 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f8325ad7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200c64b0 con 0x7f8320071040 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f8325ad7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83200c5ae0 con 0x7f832006dee0 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.363+0000 7f831f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f8320144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f8320144fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39942/0 (socket says 172.21.15.153:39942) 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 -- 172.21.15.153:0/617425857 learned_addr learned my addr 172.21.15.153:0/617425857 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831ffff700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8320071040 0x7f832015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831effd700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832006dee0 0x7f83200c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 -- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832006dee0 msgr2=0x7f83200c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832006dee0 0x7f83200c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 -- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8320071040 msgr2=0x7f832015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8320071040 0x7f832015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 -- 172.21.15.153:0/617425857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318009b30 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f831f7fe700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f8320144fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f831000e8d0 tx=0x7f831000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f8307fff700 1 -- 172.21.15.153:0/617425857 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310009590 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f8307fff700 1 -- 172.21.15.153:0/617425857 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8310013680 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f8307fff700 1 -- 172.21.15.153:0/617425857 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83100139f0 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832006e890 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.364+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832015d900 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.365+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832015ccd0 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.366+0000 7f8307fff700 1 -- 172.21.15.153:0/617425857 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8310019030 con 0x7f832006cdc0 2023-11-28T13:01:07.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.366+0000 7f8307fff700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f830c0682f0 0x7f830c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.366+0000 7f8307fff700 1 -- 172.21.15.153:0/617425857 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8310085640 con 0x7f832006cdc0 2023-11-28T13:01:07.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.366+0000 7f831effd700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f830c0682f0 0x7f830c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.366+0000 7f831effd700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f830c0682f0 0x7f830c06a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83201f6190 tx=0x7f8314008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.370+0000 7f8307fff700 1 -- 172.21.15.153:0/617425857 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f83100521f0 con 0x7f832006cdc0 2023-11-28T13:01:07.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.628+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f83200720a0 con 0x7f832006cdc0 2023-11-28T13:01:07.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.631+0000 7f8307fff700 1 -- 172.21.15.153:0/617425857 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8310055810 con 0x7f832006cdc0 2023-11-28T13:01:07.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f830c0682f0 msgr2=0x7f830c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f830c0682f0 0x7f830c06a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83201f6190 tx=0x7f8314008040 comp rx=0 tx=0).stop 2023-11-28T13:01:07.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 msgr2=0x7f8320144fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f8320144fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f831000e8d0 tx=0x7f831000ebe0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 shutdown_connections 2023-11-28T13:01:07.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f830c0682f0 0x7f830c06a7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f832006cdc0 0x7f8320144fa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8320071040 0x7f832015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 --2- 172.21.15.153:0/617425857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f832006dee0 0x7f83200c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 >> 172.21.15.153:0/617425857 conn(0x7f83200d0000 msgr2=0x7f832015e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 shutdown_connections 2023-11-28T13:01:07.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.633+0000 7f8325ad7700 1 -- 172.21.15.153:0/617425857 wait complete. 2023-11-28T13:01:07.646 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T13:01:07.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.766+0000 7faa92280700 1 Processor -- start 2023-11-28T13:01:07.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.766+0000 7faa92280700 1 -- start start 2023-11-28T13:01:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa92280700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faa8c071040 0x7faa8c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa92280700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7faa8c06dee0 0x7faa8c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa92280700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c06cdc0 0x7faa8c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa92280700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c071950 con 0x7faa8c071040 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa92280700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c15d060 con 0x7faa8c06dee0 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa92280700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c15d340 con 0x7faa8c06cdc0 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faa8c071040 0x7faa8c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7faa8c071040 0x7faa8c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39950/0 (socket says 172.21.15.153:39950) 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 -- 172.21.15.153:0/3733490766 learned_addr learned my addr 172.21.15.153:0/3733490766 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8b7fe700 1 --2- 172.21.15.153:0/3733490766 >> v2:172.21.15.177:3300/0 conn(0x7faa8c06dee0 0x7faa8c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa9081d700 1 --2- 172.21.15.153:0/3733490766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c06cdc0 0x7faa8c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 -- 172.21.15.153:0/3733490766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c06cdc0 msgr2=0x7faa8c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 --2- 172.21.15.153:0/3733490766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c06cdc0 0x7faa8c06d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 -- 172.21.15.153:0/3733490766 >> v2:172.21.15.177:3300/0 conn(0x7faa8c06dee0 msgr2=0x7faa8c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 --2- 172.21.15.153:0/3733490766 >> v2:172.21.15.177:3300/0 conn(0x7faa8c06dee0 0x7faa8c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 -- 172.21.15.153:0/3733490766 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c15d620 con 0x7faa8c071040 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa9081d700 1 --2- 172.21.15.153:0/3733490766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c06cdc0 0x7faa8c06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8bfff700 1 --2- 172.21.15.153:0/3733490766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c071040 0x7faa8c071410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa8c06fb00 tx=0x7faa7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b346ccc90d56c5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8affd700 1 -- 172.21.15.153:0/3733490766 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c011830 con 0x7faa8c071040 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8affd700 1 -- 172.21.15.153:0/3733490766 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c011990 con 0x7faa8c071040 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.767+0000 7faa8affd700 1 -- 172.21.15.153:0/3733490766 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c011d20 con 0x7faa8c071040 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 -- 172.21.15.153:0/3733490766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c071040 msgr2=0x7faa8c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 --2- 172.21.15.153:0/3733490766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c071040 0x7faa8c071410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa8c06fb00 tx=0x7faa7c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 -- 172.21.15.153:0/3733490766 shutdown_connections 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 --2- 172.21.15.153:0/3733490766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c06cdc0 0x7faa8c06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 --2- 172.21.15.153:0/3733490766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c071040 0x7faa8c071410 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 --2- 172.21.15.153:0/3733490766 >> v2:172.21.15.177:3300/0 conn(0x7faa8c06dee0 0x7faa8c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 -- 172.21.15.153:0/3733490766 >> 172.21.15.153:0/3733490766 conn(0x7faa8c0d0000 msgr2=0x7faa8c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 -- 172.21.15.153:0/3733490766 shutdown_connections 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.768+0000 7faa92280700 1 -- 172.21.15.153:0/3733490766 wait complete. 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.769+0000 7faa92280700 1 Processor -- start 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.769+0000 7faa92280700 1 -- start start 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa92280700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa8c06cdc0 0x7faa8c157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa92280700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c06dee0 0x7faa8c0c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa92280700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c071040 0x7faa8c0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa92280700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0c5bc0 con 0x7faa8c06dee0 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa92280700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0c6520 con 0x7faa8c06cdc0 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa92280700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0b5b70 con 0x7faa8c071040 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c06dee0 0x7faa8c0c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c06dee0 0x7faa8c0c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39958/0 (socket says 172.21.15.153:39958) 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 -- 172.21.15.153:0/189724907 learned_addr learned my addr 172.21.15.153:0/189724907 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa9081d700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c071040 0x7faa8c0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8bfff700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa8c06cdc0 0x7faa8c157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 -- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c071040 msgr2=0x7faa8c0b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c071040 0x7faa8c0b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 -- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa8c06cdc0 msgr2=0x7faa8c157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa8c06cdc0 0x7faa8c157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.770+0000 7faa8b7fe700 1 -- 172.21.15.153:0/189724907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c009550 con 0x7faa8c06dee0 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.771+0000 7faa8bfff700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa8c06cdc0 0x7faa8c157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.771+0000 7faa8b7fe700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c06dee0 0x7faa8c0c5fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa8000e8a0 tx=0x7faa8000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.771+0000 7faa88ff9700 1 -- 172.21.15.153:0/189724907 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80009590 con 0x7faa8c06dee0 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.771+0000 7faa88ff9700 1 -- 172.21.15.153:0/189724907 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8000eea0 con 0x7faa8c06dee0 2023-11-28T13:01:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.771+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c06e870 con 0x7faa8c06dee0 2023-11-28T13:01:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.771+0000 7faa88ff9700 1 -- 172.21.15.153:0/189724907 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000ca50 con 0x7faa8c06dee0 2023-11-28T13:01:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.771+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c15d900 con 0x7faa8c06dee0 2023-11-28T13:01:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.772+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c15ccd0 con 0x7faa8c06dee0 2023-11-28T13:01:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.772+0000 7faa88ff9700 1 -- 172.21.15.153:0/189724907 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa80019030 con 0x7faa8c06dee0 2023-11-28T13:01:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.772+0000 7faa88ff9700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faa780682f0 0x7faa7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.773+0000 7faa88ff9700 1 -- 172.21.15.153:0/189724907 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faa80085780 con 0x7faa8c06dee0 2023-11-28T13:01:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.773+0000 7faa8bfff700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faa780682f0 0x7faa7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.773+0000 7faa8bfff700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faa780682f0 0x7faa7806a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faa8c1f5c70 tx=0x7faa7c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:07.777+0000 7faa88ff9700 1 -- 172.21.15.153:0/189724907 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa80052330 con 0x7faa8c06dee0 2023-11-28T13:01:08.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.032+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7faa8c0c5d00 con 0x7faa8c06dee0 2023-11-28T13:01:08.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.036+0000 7faa88ff9700 1 -- 172.21.15.153:0/189724907 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7faa8c0c5d00 con 0x7faa8c06dee0 2023-11-28T13:01:08.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faa780682f0 msgr2=0x7faa7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faa780682f0 0x7faa7806a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faa8c1f5c70 tx=0x7faa7c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:08.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c06dee0 msgr2=0x7faa8c0c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c06dee0 0x7faa8c0c5fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa8000e8a0 tx=0x7faa8000ec60 comp rx=0 tx=0).stop 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 shutdown_connections 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa8c071040 0x7faa8c0b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa8c06cdc0 0x7faa8c157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7faa780682f0 0x7faa7806a7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 --2- 172.21.15.153:0/189724907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa8c06dee0 0x7faa8c0c5fe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 >> 172.21.15.153:0/189724907 conn(0x7faa8c0d0000 msgr2=0x7faa8c0cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 shutdown_connections 2023-11-28T13:01:08.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.038+0000 7faa92280700 1 -- 172.21.15.153:0/189724907 wait complete. 2023-11-28T13:01:08.051 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T13:01:08.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f5206b5e700 1 Processor -- start 2023-11-28T13:01:08.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f5206b5e700 1 -- start start 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f5206b5e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f520006ba90 0x7f52000af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f5206b5e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f52000cbdf0 0x7f52000ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f5206b5e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52000cc5f0 con 0x7f520015cef0 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f5206b5e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520015d470 con 0x7f520006ba90 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f5206b5e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520015d750 con 0x7f52000cbdf0 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.172+0000 7f51fffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f520006ba90 0x7f52000af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51fffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f520006ba90 0x7f52000af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39966/0 (socket says 172.21.15.153:39966) 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51fffff700 1 -- 172.21.15.153:0/2784542411 learned_addr learned my addr 172.21.15.153:0/2784542411 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f52050fb700 1 --2- 172.21.15.153:0/2784542411 >> v2:172.21.15.177:3300/0 conn(0x7f52000cbdf0 0x7f52000ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51fffff700 1 -- 172.21.15.153:0/2784542411 >> v2:172.21.15.177:3300/0 conn(0x7f52000cbdf0 msgr2=0x7f52000ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51fffff700 1 --2- 172.21.15.153:0/2784542411 >> v2:172.21.15.177:3300/0 conn(0x7f52000cbdf0 0x7f52000ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51fffff700 1 -- 172.21.15.153:0/2784542411 >> v1:172.21.15.177:6789/0 conn(0x7f520015cef0 legacy=0x7f520006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51fffff700 1 -- 172.21.15.153:0/2784542411 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520015da30 con 0x7f520006ba90 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51fffff700 1 --2- 172.21.15.153:0/2784542411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520006ba90 0x7f52000af400 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5200073b10 tx=0x7f51ec00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beacdee594142ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51ff7fe700 1 -- 172.21.15.153:0/2784542411 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00dcf0 con 0x7f520006ba90 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51ff7fe700 1 -- 172.21.15.153:0/2784542411 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00de50 con 0x7f520006ba90 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.173+0000 7f51ff7fe700 1 -- 172.21.15.153:0/2784542411 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec004250 con 0x7f520006ba90 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 -- 172.21.15.153:0/2784542411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520006ba90 msgr2=0x7f52000af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2784542411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520006ba90 0x7f52000af400 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5200073b10 tx=0x7f51ec00be70 comp rx=0 tx=0).stop 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 -- 172.21.15.153:0/2784542411 shutdown_connections 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2784542411 >> v2:172.21.15.177:3300/0 conn(0x7f52000cbdf0 0x7f52000ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2784542411 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f520006ba90 0x7f52000af400 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 -- 172.21.15.153:0/2784542411 >> 172.21.15.153:0/2784542411 conn(0x7f52000d0000 msgr2=0x7f52000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 -- 172.21.15.153:0/2784542411 shutdown_connections 2023-11-28T13:01:08.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.174+0000 7f5206b5e700 1 -- 172.21.15.153:0/2784542411 wait complete. 2023-11-28T13:01:08.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.175+0000 7f5206b5e700 1 Processor -- start 2023-11-28T13:01:08.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.175+0000 7f5206b5e700 1 -- start start 2023-11-28T13:01:08.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f5206b5e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f520006ba90 0x7f520015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f5206b5e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52000cbdf0 0x7f52000c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f5206b5e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f520015cef0 0x7f52000c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f5206b5e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52000c4350 con 0x7f52000cbdf0 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f5206b5e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52000c45a0 con 0x7f520006ba90 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f5206b5e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52000c4880 con 0x7f520015cef0 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52000cbdf0 0x7f52000c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52000cbdf0 0x7f52000c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39972/0 (socket says 172.21.15.153:39972) 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 -- 172.21.15.153:0/2829805983 learned_addr learned my addr 172.21.15.153:0/2829805983 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f52048fa700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f520006ba90 0x7f520015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 -- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f520015cef0 msgr2=0x7f52000c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f52050fb700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f520015cef0 0x7f52000c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f520015cef0 0x7f52000c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 -- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f520006ba90 msgr2=0x7f520015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f520006ba90 0x7f520015bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 -- 172.21.15.153:0/2829805983 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec00b7f0 con 0x7f52000cbdf0 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f52048fa700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f520006ba90 0x7f520015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f52050fb700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f520015cef0 0x7f52000c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.176+0000 7f51fffff700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52000cbdf0 0x7f52000c5360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00beb0 tx=0x7f51ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.177+0000 7f51fd7fa700 1 -- 172.21.15.153:0/2829805983 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec020070 con 0x7f52000cbdf0 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.177+0000 7f51fd7fa700 1 -- 172.21.15.153:0/2829805983 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec004030 con 0x7f52000cbdf0 2023-11-28T13:01:08.181 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.177+0000 7f51fd7fa700 1 -- 172.21.15.153:0/2829805983 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec010470 con 0x7f52000cbdf0 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.177+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520015dff0 con 0x7f52000cbdf0 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.177+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520015dd10 con 0x7f52000cbdf0 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.178+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52000cab90 con 0x7f52000cbdf0 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.178+0000 7f51fd7fa700 1 -- 172.21.15.153:0/2829805983 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f51ec0193f0 con 0x7f52000cbdf0 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.178+0000 7f51fd7fa700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51e40682f0 0x7f51e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.178+0000 7f51fd7fa700 1 -- 172.21.15.153:0/2829805983 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f51ec089020 con 0x7f52000cbdf0 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.178+0000 7f52048fa700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51e40682f0 0x7f51e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.179+0000 7f52048fa700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51e40682f0 0x7f51e406a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006fd0 tx=0x7f51f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.183+0000 7f51fd7fa700 1 -- 172.21.15.153:0/2829805983 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f51ec055b70 con 0x7f52000cbdf0 2023-11-28T13:01:08.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.438+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f52000d0e60 con 0x7f52000cbdf0 2023-11-28T13:01:08.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.442+0000 7f51fd7fa700 1 -- 172.21.15.153:0/2829805983 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f51ec059190 con 0x7f52000cbdf0 2023-11-28T13:01:08.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51e40682f0 msgr2=0x7f51e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51e40682f0 0x7f51e406a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006fd0 tx=0x7f51f4008040 comp rx=0 tx=0).stop 2023-11-28T13:01:08.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52000cbdf0 msgr2=0x7f52000c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52000cbdf0 0x7f52000c5360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00beb0 tx=0x7f51ec004750 comp rx=0 tx=0).stop 2023-11-28T13:01:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 shutdown_connections 2023-11-28T13:01:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f520015cef0 0x7f52000c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f520006ba90 0x7f520015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51e40682f0 0x7f51e406a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 --2- 172.21.15.153:0/2829805983 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52000cbdf0 0x7f52000c5360 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 >> 172.21.15.153:0/2829805983 conn(0x7f52000d0000 msgr2=0x7f52000aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 shutdown_connections 2023-11-28T13:01:08.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.444+0000 7f5206b5e700 1 -- 172.21.15.153:0/2829805983 wait complete. 2023-11-28T13:01:08.457 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T13:01:08.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.579+0000 7f8e2a798700 1 Processor -- start 2023-11-28T13:01:08.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.579+0000 7f8e2a798700 1 -- start start 2023-11-28T13:01:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e2a798700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e24071040 0x7f8e24071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e2a798700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8e2406dee0 0x7f8e2406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e2a798700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e2406cdc0 0x7f8e2406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e2a798700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e24071950 con 0x7f8e24071040 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e2a798700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2415d060 con 0x7f8e2406dee0 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e2a798700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2415d340 con 0x7f8e2406cdc0 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e24071040 0x7f8e24071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e24071040 0x7f8e24071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39978/0 (socket says 172.21.15.153:39978) 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 -- 172.21.15.153:0/1128947935 learned_addr learned my addr 172.21.15.153:0/1128947935 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e237fe700 1 --2- 172.21.15.153:0/1128947935 >> v2:172.21.15.177:3300/0 conn(0x7f8e2406dee0 0x7f8e2406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e28d35700 1 --2- 172.21.15.153:0/1128947935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e2406cdc0 0x7f8e2406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 -- 172.21.15.153:0/1128947935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e2406cdc0 msgr2=0x7f8e2406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 --2- 172.21.15.153:0/1128947935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e2406cdc0 0x7f8e2406d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 -- 172.21.15.153:0/1128947935 >> v2:172.21.15.177:3300/0 conn(0x7f8e2406dee0 msgr2=0x7f8e2406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 --2- 172.21.15.153:0/1128947935 >> v2:172.21.15.177:3300/0 conn(0x7f8e2406dee0 0x7f8e2406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 -- 172.21.15.153:0/1128947935 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2415d620 con 0x7f8e24071040 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e23fff700 1 --2- 172.21.15.153:0/1128947935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e24071040 0x7f8e24071410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406fb00 tx=0x7f8e18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6330c0151e63f30f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.580+0000 7f8e22ffd700 1 -- 172.21.15.153:0/1128947935 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800b870 con 0x7f8e24071040 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e22ffd700 1 -- 172.21.15.153:0/1128947935 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1800b9d0 con 0x7f8e24071040 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e22ffd700 1 -- 172.21.15.153:0/1128947935 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800bd60 con 0x7f8e24071040 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 -- 172.21.15.153:0/1128947935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e24071040 msgr2=0x7f8e24071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 --2- 172.21.15.153:0/1128947935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e24071040 0x7f8e24071410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406fb00 tx=0x7f8e18009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 -- 172.21.15.153:0/1128947935 shutdown_connections 2023-11-28T13:01:08.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 --2- 172.21.15.153:0/1128947935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e2406cdc0 0x7f8e2406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 --2- 172.21.15.153:0/1128947935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e24071040 0x7f8e24071410 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 --2- 172.21.15.153:0/1128947935 >> v2:172.21.15.177:3300/0 conn(0x7f8e2406dee0 0x7f8e2406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 -- 172.21.15.153:0/1128947935 >> 172.21.15.153:0/1128947935 conn(0x7f8e240d0000 msgr2=0x7f8e240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 -- 172.21.15.153:0/1128947935 shutdown_connections 2023-11-28T13:01:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.581+0000 7f8e2a798700 1 -- 172.21.15.153:0/1128947935 wait complete. 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.582+0000 7f8e2a798700 1 Processor -- start 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.582+0000 7f8e2a798700 1 -- start start 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e2a798700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e2406cdc0 0x7f8e24154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e2a798700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e2406dee0 0x7f8e24152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e2a798700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e24071040 0x7f8e240b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e2a798700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240c3aa0 con 0x7f8e2406cdc0 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e2a798700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240c3d80 con 0x7f8e2406dee0 2023-11-28T13:01:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e2a798700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240c4060 con 0x7f8e24071040 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e23fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e2406cdc0 0x7f8e24154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e23fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e2406cdc0 0x7f8e24154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39984/0 (socket says 172.21.15.153:39984) 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e23fff700 1 -- 172.21.15.153:0/3507829714 learned_addr learned my addr 172.21.15.153:0/3507829714 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e237fe700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e2406dee0 0x7f8e24152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.583+0000 7f8e28d35700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e24071040 0x7f8e240b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e23fff700 1 -- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e24071040 msgr2=0x7f8e240b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e23fff700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e24071040 0x7f8e240b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e23fff700 1 -- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e2406dee0 msgr2=0x7f8e24152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e23fff700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e2406dee0 0x7f8e24152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e23fff700 1 -- 172.21.15.153:0/3507829714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18009550 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e28d35700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e24071040 0x7f8e240b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e23fff700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e2406cdc0 0x7f8e24154420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e18014040 tx=0x7f8e18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e20ff9700 1 -- 172.21.15.153:0/3507829714 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18004310 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e20ff9700 1 -- 172.21.15.153:0/3507829714 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1801e070 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2406e870 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e20ff9700 1 -- 172.21.15.153:0/3507829714 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180036a0 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.584+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2415d900 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.585+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2415ccd0 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.585+0000 7f8e20ff9700 1 -- 172.21.15.153:0/3507829714 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8e18029020 con 0x7f8e2406cdc0 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.586+0000 7f8e20ff9700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e100682f0 0x7f8e1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.586+0000 7f8e20ff9700 1 -- 172.21.15.153:0/3507829714 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8e18086a20 con 0x7f8e2406cdc0 2023-11-28T13:01:08.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.586+0000 7f8e237fe700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e100682f0 0x7f8e1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.586+0000 7f8e237fe700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e100682f0 0x7f8e1006a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406d9b0 tx=0x7f8e14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.590+0000 7f8e20ff9700 1 -- 172.21.15.153:0/3507829714 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8e180535d0 con 0x7f8e2406cdc0 2023-11-28T13:01:08.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.852+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8e240c6d70 con 0x7f8e2406cdc0 2023-11-28T13:01:08.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.855+0000 7f8e20ff9700 1 -- 172.21.15.153:0/3507829714 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8e18056bf0 con 0x7f8e2406cdc0 2023-11-28T13:01:08.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e100682f0 msgr2=0x7f8e1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e100682f0 0x7f8e1006a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406d9b0 tx=0x7f8e14008040 comp rx=0 tx=0).stop 2023-11-28T13:01:08.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e2406cdc0 msgr2=0x7f8e24154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e2406cdc0 0x7f8e24154420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e18014040 tx=0x7f8e18004750 comp rx=0 tx=0).stop 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 shutdown_connections 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e100682f0 0x7f8e1006a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e2406cdc0 0x7f8e24154420 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e24071040 0x7f8e240b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 --2- 172.21.15.153:0/3507829714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e2406dee0 0x7f8e24152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 >> 172.21.15.153:0/3507829714 conn(0x7f8e240d0000 msgr2=0x7f8e2415e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 shutdown_connections 2023-11-28T13:01:08.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.858+0000 7f8e2a798700 1 -- 172.21.15.153:0/3507829714 wait complete. 2023-11-28T13:01:08.872 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T13:01:08.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.993+0000 7fbecb20d700 1 Processor -- start 2023-11-28T13:01:08.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.993+0000 7fbecb20d700 1 -- start start 2023-11-28T13:01:09.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbecb20d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbec41658c0 0x7fbec4167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbecb20d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec415d060 con 0x7fbec4071040 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbecb20d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec415d340 con 0x7fbec406dee0 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbecb20d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec4161550 con 0x7fbec41658c0 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec8fa9700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbec4071040 0x7fbec4071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58156/0 (socket says 172.21.15.153:58156) 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec8fa9700 1 -- 172.21.15.153:0/557369490 learned_addr learned my addr 172.21.15.153:0/557369490 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec97aa700 1 --2- 172.21.15.153:0/557369490 >> v2:172.21.15.177:3300/0 conn(0x7fbec41658c0 0x7fbec4167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364190008 0 0) 0x7fbec415d060 con 0x7fbec4071040 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb40035b0 con 0x7fbec4071040 2023-11-28T13:01:09.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589831370 0 0) 0x7fbeb40035b0 con 0x7fbec4071040 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 >> v2:172.21.15.177:3300/0 conn(0x7fbec41658c0 msgr2=0x7fbec4167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 --2- 172.21.15.153:0/557369490 >> v2:172.21.15.177:3300/0 conn(0x7fbec41658c0 0x7fbec4167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 >> v1:172.21.15.177:6789/0 conn(0x7fbec406dee0 legacy=0x7fbec406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec415d620 con 0x7fbec4071040 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fbeb8001670 con 0x7fbec4071040 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.994+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 >> v1:172.21.15.153:6789/0 conn(0x7fbec4071040 legacy=0x7fbec4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbeb4003a00 0x7fbeb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbeb4004850 0x7fbeb4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4005200 0x7fbeb400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4161550 con 0x7fbeb4003a00 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb400c190 con 0x7fbeb4005200 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb400c350 con 0x7fbeb4004850 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec97aa700 1 --2- 172.21.15.153:0/557369490 >> v2:172.21.15.177:3300/0 conn(0x7fbec41658c0 0x7fbec4167ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbeb4003a00 0x7fbeb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec97aa700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4005200 0x7fbeb400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec3fff700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbeb4004850 0x7fbeb4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec8fa9700 1 -- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbeb4004850 msgr2=0x7fbeb4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbeb4004850 0x7fbeb4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec8fa9700 1 -- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4005200 msgr2=0x7fbeb400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4005200 0x7fbeb400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec8fa9700 1 -- 172.21.15.153:0/557369490 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40035b0 con 0x7fbeb4003a00 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec97aa700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4005200 0x7fbeb400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec97aa700 1 -- 172.21.15.153:0/557369490 reap_dead start 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbeb4003a00 0x7fbeb4003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8001eb0 tx=0x7fbeb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776019ac2eec740f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800ce00 con 0x7fbeb4003a00 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb8005960 con 0x7fbeb4003a00 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.995+0000 7fbec37fe700 1 -- 172.21.15.153:0/557369490 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8005d10 con 0x7fbeb4003a00 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.996+0000 7fbecb20d700 1 -- 172.21.15.153:0/557369490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbeb4003a00 msgr2=0x7fbeb4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.996+0000 7fbecb20d700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbeb4003a00 0x7fbeb4003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8001eb0 tx=0x7fbeb4005880 comp rx=0 tx=0).stop 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.996+0000 7fbecb20d700 1 -- 172.21.15.153:0/557369490 shutdown_connections 2023-11-28T13:01:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.996+0000 7fbecb20d700 1 --2- 172.21.15.153:0/557369490 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbeb4003a00 0x7fbeb4003dd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.996+0000 7fbecb20d700 1 -- 172.21.15.153:0/557369490 >> 172.21.15.153:0/557369490 conn(0x7fbec40d0000 msgr2=0x7fbec40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.996+0000 7fbecb20d700 1 -- 172.21.15.153:0/557369490 shutdown_connections 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.996+0000 7fbecb20d700 1 -- 172.21.15.153:0/557369490 wait complete. 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.997+0000 7fbecb20d700 1 Processor -- start 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.997+0000 7fbecb20d700 1 -- start start 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbecb20d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4003a00 0x7fbec41452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbecb20d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbec4159cd0 0x7fbec4145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbecb20d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbec4146100 0x7fbec40b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbecb20d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40b05f0 con 0x7fbec4146100 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbecb20d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40afa00 con 0x7fbeb4003a00 2023-11-28T13:01:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbecb20d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40afcc0 con 0x7fbec4159cd0 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbec4146100 0x7fbec40b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbec4146100 0x7fbec40b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40008/0 (socket says 172.21.15.153:40008) 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 -- 172.21.15.153:0/2595290377 learned_addr learned my addr 172.21.15.153:0/2595290377 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4003a00 0x7fbec41452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 -- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbec4159cd0 msgr2=0x7fbec4145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec3fff700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbec4159cd0 0x7fbec4145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbec4159cd0 0x7fbec4145bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 -- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4003a00 msgr2=0x7fbec41452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4003a00 0x7fbec41452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 -- 172.21.15.153:0/2595290377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec415ccd0 con 0x7fbec4146100 2023-11-28T13:01:09.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4003a00 0x7fbec41452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.998+0000 7fbec97aa700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbec4146100 0x7fbec40b0a80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbec40c7060 tx=0x7fbebc0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.999+0000 7fbec17fa700 1 -- 172.21.15.153:0/2595290377 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00e070 con 0x7fbec4146100 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.999+0000 7fbec17fa700 1 -- 172.21.15.153:0/2595290377 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc011c80 con 0x7fbec4146100 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.999+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec415d900 con 0x7fbec4146100 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.999+0000 7fbec17fa700 1 -- 172.21.15.153:0/2595290377 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc011e10 con 0x7fbec4146100 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:08.999+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec406a270 con 0x7fbec4146100 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.000+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec406ee20 con 0x7fbec4146100 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.000+0000 7fbec17fa700 1 -- 172.21.15.153:0/2595290377 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbebc003a20 con 0x7fbec4146100 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.001+0000 7fbec17fa700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbea80682f0 0x7fbea806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.001+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbea80682f0 0x7fbea806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.001+0000 7fbec17fa700 1 -- 172.21.15.153:0/2595290377 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbebc085f80 con 0x7fbec4146100 2023-11-28T13:01:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.001+0000 7fbec8fa9700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbea80682f0 0x7fbea806a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8015040 tx=0x7fbeb8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.005+0000 7fbec17fa700 1 -- 172.21.15.153:0/2595290377 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbebc052ad0 con 0x7fbec4146100 2023-11-28T13:01:09.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.262+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fbec40d0530 con 0x7fbec4146100 2023-11-28T13:01:09.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.266+0000 7fbec17fa700 1 -- 172.21.15.153:0/2595290377 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbebc003cb0 con 0x7fbec4146100 2023-11-28T13:01:09.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbea80682f0 msgr2=0x7fbea806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbea80682f0 0x7fbea806a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8015040 tx=0x7fbeb8007000 comp rx=0 tx=0).stop 2023-11-28T13:01:09.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbec4146100 msgr2=0x7fbec40b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbec4146100 0x7fbec40b0a80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbec40c7060 tx=0x7fbebc0114b0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 shutdown_connections 2023-11-28T13:01:09.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbea80682f0 0x7fbea806a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbec4146100 0x7fbec40b0a80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbeb4003a00 0x7fbec41452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 --2- 172.21.15.153:0/2595290377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbec4159cd0 0x7fbec4145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 >> 172.21.15.153:0/2595290377 conn(0x7fbec40d0000 msgr2=0x7fbec406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 shutdown_connections 2023-11-28T13:01:09.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.268+0000 7fbecb20d700 1 -- 172.21.15.153:0/2595290377 wait complete. 2023-11-28T13:01:09.281 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T13:01:09.402 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.402+0000 7f18e6482700 1 Processor -- start 2023-11-28T13:01:09.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.402+0000 7f18e6482700 1 -- start start 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.402+0000 7f18e6482700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f18e0140950 0x7f18e00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.402+0000 7f18e6482700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b2e70 0x7f18e006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.402+0000 7f18e6482700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e013ff00 con 0x7f18e00cd8c0 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.402+0000 7f18e6482700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e01404c0 con 0x7f18e0140950 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.402+0000 7f18e6482700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e0140dc0 con 0x7f18e00b2e70 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18df7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f18e0140950 0x7f18e00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18dffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f18e00cd8c0 0x7f18e00cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58176/0 (socket says 172.21.15.153:58176) 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18df7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f18e0140950 0x7f18e00b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40014/0 (socket says 172.21.15.153:40014) 2023-11-28T13:01:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18dffff700 1 -- 172.21.15.153:0/86566371 learned_addr learned my addr 172.21.15.153:0/86566371 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18e4a1f700 1 --2- 172.21.15.153:0/86566371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b2e70 0x7f18e006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18df7fe700 1 -- 172.21.15.153:0/86566371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b2e70 msgr2=0x7f18e006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18df7fe700 1 --2- 172.21.15.153:0/86566371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b2e70 0x7f18e006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18df7fe700 1 -- 172.21.15.153:0/86566371 >> v1:172.21.15.153:6789/0 conn(0x7f18e00cd8c0 legacy=0x7f18e00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18df7fe700 1 -- 172.21.15.153:0/86566371 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e015aa80 con 0x7f18e0140950 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18df7fe700 1 --2- 172.21.15.153:0/86566371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e00b2930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009840 tx=0x7f18c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86719a5afed649c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18deffd700 1 -- 172.21.15.153:0/86566371 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011830 con 0x7f18e0140950 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18deffd700 1 -- 172.21.15.153:0/86566371 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8011990 con 0x7f18e0140950 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.403+0000 7f18deffd700 1 -- 172.21.15.153:0/86566371 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8011d20 con 0x7f18e0140950 2023-11-28T13:01:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 -- 172.21.15.153:0/86566371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 msgr2=0x7f18e00b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 --2- 172.21.15.153:0/86566371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e00b2930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009840 tx=0x7f18c8009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 -- 172.21.15.153:0/86566371 shutdown_connections 2023-11-28T13:01:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 --2- 172.21.15.153:0/86566371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00b2e70 0x7f18e006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 --2- 172.21.15.153:0/86566371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e00b2930 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 -- 172.21.15.153:0/86566371 >> 172.21.15.153:0/86566371 conn(0x7f18e006aed0 msgr2=0x7f18e006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 -- 172.21.15.153:0/86566371 shutdown_connections 2023-11-28T13:01:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.404+0000 7f18e6482700 1 -- 172.21.15.153:0/86566371 wait complete. 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.405+0000 7f18e6482700 1 Processor -- start 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.405+0000 7f18e6482700 1 -- start start 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e6482700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00b2e70 0x7f18e00b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e6482700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00cd8c0 0x7f18e0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e6482700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e01f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e6482700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e015af50 con 0x7f18e0140950 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e6482700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e0143d60 con 0x7f18e00b2e70 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e6482700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e015b1a0 con 0x7f18e00cd8c0 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e01f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e01f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40022/0 (socket says 172.21.15.153:40022) 2023-11-28T13:01:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 -- 172.21.15.153:0/3860530675 learned_addr learned my addr 172.21.15.153:0/3860530675 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18dffff700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00b2e70 0x7f18e00b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18df7fe700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00cd8c0 0x7f18e0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 -- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00cd8c0 msgr2=0x7f18e0143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00cd8c0 0x7f18e0143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 -- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00b2e70 msgr2=0x7f18e00b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00b2e70 0x7f18e00b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 -- 172.21.15.153:0/3860530675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c80094a0 con 0x7f18e0140950 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.406+0000 7f18e4a1f700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e01f21d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f18d8007e30 tx=0x7f18d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.407+0000 7f18dcff9700 1 -- 172.21.15.153:0/3860530675 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d800eea0 con 0x7f18e0140950 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.407+0000 7f18dcff9700 1 -- 172.21.15.153:0/3860530675 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8014420 con 0x7f18e0140950 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.407+0000 7f18dcff9700 1 -- 172.21.15.153:0/3860530675 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80147d0 con 0x7f18e0140950 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.407+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e00b35f0 con 0x7f18e0140950 2023-11-28T13:01:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.407+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e00b3310 con 0x7f18e0140950 2023-11-28T13:01:09.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.407+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e0140150 con 0x7f18e0140950 2023-11-28T13:01:09.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.408+0000 7f18dcff9700 1 -- 172.21.15.153:0/3860530675 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f18d8012050 con 0x7f18e0140950 2023-11-28T13:01:09.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.408+0000 7f18dcff9700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f18c00682f0 0x7f18c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.408+0000 7f18dcff9700 1 -- 172.21.15.153:0/3860530675 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f18d8085ff0 con 0x7f18e0140950 2023-11-28T13:01:09.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.408+0000 7f18dffff700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f18c00682f0 0x7f18c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.409+0000 7f18dffff700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f18c00682f0 0x7f18c006a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18d00010e0 tx=0x7f18d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.413+0000 7f18dcff9700 1 -- 172.21.15.153:0/3860530675 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f18d8052ba0 con 0x7f18e0140950 2023-11-28T13:01:09.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.673+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f18e00d0330 con 0x7f18e0140950 2023-11-28T13:01:09.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.677+0000 7f18dcff9700 1 -- 172.21.15.153:0/3860530675 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f18d80561c0 con 0x7f18e0140950 2023-11-28T13:01:09.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f18c00682f0 msgr2=0x7f18c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f18c00682f0 0x7f18c006a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18d00010e0 tx=0x7f18d0009040 comp rx=0 tx=0).stop 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 msgr2=0x7f18e01f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e01f21d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f18d8007e30 tx=0x7f18d800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 shutdown_connections 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f18c00682f0 0x7f18c006a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18e0140950 0x7f18e01f21d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e00b2e70 0x7f18e00b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 --2- 172.21.15.153:0/3860530675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18e00cd8c0 0x7f18e0143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 >> 172.21.15.153:0/3860530675 conn(0x7f18e006aed0 msgr2=0x7f18e00cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 shutdown_connections 2023-11-28T13:01:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.679+0000 7f18e6482700 1 -- 172.21.15.153:0/3860530675 wait complete. 2023-11-28T13:01:09.692 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:09.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.812+0000 7f5f761c8700 1 Processor -- start 2023-11-28T13:01:09.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.812+0000 7f5f761c8700 1 -- start start 2023-11-28T13:01:09.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.813+0000 7f5f761c8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 0x7f5f700cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.813+0000 7f5f761c8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700d1950 con 0x7f5f7015c9b0 2023-11-28T13:01:09.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.813+0000 7f5f761c8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700ce140 con 0x7f5f70161340 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.813+0000 7f5f761c8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b5800 con 0x7f5f700cd7c0 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.813+0000 7f5f6f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5f7015c9b0 0x7f5f700b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58190/0 (socket says 172.21.15.153:58190) 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.813+0000 7f5f6f7fe700 1 -- 172.21.15.153:0/1549229562 learned_addr learned my addr 172.21.15.153:0/1549229562 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6ffff700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 0x7f5f700cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698111949 0 0) 0x7f5f700d1950 con 0x7f5f7015c9b0 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f580035b0 con 0x7f5f7015c9b0 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017415754 0 0) 0x7f5f580035b0 con 0x7f5f7015c9b0 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 msgr2=0x7f5f700cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 0x7f5f700cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 >> v1:172.21.15.177:6789/0 conn(0x7f5f70161340 legacy=0x7f5f700d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700d24f0 con 0x7f5f7015c9b0 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f5f64001670 con 0x7f5f7015c9b0 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 >> v1:172.21.15.153:6789/0 conn(0x7f5f7015c9b0 legacy=0x7f5f700b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 0x7f5f58003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f58003f70 0x7f5f58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58004d90 0x7f5f58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b5800 con 0x7f5f58003f70 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5800baf0 con 0x7f5f58004d90 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5800bcb0 con 0x7f5f700cd7c0 2023-11-28T13:01:09.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6effd700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f58003f70 0x7f5f58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6f7fe700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 0x7f5f58003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6ffff700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58004d90 0x7f5f58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6effd700 1 -- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 msgr2=0x7f5f58003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6effd700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700cd7c0 0x7f5f58003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6effd700 1 -- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58004d90 msgr2=0x7f5f58005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6effd700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58004d90 0x7f5f58005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6effd700 1 -- 172.21.15.153:0/1549229562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f580035b0 con 0x7f5f58003f70 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.814+0000 7f5f6effd700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f58003f70 0x7f5f58004850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f60007fd0 tx=0x7f5f58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda108ead8203858 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000c660 con 0x7f5f58003f70 2023-11-28T13:01:09.820 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6000c7c0 con 0x7f5f58003f70 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f6e7fc700 1 -- 172.21.15.153:0/1549229562 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cb70 con 0x7f5f58003f70 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f761c8700 1 -- 172.21.15.153:0/1549229562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f58003f70 msgr2=0x7f5f58004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f761c8700 1 --2- 172.21.15.153:0/1549229562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f58003f70 0x7f5f58004850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5f60007fd0 tx=0x7f5f58005880 comp rx=0 tx=0).stop 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f6ffff700 1 -- 172.21.15.153:0/1549229562 reap_dead start 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f761c8700 1 -- 172.21.15.153:0/1549229562 shutdown_connections 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f761c8700 1 -- 172.21.15.153:0/1549229562 >> 172.21.15.153:0/1549229562 conn(0x7f5f70072600 msgr2=0x7f5f700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f761c8700 1 -- 172.21.15.153:0/1549229562 shutdown_connections 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.815+0000 7f5f761c8700 1 -- 172.21.15.153:0/1549229562 wait complete. 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.816+0000 7f5f761c8700 1 Processor -- start 2023-11-28T13:01:09.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f761c8700 1 -- start start 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f761c8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58003f70 0x7f5f70156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f761c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f700cca50 0x7f5f700b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f761c8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700b7770 0x7f5f700c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f761c8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b7c10 con 0x7f5f700cca50 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f761c8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700ca900 con 0x7f5f58003f70 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f761c8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70157040 con 0x7f5f700b7770 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f6effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f700cca50 0x7f5f700b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f6effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f700cca50 0x7f5f700b7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40040/0 (socket says 172.21.15.153:40040) 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.817+0000 7f5f6effd700 1 -- 172.21.15.153:0/3325662566 learned_addr learned my addr 172.21.15.153:0/3325662566 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6f7fe700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58003f70 0x7f5f70156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6ffff700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700b7770 0x7f5f700c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6effd700 1 -- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700b7770 msgr2=0x7f5f700c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6effd700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700b7770 0x7f5f700c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6effd700 1 -- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58003f70 msgr2=0x7f5f70156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6effd700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58003f70 0x7f5f70156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6effd700 1 -- 172.21.15.153:0/3325662566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700d1300 con 0x7f5f700cca50 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f6effd700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f700cca50 0x7f5f700b7230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f60001e90 tx=0x7f5f60001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f57fff700 1 -- 172.21.15.153:0/3325662566 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60000ed0 con 0x7f5f700cca50 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f57fff700 1 -- 172.21.15.153:0/3325662566 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60021070 con 0x7f5f700cca50 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f700b5520 con 0x7f5f700cca50 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f57fff700 1 -- 172.21.15.153:0/3325662566 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cb50 con 0x7f5f700cca50 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.818+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f700d15e0 con 0x7f5f700cca50 2023-11-28T13:01:09.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.819+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f700745c0 con 0x7f5f700cca50 2023-11-28T13:01:09.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.819+0000 7f5f57fff700 1 -- 172.21.15.153:0/3325662566 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f6002c020 con 0x7f5f700cca50 2023-11-28T13:01:09.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.820+0000 7f5f57fff700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f5c0682f0 0x7f5f5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.820+0000 7f5f57fff700 1 -- 172.21.15.153:0/3325662566 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5f60086b80 con 0x7f5f700cca50 2023-11-28T13:01:09.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.820+0000 7f5f6f7fe700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f5c0682f0 0x7f5f5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.820+0000 7f5f6f7fe700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f5c0682f0 0x7f5f5c06a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5f701f5d70 tx=0x7f5f64000fe0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:09.824+0000 7f5f57fff700 1 -- 172.21.15.153:0/3325662566 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f60053730 con 0x7f5f700cca50 2023-11-28T13:01:10.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.005+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f700d08a0 con 0x7f5f700cca50 2023-11-28T13:01:10.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.006+0000 7f5f57fff700 1 -- 172.21.15.153:0/3325662566 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+14315 (secure 0 0 0) 0x7f5f60056d50 con 0x7f5f700cca50 2023-11-28T13:01:10.007 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:10.007 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":35,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:06.801182+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:10.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f5c0682f0 msgr2=0x7f5f5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f5c0682f0 0x7f5f5c06a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5f701f5d70 tx=0x7f5f64000fe0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f700cca50 msgr2=0x7f5f700b7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f700cca50 0x7f5f700b7230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5f60001e90 tx=0x7f5f60001230 comp rx=0 tx=0).stop 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 shutdown_connections 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f700b7770 0x7f5f700c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f58003f70 0x7f5f70156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f5c0682f0 0x7f5f5c06a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 --2- 172.21.15.153:0/3325662566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5f700cca50 0x7f5f700b7230 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 >> 172.21.15.153:0/3325662566 conn(0x7f5f70072600 msgr2=0x7f5f7015b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 shutdown_connections 2023-11-28T13:01:10.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.008+0000 7f5f761c8700 1 -- 172.21.15.153:0/3325662566 wait complete. 2023-11-28T13:01:10.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:01:10.022 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:10.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.144+0000 7fb83c24c700 1 Processor -- start 2023-11-28T13:01:10.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.145+0000 7fb83c24c700 1 -- start start 2023-11-28T13:01:10.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.145+0000 7fb83c24c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb834159860 0x7fb8340c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.145+0000 7fb83c24c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8340ccc00 con 0x7fb8340b0d80 2023-11-28T13:01:10.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.145+0000 7fb83c24c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8340ccec0 con 0x7fb834159490 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.145+0000 7fb83c24c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340cd1a0 con 0x7fb834159860 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb83a7e9700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb8340b0d80 0x7fb8340c9a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58214/0 (socket says 172.21.15.153:58214) 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb83a7e9700 1 -- 172.21.15.153:0/2687983559 learned_addr learned my addr 172.21.15.153:0/2687983559 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb8397e7700 1 --2- 172.21.15.153:0/2687983559 >> v2:172.21.15.177:3300/0 conn(0x7fb834159860 0x7fb8340c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128447782 0 0) 0x7fb8340ccc00 con 0x7fb8340b0d80 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8200035b0 con 0x7fb8340b0d80 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1199712220 0 0) 0x7fb8200035b0 con 0x7fb8340b0d80 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 >> v2:172.21.15.177:3300/0 conn(0x7fb834159860 msgr2=0x7fb8340c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 --2- 172.21.15.153:0/2687983559 >> v2:172.21.15.177:3300/0 conn(0x7fb834159860 0x7fb8340c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 >> v1:172.21.15.177:6789/0 conn(0x7fb834159490 legacy=0x7fb8340b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340cd480 con 0x7fb8340b0d80 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fb82c0019f0 con 0x7fb8340b0d80 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 >> v1:172.21.15.153:6789/0 conn(0x7fb8340b0d80 legacy=0x7fb8340c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb820003a00 0x7fb820003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820004850 0x7fb820004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb820005200 0x7fb82000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340cd1a0 con 0x7fb820004850 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82000c080 con 0x7fb820003a00 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82000c240 con 0x7fb820005200 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb839fe8700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb820003a00 0x7fb820003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb8397e7700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820004850 0x7fb820004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.146+0000 7fb83a7e9700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb820005200 0x7fb82000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb839fe8700 1 -- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb820005200 msgr2=0x7fb82000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb839fe8700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb820005200 0x7fb82000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb839fe8700 1 -- 172.21.15.153:0/2687983559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820004850 msgr2=0x7fb820004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb839fe8700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820004850 0x7fb820004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb839fe8700 1 -- 172.21.15.153:0/2687983559 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8200035b0 con 0x7fb820003a00 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb8397e7700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820004850 0x7fb820004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb83a7e9700 1 -- 172.21.15.153:0/2687983559 reap_dead start 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb839fe8700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb820003a00 0x7fb820003e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8300017c0 tx=0x7fb820005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a03ff2e28bc8ce8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8300052d0 con 0x7fb820003a00 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830005430 con 0x7fb820003a00 2023-11-28T13:01:10.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.147+0000 7fb838fe6700 1 -- 172.21.15.153:0/2687983559 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8300038c0 con 0x7fb820003a00 2023-11-28T13:01:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.148+0000 7fb83c24c700 1 -- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb820003a00 msgr2=0x7fb820003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.148+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb820003a00 0x7fb820003e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8300017c0 tx=0x7fb820005880 comp rx=0 tx=0).stop 2023-11-28T13:01:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.148+0000 7fb83c24c700 1 -- 172.21.15.153:0/2687983559 shutdown_connections 2023-11-28T13:01:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.148+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2687983559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb820003a00 0x7fb820003e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.148+0000 7fb83c24c700 1 -- 172.21.15.153:0/2687983559 >> 172.21.15.153:0/2687983559 conn(0x7fb8340cb670 msgr2=0x7fb8340cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.148+0000 7fb83c24c700 1 -- 172.21.15.153:0/2687983559 shutdown_connections 2023-11-28T13:01:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.148+0000 7fb83c24c700 1 -- 172.21.15.153:0/2687983559 wait complete. 2023-11-28T13:01:10.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.149+0000 7fb83c24c700 1 Processor -- start 2023-11-28T13:01:10.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.149+0000 7fb83c24c700 1 -- start start 2023-11-28T13:01:10.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb83c24c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820003a00 0x7fb8340c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb83c24c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb834153020 0x7fb83415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb83c24c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb83415d940 0x7fb8340c1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb83c24c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83415ddb0 con 0x7fb820003a00 2023-11-28T13:01:10.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb83c24c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340bf850 con 0x7fb83415d940 2023-11-28T13:01:10.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb83c24c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340bfb30 con 0x7fb834153020 2023-11-28T13:01:10.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820003a00 0x7fb8340c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820003a00 0x7fb8340c08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40058/0 (socket says 172.21.15.153:40058) 2023-11-28T13:01:10.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 -- 172.21.15.153:0/2032060728 learned_addr learned my addr 172.21.15.153:0/2032060728 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:10.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb8397e7700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb834153020 0x7fb83415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 -- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb834153020 msgr2=0x7fb83415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb83a7e9700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb83415d940 0x7fb8340c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb834153020 0x7fb83415d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 -- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb83415d940 msgr2=0x7fb8340c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb83415d940 0x7fb8340c1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.150+0000 7fb839fe8700 1 -- 172.21.15.153:0/2032060728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340cdc80 con 0x7fb820003a00 2023-11-28T13:01:10.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.151+0000 7fb839fe8700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820003a00 0x7fb8340c08f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb830015040 tx=0x7fb830001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.151+0000 7fb82a7fc700 1 -- 172.21.15.153:0/2032060728 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830001290 con 0x7fb820003a00 2023-11-28T13:01:10.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.151+0000 7fb82a7fc700 1 -- 172.21.15.153:0/2032060728 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83001d480 con 0x7fb820003a00 2023-11-28T13:01:10.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.151+0000 7fb82a7fc700 1 -- 172.21.15.153:0/2032060728 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83001d830 con 0x7fb820003a00 2023-11-28T13:01:10.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.151+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8340cd760 con 0x7fb820003a00 2023-11-28T13:01:10.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.151+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8340cde10 con 0x7fb820003a00 2023-11-28T13:01:10.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.152+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83415dd10 con 0x7fb820003a00 2023-11-28T13:01:10.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.152+0000 7fb82a7fc700 1 -- 172.21.15.153:0/2032060728 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb83002f070 con 0x7fb820003a00 2023-11-28T13:01:10.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.153+0000 7fb82a7fc700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8180682f0 0x7fb81806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.153+0000 7fb8397e7700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8180682f0 0x7fb81806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.153+0000 7fb82a7fc700 1 -- 172.21.15.153:0/2032060728 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb830086670 con 0x7fb820003a00 2023-11-28T13:01:10.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.153+0000 7fb8397e7700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8180682f0 0x7fb81806a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8340b0770 tx=0x7fb824008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.157+0000 7fb82a7fc700 1 -- 172.21.15.153:0/2032060728 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb830053220 con 0x7fb820003a00 2023-11-28T13:01:10.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.351+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8340745c0 con 0x7fb820003a00 2023-11-28T13:01:10.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.351+0000 7fb82a7fc700 1 -- 172.21.15.153:0/2032060728 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb83002d020 con 0x7fb820003a00 2023-11-28T13:01:10.352 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8180682f0 msgr2=0x7fb81806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8180682f0 0x7fb81806a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8340b0770 tx=0x7fb824008040 comp rx=0 tx=0).stop 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820003a00 msgr2=0x7fb8340c08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820003a00 0x7fb8340c08f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb830015040 tx=0x7fb830001120 comp rx=0 tx=0).stop 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 shutdown_connections 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8180682f0 0x7fb81806a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb820003a00 0x7fb8340c08f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb83415d940 0x7fb8340c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 --2- 172.21.15.153:0/2032060728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb834153020 0x7fb83415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 >> 172.21.15.153:0/2032060728 conn(0x7fb8340cb670 msgr2=0x7fb83415a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 shutdown_connections 2023-11-28T13:01:10.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.353+0000 7fb83c24c700 1 -- 172.21.15.153:0/2032060728 wait complete. 2023-11-28T13:01:10.366 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:10.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.489+0000 7f49d5c98700 1 Processor -- start 2023-11-28T13:01:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.489+0000 7f49d5c98700 1 -- start start 2023-11-28T13:01:10.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.489+0000 7f49d5c98700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 0x7f49d0140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.496 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.489+0000 7f49d5c98700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d013ff00 con 0x7f49d006f850 2023-11-28T13:01:10.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.489+0000 7f49d5c98700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d01404c0 con 0x7f49d00cdc90 2023-11-28T13:01:10.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.489+0000 7f49d5c98700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015aa80 con 0x7f49d00cd8c0 2023-11-28T13:01:10.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49cffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f49d006f850 0x7f49d006fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58234/0 (socket says 172.21.15.153:58234) 2023-11-28T13:01:10.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49cffff700 1 -- 172.21.15.153:0/2252758777 learned_addr learned my addr 172.21.15.153:0/2252758777 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:10.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49cf7fe700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 0x7f49d0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275339021 0 0) 0x7f49d013ff00 con 0x7f49d006f850 2023-11-28T13:01:10.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49bc0035b0 con 0x7f49d006f850 2023-11-28T13:01:10.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 437603246 0 0) 0x7f49bc0035b0 con 0x7f49d006f850 2023-11-28T13:01:10.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 msgr2=0x7f49d0140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 0x7f49d0140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 >> v1:172.21.15.177:6789/0 conn(0x7f49d00cdc90 legacy=0x7f49d00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00b3040 con 0x7f49d006f850 2023-11-28T13:01:10.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f49c8001670 con 0x7f49d006f850 2023-11-28T13:01:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 >> v1:172.21.15.153:6789/0 conn(0x7f49d006f850 legacy=0x7f49d006fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 0x7f49bc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc003f70 0x7f49bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49bc004d90 0x7f49bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015aa80 con 0x7f49bc004d90 2023-11-28T13:01:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc00baf0 con 0x7f49bc003f70 2023-11-28T13:01:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc00bcb0 con 0x7f49d00cd8c0 2023-11-28T13:01:10.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49cffff700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49bc004d90 0x7f49bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49cf7fe700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 0x7f49bc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.490+0000 7f49ceffd700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc003f70 0x7f49bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49cffff700 1 -- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 msgr2=0x7f49bc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49cffff700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00cd8c0 0x7f49bc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.500 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49cffff700 1 -- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc003f70 msgr2=0x7f49bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49cffff700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc003f70 0x7f49bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49cffff700 1 -- 172.21.15.153:0/2252758777 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0035b0 con 0x7f49bc004d90 2023-11-28T13:01:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49ceffd700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49bc003f70 0x7f49bc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49cffff700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49bc004d90 0x7f49bc005200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f49c8001eb0 tx=0x7f49bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9f3733ab45edfe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800ce00 con 0x7f49bc004d90 2023-11-28T13:01:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c8005960 con 0x7f49bc004d90 2023-11-28T13:01:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49ce7fc700 1 -- 172.21.15.153:0/2252758777 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8005d10 con 0x7f49bc004d90 2023-11-28T13:01:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49d5c98700 1 -- 172.21.15.153:0/2252758777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49bc004d90 msgr2=0x7f49bc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49d5c98700 1 --2- 172.21.15.153:0/2252758777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49bc004d90 0x7f49bc005200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f49c8001eb0 tx=0x7f49bc005880 comp rx=0 tx=0).stop 2023-11-28T13:01:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.491+0000 7f49cffff700 1 -- 172.21.15.153:0/2252758777 reap_dead start 2023-11-28T13:01:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.492+0000 7f49d5c98700 1 -- 172.21.15.153:0/2252758777 shutdown_connections 2023-11-28T13:01:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.492+0000 7f49d5c98700 1 -- 172.21.15.153:0/2252758777 >> 172.21.15.153:0/2252758777 conn(0x7f49d006aed0 msgr2=0x7f49d006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.492+0000 7f49d5c98700 1 -- 172.21.15.153:0/2252758777 shutdown_connections 2023-11-28T13:01:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.492+0000 7f49d5c98700 1 -- 172.21.15.153:0/2252758777 wait complete. 2023-11-28T13:01:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.493+0000 7f49d5c98700 1 Processor -- start 2023-11-28T13:01:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.493+0000 7f49d5c98700 1 -- start start 2023-11-28T13:01:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49d5c98700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d01efb90 0x7f49d00c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49d5c98700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d00b1c80 0x7f49d00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49d5c98700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00c67e0 0x7f49d00c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49d5c98700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00b2630 con 0x7f49d00b1c80 2023-11-28T13:01:10.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49d5c98700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00c6060 con 0x7f49d01efb90 2023-11-28T13:01:10.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49d5c98700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00c4a80 con 0x7f49d00c67e0 2023-11-28T13:01:10.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d00b1c80 0x7f49d00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d00b1c80 0x7f49d00b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40080/0 (socket says 172.21.15.153:40080) 2023-11-28T13:01:10.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 -- 172.21.15.153:0/4223500731 learned_addr learned my addr 172.21.15.153:0/4223500731 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49cf7fe700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d01efb90 0x7f49d00c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49cffff700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00c67e0 0x7f49d00c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 -- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00c67e0 msgr2=0x7f49d00c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00c67e0 0x7f49d00c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 -- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d01efb90 msgr2=0x7f49d00c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d01efb90 0x7f49d00c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 -- 172.21.15.153:0/4223500731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00b35f0 con 0x7f49d00b1c80 2023-11-28T13:01:10.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49cf7fe700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d01efb90 0x7f49d00c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:10.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49ceffd700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d00b1c80 0x7f49d00b20f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f49b800fb30 tx=0x7f49b800fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49c7fff700 1 -- 172.21.15.153:0/4223500731 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8001d10 con 0x7f49d00b1c80 2023-11-28T13:01:10.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49c7fff700 1 -- 172.21.15.153:0/4223500731 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b80052e0 con 0x7f49d00b1c80 2023-11-28T13:01:10.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49c7fff700 1 -- 172.21.15.153:0/4223500731 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8003680 con 0x7f49d00b1c80 2023-11-28T13:01:10.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.494+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d00b3310 con 0x7f49d00b1c80 2023-11-28T13:01:10.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.495+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d0140150 con 0x7f49d00b1c80 2023-11-28T13:01:10.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.495+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d0068690 con 0x7f49d00b1c80 2023-11-28T13:01:10.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.496+0000 7f49c7fff700 1 -- 172.21.15.153:0/4223500731 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f49b80037e0 con 0x7f49d00b1c80 2023-11-28T13:01:10.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.496+0000 7f49c7fff700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49b00682f0 0x7f49b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.496+0000 7f49c7fff700 1 -- 172.21.15.153:0/4223500731 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f49b8085b40 con 0x7f49d00b1c80 2023-11-28T13:01:10.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.496+0000 7f49cf7fe700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49b00682f0 0x7f49b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.497+0000 7f49cf7fe700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49b00682f0 0x7f49b006a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011830 tx=0x7f49c000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.500+0000 7f49c7fff700 1 -- 172.21.15.153:0/4223500731 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f49b8052570 con 0x7f49d00b1c80 2023-11-28T13:01:10.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.683+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f49d0070190 con 0x7f49d00b1c80 2023-11-28T13:01:10.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.684+0000 7f49c7fff700 1 -- 172.21.15.153:0/4223500731 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f49b8085e60 con 0x7f49d00b1c80 2023-11-28T13:01:10.685 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:01:10.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.685+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49b00682f0 msgr2=0x7f49b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.685+0000 7f49d5c98700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49b00682f0 0x7f49b006a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49c0011830 tx=0x7f49c000b040 comp rx=0 tx=0).stop 2023-11-28T13:01:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.685+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d00b1c80 msgr2=0x7f49d00b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d00b1c80 0x7f49d00b20f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f49b800fb30 tx=0x7f49b800fe40 comp rx=0 tx=0).stop 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 shutdown_connections 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49d00c67e0 0x7f49d00c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d01efb90 0x7f49d00c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f49b00682f0 0x7f49b006a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 --2- 172.21.15.153:0/4223500731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49d00b1c80 0x7f49d00b20f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 >> 172.21.15.153:0/4223500731 conn(0x7f49d006aed0 msgr2=0x7f49d00cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 shutdown_connections 2023-11-28T13:01:10.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.686+0000 7f49d5c98700 1 -- 172.21.15.153:0/4223500731 wait complete. 2023-11-28T13:01:10.699 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:01:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.818+0000 7f7c6ca0a700 1 Processor -- start 2023-11-28T13:01:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.818+0000 7f7c6ca0a700 1 -- start start 2023-11-28T13:01:10.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c6ca0a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680ce040 0x7f7c6815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c6ca0a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c680b2930 0x7f7c680b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c6ca0a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6806b420 con 0x7f7c680d0a00 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c6ca0a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c680b32f0 con 0x7f7c680b2930 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c6ca0a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6806ff60 con 0x7f7c680ce040 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c66d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c680b2930 0x7f7c680b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c66d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7c680b2930 0x7f7c680b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40090/0 (socket says 172.21.15.153:40090) 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.819+0000 7f7c66d9d700 1 -- 172.21.15.153:0/2324265451 learned_addr learned my addr 172.21.15.153:0/2324265451 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c6659c700 1 --2- 172.21.15.153:0/2324265451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680ce040 0x7f7c6815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c66d9d700 1 -- 172.21.15.153:0/2324265451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680ce040 msgr2=0x7f7c6815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c66d9d700 1 --2- 172.21.15.153:0/2324265451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680ce040 0x7f7c6815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c66d9d700 1 -- 172.21.15.153:0/2324265451 >> v1:172.21.15.177:6789/0 conn(0x7f7c680d0a00 legacy=0x7f7c6806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c66d9d700 1 -- 172.21.15.153:0/2324265451 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6806b700 con 0x7f7c680b2930 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c66d9d700 1 --2- 172.21.15.153:0/2324265451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b2db0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009e50 tx=0x7f7c6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885a8d44c0793ab5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c6559a700 1 -- 172.21.15.153:0/2324265451 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000e070 con 0x7f7c680b2930 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c6559a700 1 -- 172.21.15.153:0/2324265451 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6000b9c0 con 0x7f7c680b2930 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c6559a700 1 -- 172.21.15.153:0/2324265451 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000bd70 con 0x7f7c680b2930 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/2324265451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 msgr2=0x7f7c680b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.820+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/2324265451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b2db0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009e50 tx=0x7f7c6000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.821+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/2324265451 shutdown_connections 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.821+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/2324265451 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b2db0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.821+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/2324265451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680ce040 0x7f7c6815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.823 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.821+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/2324265451 >> 172.21.15.153:0/2324265451 conn(0x7f7c6806aed0 msgr2=0x7f7c6806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.821+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/2324265451 shutdown_connections 2023-11-28T13:01:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.821+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/2324265451 wait complete. 2023-11-28T13:01:10.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.822+0000 7f7c6ca0a700 1 Processor -- start 2023-11-28T13:01:10.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.822+0000 7f7c6ca0a700 1 -- start start 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6ca0a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6ca0a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c680ce040 0x7f7c6814cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6ca0a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680d0a00 0x7f7c680c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6ca0a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b65b0 con 0x7f7c680b2930 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6ca0a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680c9cd0 con 0x7f7c680ce040 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6ca0a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b4560 con 0x7f7c680d0a00 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40092/0 (socket says 172.21.15.153:40092) 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 -- 172.21.15.153:0/936318507 learned_addr learned my addr 172.21.15.153:0/936318507 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c65d9b700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c680ce040 0x7f7c6814cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c66d9d700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680d0a00 0x7f7c680c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 -- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680d0a00 msgr2=0x7f7c680c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680d0a00 0x7f7c680c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 -- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c680ce040 msgr2=0x7f7c6814cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c680ce040 0x7f7c6814cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 -- 172.21.15.153:0/936318507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60009b30 con 0x7f7c680b2930 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c66d9d700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680d0a00 0x7f7c680c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c6659c700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b4020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00d7d0 tx=0x7f7c5c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.823+0000 7f7c56ffd700 1 -- 172.21.15.153:0/936318507 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0125f0 con 0x7f7c680b2930 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.824+0000 7f7c56ffd700 1 -- 172.21.15.153:0/936318507 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c012750 con 0x7f7c680b2930 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.824+0000 7f7c56ffd700 1 -- 172.21.15.153:0/936318507 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c012b00 con 0x7f7c680b2930 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.824+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6806bcc0 con 0x7f7c680b2930 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.824+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6806b9e0 con 0x7f7c680b2930 2023-11-28T13:01:10.828 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.825+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6806bfa0 con 0x7f7c680b2930 2023-11-28T13:01:10.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.825+0000 7f7c56ffd700 1 -- 172.21.15.153:0/936318507 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7c5c012c60 con 0x7f7c680b2930 2023-11-28T13:01:10.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.825+0000 7f7c56ffd700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c480682f0 0x7f7c4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.825+0000 7f7c65d9b700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c480682f0 0x7f7c4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.825+0000 7f7c56ffd700 1 -- 172.21.15.153:0/936318507 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7c5c085a20 con 0x7f7c680b2930 2023-11-28T13:01:10.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.826+0000 7f7c65d9b700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c480682f0 0x7f7c4806a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7c680cf3e0 tx=0x7f7c58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:10.830+0000 7f7c56ffd700 1 -- 172.21.15.153:0/936318507 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7c5c0525d0 con 0x7f7c680b2930 2023-11-28T13:01:11.013 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.013+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7c680c52e0 con 0x7f7c680b2930 2023-11-28T13:01:11.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.013+0000 7f7c56ffd700 1 -- 172.21.15.153:0/936318507 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7c5c055bf0 con 0x7f7c680b2930 2023-11-28T13:01:11.014 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:11.014 INFO:teuthology.orchestra.run.smithi153.stdout:[ 2023-11-28T13:01:11.014 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:11.014 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 1, 2023-11-28T13:01:11.014 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": ".mgr" 2023-11-28T13:01:11.015 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:11.015 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:11.015 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 2, 2023-11-28T13:01:11.015 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": "rbd" 2023-11-28T13:01:11.015 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:11.015 INFO:teuthology.orchestra.run.smithi153.stdout:] 2023-11-28T13:01:11.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c480682f0 msgr2=0x7f7c4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c480682f0 0x7f7c4806a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7c680cf3e0 tx=0x7f7c58009040 comp rx=0 tx=0).stop 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 msgr2=0x7f7c680b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b4020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00d7d0 tx=0x7f7c5c00dae0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 shutdown_connections 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7c480682f0 0x7f7c4806a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c680b2930 0x7f7c680b4020 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c680d0a00 0x7f7c680c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 --2- 172.21.15.153:0/936318507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c680ce040 0x7f7c6814cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 >> 172.21.15.153:0/936318507 conn(0x7f7c6806aed0 msgr2=0x7f7c6815c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 shutdown_connections 2023-11-28T13:01:11.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.015+0000 7f7c6ca0a700 1 -- 172.21.15.153:0/936318507 wait complete. 2023-11-28T13:01:11.028 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:01:11.028 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:01:11.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.152+0000 7f906070c700 1 Processor -- start 2023-11-28T13:01:11.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.152+0000 7f906070c700 1 -- start start 2023-11-28T13:01:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f906070c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af060 0x7f9058070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f906070c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f905806a980 0x7f905806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f906070c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90580713c0 con 0x7f90580af430 2023-11-28T13:01:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f906070c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905806dee0 con 0x7f905806a980 2023-11-28T13:01:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f906070c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580716a0 con 0x7f90580af060 2023-11-28T13:01:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905e4a8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af060 0x7f9058070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905eca9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f905806a980 0x7f905806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905e4a8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af060 0x7f9058070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:57710/0 (socket says 172.21.15.153:57710) 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905dca7700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f90580af430 0x7f905806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:34878/0 (socket says 172.21.15.153:34878) 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905e4a8700 1 -- 172.21.15.153:0/302234119 learned_addr learned my addr 172.21.15.153:0/302234119 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905eca9700 1 -- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af060 msgr2=0x7f9058070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905eca9700 1 --2- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af060 0x7f9058070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905eca9700 1 -- 172.21.15.153:0/302234119 >> v1:172.21.15.177:6789/0 conn(0x7f90580af430 legacy=0x7f905806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905eca9700 1 -- 172.21.15.153:0/302234119 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058071980 con 0x7f905806a980 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905d4a6700 1 -- 172.21.15.153:0/302234119 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554098843 0 0) 0x7f90580713c0 con 0x7f90580af430 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.153+0000 7f905e4a8700 1 --2- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af060 0x7f9058070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f905eca9700 1 --2- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f905806a980 0x7f905806d9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9050009d90 tx=0x7f905000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8544aa86291e30fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f905d4a6700 1 -- 172.21.15.153:0/302234119 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000e070 con 0x7f905806a980 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f905d4a6700 1 -- 172.21.15.153:0/302234119 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905000bb20 con 0x7f905806a980 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f905d4a6700 1 -- 172.21.15.153:0/302234119 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050012430 con 0x7f905806a980 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 -- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f905806a980 msgr2=0x7f905806d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 --2- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f905806a980 0x7f905806d9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9050009d90 tx=0x7f905000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 -- 172.21.15.153:0/302234119 shutdown_connections 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 --2- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f905806a980 0x7f905806d9a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 --2- 172.21.15.153:0/302234119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af060 0x7f9058070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 -- 172.21.15.153:0/302234119 >> 172.21.15.153:0/302234119 conn(0x7f905806b380 msgr2=0x7f905806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 -- 172.21.15.153:0/302234119 shutdown_connections 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.154+0000 7f906070c700 1 -- 172.21.15.153:0/302234119 wait complete. 2023-11-28T13:01:11.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.155+0000 7f906070c700 1 Processor -- start 2023-11-28T13:01:11.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f906070c700 1 -- start start 2023-11-28T13:01:11.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f906070c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f905806a980 0x7f90580c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f906070c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90580af060 0x7f90580b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f906070c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af430 0x7f905814f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f906070c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c8100 con 0x7f905806a980 2023-11-28T13:01:11.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f906070c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580b5ac0 con 0x7f90580af060 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f906070c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c74f0 con 0x7f90580af430 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f905e4a8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f905806a980 0x7f90580c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f905e4a8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f905806a980 0x7f90580c6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40106/0 (socket says 172.21.15.153:40106) 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f905e4a8700 1 -- 172.21.15.153:0/2383109191 learned_addr learned my addr 172.21.15.153:0/2383109191 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f905dca7700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90580af060 0x7f90580b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.156+0000 7f905eca9700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af430 0x7f905814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f905e4a8700 1 -- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af430 msgr2=0x7f905814f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f905e4a8700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af430 0x7f905814f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f905e4a8700 1 -- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90580af060 msgr2=0x7f90580b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f905e4a8700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90580af060 0x7f90580b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f905e4a8700 1 -- 172.21.15.153:0/2383109191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9054009550 con 0x7f905806a980 2023-11-28T13:01:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f905e4a8700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f905806a980 0x7f90580c6ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f905400ebe0 tx=0x7f905400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f904effd700 1 -- 172.21.15.153:0/2383109191 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905400c850 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f904effd700 1 -- 172.21.15.153:0/2383109191 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905400c9b0 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9050009a70 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f904effd700 1 -- 172.21.15.153:0/2383109191 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905400cd40 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.157+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9058071f40 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.158+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9058071c60 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.158+0000 7f904effd700 1 -- 172.21.15.153:0/2383109191 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9054023070 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.159+0000 7f904effd700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f903c068320 0x7f903c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.159+0000 7f904effd700 1 -- 172.21.15.153:0/2383109191 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9054085dd0 con 0x7f905806a980 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.159+0000 7f905dca7700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f903c068320 0x7f903c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.159+0000 7f905dca7700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f903c068320 0x7f903c06a7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90581e54e0 tx=0x7f9048007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.163+0000 7f904effd700 1 -- 172.21.15.153:0/2383109191 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f905408a070 con 0x7f905806a980 2023-11-28T13:01:11.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.346+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f90580ca660 con 0x7f905806a980 2023-11-28T13:01:11.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.680+0000 7f904effd700 1 -- 172.21.15.153:0/2383109191 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9054052810 con 0x7f905806a980 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f903c068320 msgr2=0x7f903c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f903c068320 0x7f903c06a7d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90581e54e0 tx=0x7f9048007de0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f905806a980 msgr2=0x7f90580c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f905806a980 0x7f90580c6ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f905400ebe0 tx=0x7f905400eef0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 shutdown_connections 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f903c068320 0x7f903c06a7d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f905806a980 0x7f90580c6ea0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90580af430 0x7f905814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 --2- 172.21.15.153:0/2383109191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90580af060 0x7f90580b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 >> 172.21.15.153:0/2383109191 conn(0x7f905806b380 msgr2=0x7f90580cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 shutdown_connections 2023-11-28T13:01:11.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.683+0000 7f906070c700 1 -- 172.21.15.153:0/2383109191 wait complete. 2023-11-28T13:01:11.685 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' created 2023-11-28T13:01:11.699 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:01:11.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.825+0000 7f2ba88bf700 1 Processor -- start 2023-11-28T13:01:11.827 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.825+0000 7f2ba88bf700 1 -- start start 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.826+0000 7f2ba88bf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.826+0000 7f2ba88bf700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ba40cdc90 0x7f2ba40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.826+0000 7f2ba88bf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2ba40b2e70 0x7f2ba406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.826+0000 7f2ba88bf700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba413ff00 con 0x7f2ba40cdc90 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.826+0000 7f2ba88bf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba41404c0 con 0x7f2ba40b2e70 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.826+0000 7f2ba88bf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415aa80 con 0x7f2ba40cd8c0 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ba40cdc90 0x7f2ba40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.829 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ba40cdc90 0x7f2ba40b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40114/0 (socket says 172.21.15.153:40114) 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 -- 172.21.15.153:0/3130201645 learned_addr learned my addr 172.21.15.153:0/3130201645 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba2d9d700 1 --2- 172.21.15.153:0/3130201645 >> v2:172.21.15.177:3300/0 conn(0x7f2ba40b2e70 0x7f2ba406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba259c700 1 --2- 172.21.15.153:0/3130201645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 -- 172.21.15.153:0/3130201645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 msgr2=0x7f2ba4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 --2- 172.21.15.153:0/3130201645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 -- 172.21.15.153:0/3130201645 >> v2:172.21.15.177:3300/0 conn(0x7f2ba40b2e70 msgr2=0x7f2ba406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 --2- 172.21.15.153:0/3130201645 >> v2:172.21.15.177:3300/0 conn(0x7f2ba40b2e70 0x7f2ba406f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 -- 172.21.15.153:0/3130201645 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40b3310 con 0x7f2ba40cdc90 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba259c700 1 --2- 172.21.15.153:0/3130201645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba1d9b700 1 --2- 172.21.15.153:0/3130201645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40cdc90 0x7f2ba40b2930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4067c30 tx=0x7f2b94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5bfba3825dc65b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba159a700 1 -- 172.21.15.153:0/3130201645 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94011830 con 0x7f2ba40cdc90 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba159a700 1 -- 172.21.15.153:0/3130201645 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b94011990 con 0x7f2ba40cdc90 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.827+0000 7f2ba159a700 1 -- 172.21.15.153:0/3130201645 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94011d20 con 0x7f2ba40cdc90 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3130201645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40cdc90 msgr2=0x7f2ba40b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3130201645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40cdc90 0x7f2ba40b2930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4067c30 tx=0x7f2b94009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3130201645 shutdown_connections 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3130201645 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3130201645 >> v2:172.21.15.177:3300/0 conn(0x7f2ba40b2e70 0x7f2ba406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3130201645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40cdc90 0x7f2ba40b2930 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3130201645 >> 172.21.15.153:0/3130201645 conn(0x7f2ba406aed0 msgr2=0x7f2ba406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3130201645 shutdown_connections 2023-11-28T13:01:11.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.828+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3130201645 wait complete. 2023-11-28T13:01:11.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.829+0000 7f2ba88bf700 1 Processor -- start 2023-11-28T13:01:11.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.829+0000 7f2ba88bf700 1 -- start start 2023-11-28T13:01:11.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba88bf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40b2e70 0x7f2ba40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba88bf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba41f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba88bf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba40cdc90 0x7f2ba413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba88bf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba406b6d0 con 0x7f2ba40b2e70 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba88bf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba406b380 con 0x7f2ba40cdc90 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba88bf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba4142010 con 0x7f2ba40cd8c0 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40b2e70 0x7f2ba40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40b2e70 0x7f2ba40b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40120/0 (socket says 172.21.15.153:40120) 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba259c700 1 -- 172.21.15.153:0/3040757106 learned_addr learned my addr 172.21.15.153:0/3040757106 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba1d9b700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba41f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba2d9d700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba40cdc90 0x7f2ba413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba259c700 1 -- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 msgr2=0x7f2ba41f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba259c700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba41f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba259c700 1 -- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba40cdc90 msgr2=0x7f2ba413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.830+0000 7f2ba259c700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba40cdc90 0x7f2ba413ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2ba259c700 1 -- 172.21.15.153:0/3040757106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009550 con 0x7f2ba40b2e70 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2ba2d9d700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba40cdc90 0x7f2ba413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2ba259c700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40b2e70 0x7f2ba40b27b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00e810 tx=0x7f2b8c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2b9affd700 1 -- 172.21.15.153:0/3040757106 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c005c20 con 0x7f2ba40b2e70 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2b9affd700 1 -- 172.21.15.153:0/3040757106 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c005d80 con 0x7f2ba40b2e70 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4140150 con 0x7f2ba40b2e70 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2b9affd700 1 -- 172.21.15.153:0/3040757106 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c006270 con 0x7f2ba40b2e70 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.831+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba40b35f0 con 0x7f2ba40b2e70 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.832+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba40ce100 con 0x7f2ba40b2e70 2023-11-28T13:01:11.836 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.832+0000 7f2b9affd700 1 -- 172.21.15.153:0/3040757106 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b8c022070 con 0x7f2ba40b2e70 2023-11-28T13:01:11.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.833+0000 7f2b9affd700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b840682f0 0x7f2b8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.833+0000 7f2b9affd700 1 -- 172.21.15.153:0/3040757106 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f2b8c0854d0 con 0x7f2ba40b2e70 2023-11-28T13:01:11.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.833+0000 7f2ba1d9b700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b840682f0 0x7f2b8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.833+0000 7f2ba1d9b700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b840682f0 0x7f2b8406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4067c30 tx=0x7f2b94008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:11.837+0000 7f2b9affd700 1 -- 172.21.15.153:0/3040757106 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2b8c051f10 con 0x7f2ba40b2e70 2023-11-28T13:01:12.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.018+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2ba406b9b0 con 0x7f2ba40b2e70 2023-11-28T13:01:12.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.686+0000 7f2b9affd700 1 -- 172.21.15.153:0/3040757106 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b8c0094d0 con 0x7f2ba40b2e70 2023-11-28T13:01:12.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b840682f0 msgr2=0x7f2b8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b840682f0 0x7f2b8406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4067c30 tx=0x7f2b94008030 comp rx=0 tx=0).stop 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40b2e70 msgr2=0x7f2ba40b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40b2e70 0x7f2ba40b27b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00e810 tx=0x7f2b8c00ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 shutdown_connections 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2b840682f0 0x7f2b8406a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ba40b2e70 0x7f2ba40b27b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba40cdc90 0x7f2ba413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 --2- 172.21.15.153:0/3040757106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ba40cd8c0 0x7f2ba41f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 >> 172.21.15.153:0/3040757106 conn(0x7f2ba406aed0 msgr2=0x7f2ba40cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 shutdown_connections 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.690+0000 7f2ba88bf700 1 -- 172.21.15.153:0/3040757106 wait complete. 2023-11-28T13:01:12.704 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' created 2023-11-28T13:01:12.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:01:12.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.840+0000 7f73fc5ba700 1 Processor -- start 2023-11-28T13:01:12.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.840+0000 7f73fc5ba700 1 -- start start 2023-11-28T13:01:12.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fc5ba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f406f850 0x7f73f406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fc5ba700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f413ff00 con 0x7f73f40cd8c0 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fc5ba700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f41404c0 con 0x7f73f4140950 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fc5ba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f4140dc0 con 0x7f73f406f850 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fa356700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f73f40cd8c0 0x7f73f40cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58294/0 (socket says 172.21.15.153:58294) 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fa356700 1 -- 172.21.15.153:0/2946019978 learned_addr learned my addr 172.21.15.153:0/2946019978 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fab57700 1 --2- 172.21.15.153:0/2946019978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f406f850 0x7f73f406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73f9354700 1 -- 172.21.15.153:0/2946019978 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764713650 0 0) 0x7f73f413ff00 con 0x7f73f40cd8c0 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73f9354700 1 -- 172.21.15.153:0/2946019978 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e00035b0 con 0x7f73f40cd8c0 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fab57700 1 -- 172.21.15.153:0/2946019978 >> v1:172.21.15.177:6789/0 conn(0x7f73f4140950 legacy=0x7f73f40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fab57700 1 -- 172.21.15.153:0/2946019978 >> v1:172.21.15.153:6789/0 conn(0x7f73f40cd8c0 legacy=0x7f73f40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73f9354700 1 -- 172.21.15.153:0/2946019978 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1489956478 0 0) 0x7f73e00035b0 con 0x7f73f40cd8c0 2023-11-28T13:01:12.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.841+0000 7f73fab57700 1 -- 172.21.15.153:0/2946019978 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f415aa80 con 0x7f73f406f850 2023-11-28T13:01:12.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fab57700 1 --2- 172.21.15.153:0/2946019978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f406f850 0x7f73f406fcc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009dc0 tx=0x7f73ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf19de66fa4bcffc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73f9354700 1 -- 172.21.15.153:0/2946019978 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00e070 con 0x7f73f406f850 2023-11-28T13:01:12.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73f9354700 1 -- 172.21.15.153:0/2946019978 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec003a20 con 0x7f73f406f850 2023-11-28T13:01:12.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73f9354700 1 -- 172.21.15.153:0/2946019978 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40017f0 con 0x7f73f406f850 2023-11-28T13:01:12.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fc5ba700 1 -- 172.21.15.153:0/2946019978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f406f850 msgr2=0x7f73f406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/2946019978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f406f850 0x7f73f406fcc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009dc0 tx=0x7f73ec00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fc5ba700 1 -- 172.21.15.153:0/2946019978 shutdown_connections 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/2946019978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f406f850 0x7f73f406fcc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fc5ba700 1 -- 172.21.15.153:0/2946019978 >> 172.21.15.153:0/2946019978 conn(0x7f73f406aed0 msgr2=0x7f73f406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fc5ba700 1 -- 172.21.15.153:0/2946019978 shutdown_connections 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.842+0000 7f73fc5ba700 1 -- 172.21.15.153:0/2946019978 wait complete. 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 Processor -- start 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 -- start start 2023-11-28T13:01:12.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73f406f850 0x7f73f41540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f40cd8c0 0x7f73f40b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f4140950 0x7f73f40b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f4141b70 con 0x7f73f406f850 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f4154630 con 0x7f73f4140950 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fc5ba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40cb200 con 0x7f73f40cd8c0 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fa356700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73f406f850 0x7f73f41540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fa356700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73f406f850 0x7f73f41540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40134/0 (socket says 172.21.15.153:40134) 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.844+0000 7f73fa356700 1 -- 172.21.15.153:0/340341931 learned_addr learned my addr 172.21.15.153:0/340341931 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73f9b55700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f40cd8c0 0x7f73f40b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fab57700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f4140950 0x7f73f40b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fa356700 1 -- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f40cd8c0 msgr2=0x7f73f40b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fa356700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f40cd8c0 0x7f73f40b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fa356700 1 -- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f4140950 msgr2=0x7f73f40b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fa356700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f4140950 0x7f73f40b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fa356700 1 -- 172.21.15.153:0/340341931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec009a70 con 0x7f73f406f850 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fa356700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73f406f850 0x7f73f41540f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73e400c750 tx=0x7f73e400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73eaffd700 1 -- 172.21.15.153:0/340341931 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40074e0 con 0x7f73f406f850 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73eaffd700 1 -- 172.21.15.153:0/340341931 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e4007640 con 0x7f73f406f850 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f40b3310 con 0x7f73f406f850 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73eaffd700 1 -- 172.21.15.153:0/340341931 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4005a70 con 0x7f73f406f850 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.845+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f40b3040 con 0x7f73f406f850 2023-11-28T13:01:12.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.846+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f40b35f0 con 0x7f73f406f850 2023-11-28T13:01:12.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.846+0000 7f73eaffd700 1 -- 172.21.15.153:0/340341931 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f73e4019030 con 0x7f73f406f850 2023-11-28T13:01:12.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.847+0000 7f73eaffd700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73d80682f0 0x7f73d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.847+0000 7f73eaffd700 1 -- 172.21.15.153:0/340341931 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f73e4085140 con 0x7f73f406f850 2023-11-28T13:01:12.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.847+0000 7f73f9b55700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73d80682f0 0x7f73d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.847+0000 7f73f9b55700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73d80682f0 0x7f73d806a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73f40c7a70 tx=0x7f73f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:12.851+0000 7f73eaffd700 1 -- 172.21.15.153:0/340341931 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73e4051890 con 0x7f73f406f850 2023-11-28T13:01:13.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.075+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f73f4068690 con 0x7f73f406f850 2023-11-28T13:01:13.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.083+0000 7f73eaffd700 1 -- 172.21.15.153:0/340341931 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f73e4054eb0 con 0x7f73f406f850 2023-11-28T13:01:13.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73d80682f0 msgr2=0x7f73d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73d80682f0 0x7f73d806a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73f40c7a70 tx=0x7f73f0009040 comp rx=0 tx=0).stop 2023-11-28T13:01:13.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73f406f850 msgr2=0x7f73f41540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73f406f850 0x7f73f41540f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73e400c750 tx=0x7f73e400ca60 comp rx=0 tx=0).stop 2023-11-28T13:01:13.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 shutdown_connections 2023-11-28T13:01:13.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73d80682f0 0x7f73d806a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73f406f850 0x7f73f41540f0 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73e400c750 tx=0x7f73e400ca60 comp rx=0 tx=0).stop 2023-11-28T13:01:13.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f4140950 0x7f73f40b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 --2- 172.21.15.153:0/340341931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73f40cd8c0 0x7f73f40b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.090+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 >> 172.21.15.153:0/340341931 conn(0x7f73f406aed0 msgr2=0x7f73f413e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.096+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 shutdown_connections 2023-11-28T13:01:13.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.097+0000 7f73fc5ba700 1 -- 172.21.15.153:0/340341931 wait complete. 2023-11-28T13:01:13.099 INFO:teuthology.orchestra.run.smithi153.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T13:01:13.115 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:13.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.234+0000 7fe709392700 1 Processor -- start 2023-11-28T13:01:13.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe709392700 1 -- start start 2023-11-28T13:01:13.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe709392700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe704158200 0x7fe704070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe709392700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe7040b0d80 0x7fe70415add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe709392700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70415e1a0 con 0x7fe7040caa70 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe709392700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70415f070 con 0x7fe7040b0d80 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe709392700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe704071190 con 0x7fe704158200 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe7037fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe7040b0d80 0x7fe70415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe7037fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe7040b0d80 0x7fe70415add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40162/0 (socket says 172.21.15.153:40162) 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.235+0000 7fe7037fe700 1 -- 172.21.15.153:0/987508644 learned_addr learned my addr 172.21.15.153:0/987508644 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe702ffd700 1 --2- 172.21.15.153:0/987508644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe704158200 0x7fe704070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe7037fe700 1 -- 172.21.15.153:0/987508644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe704158200 msgr2=0x7fe704070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe7037fe700 1 --2- 172.21.15.153:0/987508644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe704158200 0x7fe704070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe7037fe700 1 -- 172.21.15.153:0/987508644 >> v1:172.21.15.177:6789/0 conn(0x7fe7040caa70 legacy=0x7fe7040b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe7037fe700 1 -- 172.21.15.153:0/987508644 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7041585d0 con 0x7fe7040b0d80 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe702ffd700 1 --2- 172.21.15.153:0/987508644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe704158200 0x7fe704070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe7037fe700 1 --2- 172.21.15.153:0/987508644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040b0d80 0x7fe70415add0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009e80 tx=0x7fe6fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6e4ac23224eca1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe701ffb700 1 -- 172.21.15.153:0/987508644 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00e070 con 0x7fe7040b0d80 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe701ffb700 1 -- 172.21.15.153:0/987508644 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc00b980 con 0x7fe7040b0d80 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.236+0000 7fe701ffb700 1 -- 172.21.15.153:0/987508644 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00bd30 con 0x7fe7040b0d80 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 -- 172.21.15.153:0/987508644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040b0d80 msgr2=0x7fe70415add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 --2- 172.21.15.153:0/987508644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040b0d80 0x7fe70415add0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009e80 tx=0x7fe6fc00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 -- 172.21.15.153:0/987508644 shutdown_connections 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 --2- 172.21.15.153:0/987508644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040b0d80 0x7fe70415add0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 --2- 172.21.15.153:0/987508644 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe704158200 0x7fe704070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 -- 172.21.15.153:0/987508644 >> 172.21.15.153:0/987508644 conn(0x7fe7040cb670 msgr2=0x7fe7040cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 -- 172.21.15.153:0/987508644 shutdown_connections 2023-11-28T13:01:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.237+0000 7fe709392700 1 -- 172.21.15.153:0/987508644 wait complete. 2023-11-28T13:01:13.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.238+0000 7fe709392700 1 Processor -- start 2023-11-28T13:01:13.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.238+0000 7fe709392700 1 -- start start 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe709392700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7040b0d80 0x7fe7041f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe709392700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040caa70 0x7fe7040af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe709392700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704158200 0x7fe7041e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe709392700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7041f5d40 con 0x7fe7040caa70 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe709392700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7041e4aa0 con 0x7fe704158200 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe709392700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7041f56f0 con 0x7fe7040b0d80 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040caa70 0x7fe7040af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040caa70 0x7fe7040af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40166/0 (socket says 172.21.15.153:40166) 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 -- 172.21.15.153:0/782532407 learned_addr learned my addr 172.21.15.153:0/782532407 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7037fe700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704158200 0x7fe7041e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe702ffd700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7040b0d80 0x7fe7041f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 -- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7040b0d80 msgr2=0x7fe7041f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7040b0d80 0x7fe7041f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 -- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704158200 msgr2=0x7fe7041e3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704158200 0x7fe7041e3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 -- 172.21.15.153:0/782532407 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc009b30 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe7027fc700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040caa70 0x7fe7040af860 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c4e0 tx=0x7fe6f400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.239+0000 7fe6eb7fe700 1 -- 172.21.15.153:0/782532407 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40052d0 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.240+0000 7fe6eb7fe700 1 -- 172.21.15.153:0/782532407 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4005430 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.240+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70415e6d0 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.240+0000 7fe6eb7fe700 1 -- 172.21.15.153:0/782532407 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40038c0 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.240+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70415e3f0 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.240+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70415e9b0 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.241+0000 7fe6eb7fe700 1 -- 172.21.15.153:0/782532407 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe6f4019030 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.241+0000 7fe6eb7fe700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6f00682f0 0x7fe6f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.241+0000 7fe6eb7fe700 1 -- 172.21.15.153:0/782532407 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6f4084ee0 con 0x7fe7040caa70 2023-11-28T13:01:13.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.241+0000 7fe702ffd700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6f00682f0 0x7fe6f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.242+0000 7fe702ffd700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6f00682f0 0x7fe6f006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8006f10 tx=0x7fe6f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.246+0000 7fe6eb7fe700 1 -- 172.21.15.153:0/782532407 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe6f4051740 con 0x7fe7040caa70 2023-11-28T13:01:13.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.429+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe70406fd60 con 0x7fe7040caa70 2023-11-28T13:01:13.431 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.430+0000 7fe6eb7fe700 1 -- 172.21.15.153:0/782532407 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18040 (secure 0 0 0) 0x7fe6f4017090 con 0x7fe7040caa70 2023-11-28T13:01:13.431 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:13.431 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":38,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:13.076193+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:11.364224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:12.036978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:13.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6f00682f0 msgr2=0x7fe6f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6f00682f0 0x7fe6f006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8006f10 tx=0x7fe6f8008040 comp rx=0 tx=0).stop 2023-11-28T13:01:13.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040caa70 msgr2=0x7fe7040af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040caa70 0x7fe7040af860 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c4e0 tx=0x7fe6f400c7f0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 shutdown_connections 2023-11-28T13:01:13.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe704158200 0x7fe7041e3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe7040b0d80 0x7fe7041f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe6f00682f0 0x7fe6f006a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 --2- 172.21.15.153:0/782532407 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7040caa70 0x7fe7040af860 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 >> 172.21.15.153:0/782532407 conn(0x7fe7040cb670 msgr2=0x7fe704159410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 shutdown_connections 2023-11-28T13:01:13.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.432+0000 7fe709392700 1 -- 172.21.15.153:0/782532407 wait complete. 2023-11-28T13:01:13.445 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:13.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.564+0000 7f8239537700 1 Processor -- start 2023-11-28T13:01:13.565 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.565+0000 7f8239537700 1 -- start start 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.565+0000 7f8239537700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f823406fb30 0x7f8234158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.565+0000 7f8239537700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f823415d1f0 0x7f82340b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.565+0000 7f8239537700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823406bf10 con 0x7f82340cdda0 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.565+0000 7f8239537700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823406ff00 con 0x7f823415d1f0 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.565+0000 7f8239537700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82340cb0d0 con 0x7f823406fb30 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f823415d1f0 0x7f82340b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f823415d1f0 0x7f82340b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40172/0 (socket says 172.21.15.153:40172) 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 -- 172.21.15.153:0/123036025 learned_addr learned my addr 172.21.15.153:0/123036025 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f8232ffd700 1 --2- 172.21.15.153:0/123036025 >> v2:172.21.15.177:3300/0 conn(0x7f823406fb30 0x7f8234158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 -- 172.21.15.153:0/123036025 >> v2:172.21.15.177:3300/0 conn(0x7f823406fb30 msgr2=0x7f8234158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:13.568 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 --2- 172.21.15.153:0/123036025 >> v2:172.21.15.177:3300/0 conn(0x7f823406fb30 0x7f8234158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 -- 172.21.15.153:0/123036025 >> v1:172.21.15.177:6789/0 conn(0x7f82340cdda0 legacy=0x7f82340ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 -- 172.21.15.153:0/123036025 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82340cb3b0 con 0x7f823415d1f0 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f82327fc700 1 --2- 172.21.15.153:0/123036025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340b06e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f823406fa30 tx=0x7f822400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a52d5e9bf6a595c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f8231ffb700 1 -- 172.21.15.153:0/123036025 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822400dd30 con 0x7f823415d1f0 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f8231ffb700 1 -- 172.21.15.153:0/123036025 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822400de90 con 0x7f823415d1f0 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.566+0000 7f8231ffb700 1 -- 172.21.15.153:0/123036025 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224010690 con 0x7f823415d1f0 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 -- 172.21.15.153:0/123036025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 msgr2=0x7f82340b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 --2- 172.21.15.153:0/123036025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340b06e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f823406fa30 tx=0x7f822400be70 comp rx=0 tx=0).stop 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 -- 172.21.15.153:0/123036025 shutdown_connections 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 --2- 172.21.15.153:0/123036025 >> v2:172.21.15.177:3300/0 conn(0x7f823406fb30 0x7f8234158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 --2- 172.21.15.153:0/123036025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340b06e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.569 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 -- 172.21.15.153:0/123036025 >> 172.21.15.153:0/123036025 conn(0x7f82340cb670 msgr2=0x7f82340cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.570 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 -- 172.21.15.153:0/123036025 shutdown_connections 2023-11-28T13:01:13.570 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.567+0000 7f8239537700 1 -- 172.21.15.153:0/123036025 wait complete. 2023-11-28T13:01:13.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.568+0000 7f8239537700 1 Processor -- start 2023-11-28T13:01:13.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.568+0000 7f8239537700 1 -- start start 2023-11-28T13:01:13.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f8239537700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823406fb30 0x7f82340c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f8239537700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82340cdda0 0x7f82341f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f8239537700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f8239537700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82340c3ac0 con 0x7f823415d1f0 2023-11-28T13:01:13.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f8239537700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82340c7670 con 0x7f823406fb30 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f8239537700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82341f2280 con 0x7f82340cdda0 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40180/0 (socket says 172.21.15.153:40180) 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 -- 172.21.15.153:0/3613534796 learned_addr learned my addr 172.21.15.153:0/3613534796 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f8232ffd700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823406fb30 0x7f82340c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82327fc700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82340cdda0 0x7f82341f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 -- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82340cdda0 msgr2=0x7f82341f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82340cdda0 0x7f82341f1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 -- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823406fb30 msgr2=0x7f82340c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823406fb30 0x7f82340c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.569+0000 7f82337fe700 1 -- 172.21.15.153:0/3613534796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822400b7f0 con 0x7f823415d1f0 2023-11-28T13:01:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.570+0000 7f82337fe700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340c3f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f822c00cc80 tx=0x7f822c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.570+0000 7f821b7fe700 1 -- 172.21.15.153:0/3613534796 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00ad40 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.570+0000 7f821b7fe700 1 -- 172.21.15.153:0/3613534796 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00ce80 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.570+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823406bba0 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.570+0000 7f821b7fe700 1 -- 172.21.15.153:0/3613534796 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00aea0 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.570+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823406b8c0 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.571+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823415d690 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.571+0000 7f821b7fe700 1 -- 172.21.15.153:0/3613534796 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f822c010050 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.571+0000 7f821b7fe700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82200682f0 0x7f822006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.572+0000 7f821b7fe700 1 -- 172.21.15.153:0/3613534796 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f822c085c80 con 0x7f823415d1f0 2023-11-28T13:01:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.572+0000 7f8232ffd700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82200682f0 0x7f822006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.572+0000 7f8232ffd700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82200682f0 0x7f822006a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8228009930 tx=0x7f8228008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.576+0000 7f821b7fe700 1 -- 172.21.15.153:0/3613534796 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f822c0524e0 con 0x7f823415d1f0 2023-11-28T13:01:13.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.764+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82340c3c70 con 0x7f823415d1f0 2023-11-28T13:01:13.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.765+0000 7f821b7fe700 1 -- 172.21.15.153:0/3613534796 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18040 (secure 0 0 0) 0x7f82340c3c70 con 0x7f823415d1f0 2023-11-28T13:01:13.766 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:13.766 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":38,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:13.076193+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:11.364224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:12.036978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82200682f0 msgr2=0x7f822006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82200682f0 0x7f822006a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8228009930 tx=0x7f8228008040 comp rx=0 tx=0).stop 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 msgr2=0x7f82340c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340c3f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f822c00cc80 tx=0x7f822c00a340 comp rx=0 tx=0).stop 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 shutdown_connections 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82200682f0 0x7f822006a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823415d1f0 0x7f82340c3f50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823406fb30 0x7f82340c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 --2- 172.21.15.153:0/3613534796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82340cdda0 0x7f82341f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.767+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 >> 172.21.15.153:0/3613534796 conn(0x7f82340cb670 msgr2=0x7f8234157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.768+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 shutdown_connections 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.768+0000 7f8239537700 1 -- 172.21.15.153:0/3613534796 wait complete. 2023-11-28T13:01:13.781 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:01:13.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.903+0000 7fdcdcedb700 1 Processor -- start 2023-11-28T13:01:13.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.903+0000 7fdcdcedb700 1 -- start start 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcdcedb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd81658c0 0x7fdcd8167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcdcedb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd815d060 con 0x7fdcd806dee0 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcdcedb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd815d340 con 0x7fdcd8071040 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcdcedb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8161550 con 0x7fdcd81658c0 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcd5d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fdcd806dee0 0x7fdcd806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58350/0 (socket says 172.21.15.153:58350) 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcd5d9b700 1 -- 172.21.15.153:0/2573077772 learned_addr learned my addr 172.21.15.153:0/2573077772 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcd6d9d700 1 --2- 172.21.15.153:0/2573077772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd81658c0 0x7fdcd8167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241262317 0 0) 0x7fdcd815d060 con 0x7fdcd806dee0 2023-11-28T13:01:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.904+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc00035b0 con 0x7fdcd806dee0 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170371759 0 0) 0x7fdcd815d340 con 0x7fdcd8071040 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd815d060 con 0x7fdcd8071040 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd6d9d700 1 -- 172.21.15.153:0/2573077772 >> v1:172.21.15.177:6789/0 conn(0x7fdcd8071040 legacy=0x7fdcd8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd6d9d700 1 -- 172.21.15.153:0/2573077772 >> v1:172.21.15.153:6789/0 conn(0x7fdcd806dee0 legacy=0x7fdcd806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446114413 0 0) 0x7fdcc00035b0 con 0x7fdcd806dee0 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd6d9d700 1 -- 172.21.15.153:0/2573077772 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd815d620 con 0x7fdcd81658c0 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd6d9d700 1 --2- 172.21.15.153:0/2573077772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd81658c0 0x7fdcd8167ca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009dc0 tx=0x7fdcd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e19c52a7357f12d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd000e070 con 0x7fdcd81658c0 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0003a20 con 0x7fdcd81658c0 2023-11-28T13:01:13.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.905+0000 7fdcd559a700 1 -- 172.21.15.153:0/2573077772 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8001670 con 0x7fdcd81658c0 2023-11-28T13:01:13.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.906+0000 7fdcdcedb700 1 -- 172.21.15.153:0/2573077772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd81658c0 msgr2=0x7fdcd8167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.906+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/2573077772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd81658c0 0x7fdcd8167ca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009dc0 tx=0x7fdcd000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:13.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.906+0000 7fdcdcedb700 1 -- 172.21.15.153:0/2573077772 shutdown_connections 2023-11-28T13:01:13.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.906+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/2573077772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd81658c0 0x7fdcd8167ca0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.906+0000 7fdcdcedb700 1 -- 172.21.15.153:0/2573077772 >> 172.21.15.153:0/2573077772 conn(0x7fdcd80d0000 msgr2=0x7fdcd80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.906+0000 7fdcdcedb700 1 -- 172.21.15.153:0/2573077772 shutdown_connections 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.906+0000 7fdcdcedb700 1 -- 172.21.15.153:0/2573077772 wait complete. 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.907+0000 7fdcdcedb700 1 Processor -- start 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.907+0000 7fdcdcedb700 1 -- start start 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcdcedb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd806dee0 0x7fdcd815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcdcedb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd8071040 0x7fdcd80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcdcedb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd81658c0 0x7fdcd80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcdcedb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80afb30 con 0x7fdcd8071040 2023-11-28T13:01:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcdcedb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80afdd0 con 0x7fdcd81658c0 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcdcedb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80b00b0 con 0x7fdcd806dee0 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd8071040 0x7fdcd80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd8071040 0x7fdcd80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40192/0 (socket says 172.21.15.153:40192) 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 -- 172.21.15.153:0/786974892 learned_addr learned my addr 172.21.15.153:0/786974892 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd659c700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd806dee0 0x7fdcd815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd6d9d700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd81658c0 0x7fdcd80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 -- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd806dee0 msgr2=0x7fdcd815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd806dee0 0x7fdcd815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 -- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd81658c0 msgr2=0x7fdcd80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd81658c0 0x7fdcd80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 -- 172.21.15.153:0/786974892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0009a70 con 0x7fdcd8071040 2023-11-28T13:01:13.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd6d9d700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd81658c0 0x7fdcd80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:13.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcd5d9b700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd8071040 0x7fdcd80b0b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c4d0 tx=0x7fdcc800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcc6ffd700 1 -- 172.21.15.153:0/786974892 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8004c80 con 0x7fdcd8071040 2023-11-28T13:01:13.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcc6ffd700 1 -- 172.21.15.153:0/786974892 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc8004e00 con 0x7fdcd8071040 2023-11-28T13:01:13.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.908+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd815ccd0 con 0x7fdcd8071040 2023-11-28T13:01:13.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.909+0000 7fdcc6ffd700 1 -- 172.21.15.153:0/786974892 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8007310 con 0x7fdcd8071040 2023-11-28T13:01:13.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.909+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd815d900 con 0x7fdcd8071040 2023-11-28T13:01:13.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.909+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd806a270 con 0x7fdcd8071040 2023-11-28T13:01:13.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.911+0000 7fdcc6ffd700 1 -- 172.21.15.153:0/786974892 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdcc801a020 con 0x7fdcd8071040 2023-11-28T13:01:13.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.911+0000 7fdcc6ffd700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdcb80682f0 0x7fdcb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.911+0000 7fdcc6ffd700 1 -- 172.21.15.153:0/786974892 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcc8085e50 con 0x7fdcd8071040 2023-11-28T13:01:13.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.911+0000 7fdcd659c700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdcb80682f0 0x7fdcb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.911+0000 7fdcd659c700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdcb80682f0 0x7fdcb806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdccc001220 tx=0x7fdccc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:13.915+0000 7fdcc6ffd700 1 -- 172.21.15.153:0/786974892 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdcc8052800 con 0x7fdcd8071040 2023-11-28T13:01:14.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:14.146+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdcd815f8a0 con 0x7fdcd8071040 2023-11-28T13:01:15.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.090+0000 7fdcc6ffd700 1 -- 172.21.15.153:0/786974892 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdcc8055e20 con 0x7fdcd8071040 2023-11-28T13:01:15.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdcb80682f0 msgr2=0x7fdcb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdcb80682f0 0x7fdcb806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdccc001220 tx=0x7fdccc009040 comp rx=0 tx=0).stop 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd8071040 msgr2=0x7fdcd80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd8071040 0x7fdcd80b0b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c4d0 tx=0x7fdcc800c7e0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 shutdown_connections 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcd81658c0 0x7fdcd80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcd806dee0 0x7fdcd815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdcb80682f0 0x7fdcb806a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 --2- 172.21.15.153:0/786974892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdcd8071040 0x7fdcd80b0b90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 >> 172.21.15.153:0/786974892 conn(0x7fdcd80d0000 msgr2=0x7fdcd80cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 shutdown_connections 2023-11-28T13:01:15.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.093+0000 7fdcdcedb700 1 -- 172.21.15.153:0/786974892 wait complete. 2023-11-28T13:01:15.110 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:01:15.110 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:01:15.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.237+0000 7f85ff576700 1 Processor -- start 2023-11-28T13:01:15.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.237+0000 7f85ff576700 1 -- start start 2023-11-28T13:01:15.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85ff576700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f85f806dee0 0x7f85f806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85ff576700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f815d060 con 0x7f85f8160540 2023-11-28T13:01:15.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85ff576700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f815d340 con 0x7f85f8071040 2023-11-28T13:01:15.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85ff576700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f815d620 con 0x7f85f806dee0 2023-11-28T13:01:15.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85fcb11700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f85f806dee0 0x7f85f806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85fdb13700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f85f8160540 0x7f85f8160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58366/0 (socket says 172.21.15.153:58366) 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85fdb13700 1 -- 172.21.15.153:0/661736570 learned_addr learned my addr 172.21.15.153:0/661736570 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85fcb11700 1 -- 172.21.15.153:0/661736570 >> v1:172.21.15.177:6789/0 conn(0x7f85f8071040 legacy=0x7f85f8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85fcb11700 1 -- 172.21.15.153:0/661736570 >> v1:172.21.15.153:6789/0 conn(0x7f85f8160540 legacy=0x7f85f8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85fcb11700 1 -- 172.21.15.153:0/661736570 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f815d900 con 0x7f85f806dee0 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85effff700 1 -- 172.21.15.153:0/661736570 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519430544 0 0) 0x7f85f815d060 con 0x7f85f8160540 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.238+0000 7f85fcb11700 1 --2- 172.21.15.153:0/661736570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f806e330 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009840 tx=0x7f85e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3712d5168eae560 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85effff700 1 -- 172.21.15.153:0/661736570 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8011830 con 0x7f85f806dee0 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85effff700 1 -- 172.21.15.153:0/661736570 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8011990 con 0x7f85f806dee0 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85effff700 1 -- 172.21.15.153:0/661736570 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8011d20 con 0x7f85f806dee0 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85ff576700 1 -- 172.21.15.153:0/661736570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 msgr2=0x7f85f806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85ff576700 1 --2- 172.21.15.153:0/661736570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f806e330 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009840 tx=0x7f85e8009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:15.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85ff576700 1 -- 172.21.15.153:0/661736570 shutdown_connections 2023-11-28T13:01:15.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85ff576700 1 --2- 172.21.15.153:0/661736570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f806e330 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85ff576700 1 -- 172.21.15.153:0/661736570 >> 172.21.15.153:0/661736570 conn(0x7f85f80d0000 msgr2=0x7f85f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:15.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85ff576700 1 -- 172.21.15.153:0/661736570 shutdown_connections 2023-11-28T13:01:15.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.239+0000 7f85ff576700 1 -- 172.21.15.153:0/661736570 wait complete. 2023-11-28T13:01:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.241+0000 7f85ff576700 1 Processor -- start 2023-11-28T13:01:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.241+0000 7f85ff576700 1 -- start start 2023-11-28T13:01:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85ff576700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f80ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85ff576700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85f8071040 0x7f85f80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85ff576700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85f8160540 0x7f85f80b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85ff576700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80ae7c0 con 0x7f85f806dee0 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85ff576700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80aeaa0 con 0x7f85f8071040 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85ff576700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80aed80 con 0x7f85f8160540 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f80ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f80ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40204/0 (socket says 172.21.15.153:40204) 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 -- 172.21.15.153:0/773560059 learned_addr learned my addr 172.21.15.153:0/773560059 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fcb11700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85f8071040 0x7f85f80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fdb13700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85f8160540 0x7f85f80b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 -- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85f8160540 msgr2=0x7f85f80b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85f8160540 0x7f85f80b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 -- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85f8071040 msgr2=0x7f85f80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85f8071040 0x7f85f80b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 -- 172.21.15.153:0/773560059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80094a0 con 0x7f85f806dee0 2023-11-28T13:01:15.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85fd312700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f80ade40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f85f400c4d0 tx=0x7f85f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85edffb700 1 -- 172.21.15.153:0/773560059 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40052d0 con 0x7f85f806dee0 2023-11-28T13:01:15.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85edffb700 1 -- 172.21.15.153:0/773560059 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4005430 con 0x7f85f806dee0 2023-11-28T13:01:15.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f815ccd0 con 0x7f85f806dee0 2023-11-28T13:01:15.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.242+0000 7f85edffb700 1 -- 172.21.15.153:0/773560059 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4003880 con 0x7f85f806dee0 2023-11-28T13:01:15.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.243+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f806e870 con 0x7f85f806dee0 2023-11-28T13:01:15.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.243+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f806a270 con 0x7f85f806dee0 2023-11-28T13:01:15.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.244+0000 7f85edffb700 1 -- 172.21.15.153:0/773560059 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f85f4019030 con 0x7f85f806dee0 2023-11-28T13:01:15.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.244+0000 7f85edffb700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85dc0682f0 0x7f85dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.245+0000 7f85edffb700 1 -- 172.21.15.153:0/773560059 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85f4084ec0 con 0x7f85f806dee0 2023-11-28T13:01:15.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.245+0000 7f85fcb11700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85dc0682f0 0x7f85dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.245+0000 7f85fcb11700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85dc0682f0 0x7f85dc06a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f85f80718e0 tx=0x7f85e801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.249+0000 7f85edffb700 1 -- 172.21.15.153:0/773560059 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85f404b5e0 con 0x7f85f806dee0 2023-11-28T13:01:15.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:15.477+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f85f80b09e0 con 0x7f85f806dee0 2023-11-28T13:01:16.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.093+0000 7f85edffb700 1 -- 172.21.15.153:0/773560059 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f85f80b09e0 con 0x7f85f806dee0 2023-11-28T13:01:16.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.095+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85dc0682f0 msgr2=0x7f85dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.095+0000 7f85ff576700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85dc0682f0 0x7f85dc06a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f85f80718e0 tx=0x7f85e801c040 comp rx=0 tx=0).stop 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.095+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 msgr2=0x7f85f80ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.095+0000 7f85ff576700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f80ade40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f85f400c4d0 tx=0x7f85f400c890 comp rx=0 tx=0).stop 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 shutdown_connections 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f85dc0682f0 0x7f85dc06a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85f806dee0 0x7f85f80ade40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85f8160540 0x7f85f80b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 --2- 172.21.15.153:0/773560059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85f8071040 0x7f85f80b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 >> 172.21.15.153:0/773560059 conn(0x7f85f80d0000 msgr2=0x7f85f806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 shutdown_connections 2023-11-28T13:01:16.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.096+0000 7f85ff576700 1 -- 172.21.15.153:0/773560059 wait complete. 2023-11-28T13:01:16.112 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:01:16.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:16.145 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:16.158 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:16.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.244+0000 7f4c80cb2700 1 Processor -- start 2023-11-28T13:01:16.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.244+0000 7f4c80cb2700 1 -- start start 2023-11-28T13:01:16.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c80cb2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c7c15c9b0 0x7f4c7c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c80cb2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4c7c161340 0x7f4c7c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c80cb2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c0d1950 con 0x7f4c7c0d2320 2023-11-28T13:01:16.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c80cb2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b5800 con 0x7f4c7c15c9b0 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c80cb2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b5520 con 0x7f4c7c161340 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c7a59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c7c15c9b0 0x7f4c7c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c7a59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4c7c15c9b0 0x7f4c7c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40212/0 (socket says 172.21.15.153:40212) 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c7a59c700 1 -- 172.21.15.153:0/143652763 learned_addr learned my addr 172.21.15.153:0/143652763 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c79d9b700 1 --2- 172.21.15.153:0/143652763 >> v2:172.21.15.177:3300/0 conn(0x7f4c7c161340 0x7f4c7c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c7a59c700 1 -- 172.21.15.153:0/143652763 >> v2:172.21.15.177:3300/0 conn(0x7f4c7c161340 msgr2=0x7f4c7c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c7a59c700 1 --2- 172.21.15.153:0/143652763 >> v2:172.21.15.177:3300/0 conn(0x7f4c7c161340 0x7f4c7c0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c7a59c700 1 -- 172.21.15.153:0/143652763 >> v1:172.21.15.177:6789/0 conn(0x7f4c7c0d2320 legacy=0x7f4c7c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.245+0000 7f4c7a59c700 1 -- 172.21.15.153:0/143652763 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c0d1300 con 0x7f4c7c15c9b0 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c7a59c700 1 --2- 172.21.15.153:0/143652763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c15c9b0 0x7f4c7c0b4bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0685c0 tx=0x7f4c70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac9132ac2f10628 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c7959a700 1 -- 172.21.15.153:0/143652763 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70011830 con 0x7f4c7c15c9b0 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c7959a700 1 -- 172.21.15.153:0/143652763 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c70011990 con 0x7f4c7c15c9b0 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c7959a700 1 -- 172.21.15.153:0/143652763 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70011d20 con 0x7f4c7c15c9b0 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 -- 172.21.15.153:0/143652763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c15c9b0 msgr2=0x7f4c7c0b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/143652763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c15c9b0 0x7f4c7c0b4bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0685c0 tx=0x7f4c70009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 -- 172.21.15.153:0/143652763 shutdown_connections 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/143652763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c15c9b0 0x7f4c7c0b4bf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/143652763 >> v2:172.21.15.177:3300/0 conn(0x7f4c7c161340 0x7f4c7c0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 -- 172.21.15.153:0/143652763 >> 172.21.15.153:0/143652763 conn(0x7f4c7c072600 msgr2=0x7f4c7c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 -- 172.21.15.153:0/143652763 shutdown_connections 2023-11-28T13:01:16.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.246+0000 7f4c80cb2700 1 -- 172.21.15.153:0/143652763 wait complete. 2023-11-28T13:01:16.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.248+0000 7f4c80cb2700 1 Processor -- start 2023-11-28T13:01:16.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.248+0000 7f4c80cb2700 1 -- start start 2023-11-28T13:01:16.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.248+0000 7f4c80cb2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c0d2320 0x7f4c7c1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.248+0000 7f4c80cb2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c7c15c9b0 0x7f4c7c14d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.248+0000 7f4c80cb2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c7c161340 0x7f4c7c14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c80cb2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c1e8650 con 0x7f4c7c0d2320 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c80cb2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b7330 con 0x7f4c7c15c9b0 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c80cb2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c1e7d20 con 0x7f4c7c161340 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c0d2320 0x7f4c7c1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c0d2320 0x7f4c7c1505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40218/0 (socket says 172.21.15.153:40218) 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 -- 172.21.15.153:0/3634288029 learned_addr learned my addr 172.21.15.153:0/3634288029 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c79d9b700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c7c15c9b0 0x7f4c7c14d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7ad9d700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c7c161340 0x7f4c7c14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 -- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c7c161340 msgr2=0x7f4c7c14dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c7c161340 0x7f4c7c14dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 -- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c7c15c9b0 msgr2=0x7f4c7c14d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c7c15c9b0 0x7f4c7c14d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 -- 172.21.15.153:0/3634288029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c70009550 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c7a59c700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c0d2320 0x7f4c7c1505d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4c70000c00 tx=0x7f4c70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c6affd700 1 -- 172.21.15.153:0/3634288029 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70004380 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c6affd700 1 -- 172.21.15.153:0/3634288029 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7001f070 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c6affd700 1 -- 172.21.15.153:0/3634288029 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c700036a0 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c0b5130 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.249+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c0d15e0 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.250+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c0b52c0 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.251+0000 7f4c6affd700 1 -- 172.21.15.153:0/3634288029 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c7002a020 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.251+0000 7f4c6affd700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4c5c0682f0 0x7f4c5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.251+0000 7f4c6affd700 1 -- 172.21.15.153:0/3634288029 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c70087120 con 0x7f4c7c0d2320 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.251+0000 7f4c79d9b700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4c5c0682f0 0x7f4c5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.251+0000 7f4c79d9b700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4c5c0682f0 0x7f4c5c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c154b60 tx=0x7f4c6c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.255+0000 7f4c6affd700 1 -- 172.21.15.153:0/3634288029 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4c70053900 con 0x7f4c7c0d2320 2023-11-28T13:01:16.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.481+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c7c0745c0 con 0x7f4c7c0d2320 2023-11-28T13:01:16.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.482+0000 7f4c6affd700 1 -- 172.21.15.153:0/3634288029 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f4c70056f20 con 0x7f4c7c0d2320 2023-11-28T13:01:16.483 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:16.483 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.483+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4c5c0682f0 msgr2=0x7f4c5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4c5c0682f0 0x7f4c5c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c154b60 tx=0x7f4c6c009210 comp rx=0 tx=0).stop 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c0d2320 msgr2=0x7f4c7c1505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c0d2320 0x7f4c7c1505d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4c70000c00 tx=0x7f4c70004750 comp rx=0 tx=0).stop 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 shutdown_connections 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4c5c0682f0 0x7f4c5c06a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c7c0d2320 0x7f4c7c1505d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c7c161340 0x7f4c7c14dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 --2- 172.21.15.153:0/3634288029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c7c15c9b0 0x7f4c7c14d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 >> 172.21.15.153:0/3634288029 conn(0x7f4c7c072600 msgr2=0x7f4c7c15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 shutdown_connections 2023-11-28T13:01:16.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.484+0000 7f4c80cb2700 1 -- 172.21.15.153:0/3634288029 wait complete. 2023-11-28T13:01:16.487 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:16.498 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:16.603 INFO:tasks.ceph.mds.a.smithi153.stderr:2023-11-28T13:01:16.602+0000 7f72e99b8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:01:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.616+0000 7fab2cbfd700 1 Processor -- start 2023-11-28T13:01:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.616+0000 7fab2cbfd700 1 -- start start 2023-11-28T13:01:16.618 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab2cbfd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fab28071040 0x7fab28071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.619 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab2cbfd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2806dee0 0x7fab2806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.619 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab2cbfd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab28071950 con 0x7fab2806cdc0 2023-11-28T13:01:16.619 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab2cbfd700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab28161550 con 0x7fab28071040 2023-11-28T13:01:16.619 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab2cbfd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2815d060 con 0x7fab2806dee0 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab2659c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fab28071040 0x7fab28071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab25d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2806dee0 0x7fab2806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab26d9d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fab2806cdc0 0x7fab2806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:35004/0 (socket says 172.21.15.153:35004) 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.617+0000 7fab26d9d700 1 -- 172.21.15.153:0/3242423328 learned_addr learned my addr 172.21.15.153:0/3242423328 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2559a700 1 -- 172.21.15.153:0/3242423328 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652853213 0 0) 0x7fab28071950 con 0x7fab2806cdc0 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2659c700 1 -- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2806dee0 msgr2=0x7fab2806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2659c700 1 --2- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2806dee0 0x7fab2806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2659c700 1 -- 172.21.15.153:0/3242423328 >> v1:172.21.15.177:6789/0 conn(0x7fab2806cdc0 legacy=0x7fab2806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2659c700 1 -- 172.21.15.153:0/3242423328 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2815d340 con 0x7fab28071040 2023-11-28T13:01:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab25d9b700 1 --2- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2806dee0 0x7fab2806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2659c700 1 --2- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab28071040 0x7fab28071410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009840 tx=0x7fab1c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa45a5b17bef8ca9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2559a700 1 -- 172.21.15.153:0/3242423328 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c011830 con 0x7fab28071040 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2559a700 1 -- 172.21.15.153:0/3242423328 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c011990 con 0x7fab28071040 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.618+0000 7fab2559a700 1 -- 172.21.15.153:0/3242423328 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c011d20 con 0x7fab28071040 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 -- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab28071040 msgr2=0x7fab28071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab28071040 0x7fab28071410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009840 tx=0x7fab1c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 -- 172.21.15.153:0/3242423328 shutdown_connections 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab28071040 0x7fab28071410 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/3242423328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab2806dee0 0x7fab2806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 -- 172.21.15.153:0/3242423328 >> 172.21.15.153:0/3242423328 conn(0x7fab280d0000 msgr2=0x7fab280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 -- 172.21.15.153:0/3242423328 shutdown_connections 2023-11-28T13:01:16.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.619+0000 7fab2cbfd700 1 -- 172.21.15.153:0/3242423328 wait complete. 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.620+0000 7fab2cbfd700 1 Processor -- start 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.620+0000 7fab2cbfd700 1 -- start start 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2cbfd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab2806cdc0 0x7fab28159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2cbfd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2806dee0 0x7fab2815b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2cbfd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab28071040 0x7fab28155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2cbfd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab280b3a70 con 0x7fab2806cdc0 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2cbfd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2815bf20 con 0x7fab2806dee0 2023-11-28T13:01:16.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2cbfd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab28159770 con 0x7fab28071040 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab2806cdc0 0x7fab28159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab2806cdc0 0x7fab28159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40232/0 (socket says 172.21.15.153:40232) 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 -- 172.21.15.153:0/2912947210 learned_addr learned my addr 172.21.15.153:0/2912947210 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab26d9d700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab28071040 0x7fab28155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab25d9b700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2806dee0 0x7fab2815b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 -- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab28071040 msgr2=0x7fab28155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab28071040 0x7fab28155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 -- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2806dee0 msgr2=0x7fab2815b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2806dee0 0x7fab2815b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 -- 172.21.15.153:0/2912947210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180094a0 con 0x7fab2806cdc0 2023-11-28T13:01:16.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.621+0000 7fab2659c700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab2806cdc0 0x7fab28159100 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0090b0 tx=0x7fab1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.622+0000 7fab16ffd700 1 -- 172.21.15.153:0/2912947210 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0045d0 con 0x7fab2806cdc0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.622+0000 7fab16ffd700 1 -- 172.21.15.153:0/2912947210 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c01e070 con 0x7fab2806cdc0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.622+0000 7fab16ffd700 1 -- 172.21.15.153:0/2912947210 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0038c0 con 0x7fab2806cdc0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.622+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c0094a0 con 0x7fab2806cdc0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.622+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2815d900 con 0x7fab2806cdc0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.622+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2815d620 con 0x7fab2806cdc0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.623+0000 7fab16ffd700 1 -- 172.21.15.153:0/2912947210 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab1c00f050 con 0x7fab2806cdc0 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.623+0000 7fab16ffd700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab080682f0 0x7fab0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.623+0000 7fab16ffd700 1 -- 172.21.15.153:0/2912947210 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab1c086ab0 con 0x7fab2806cdc0 2023-11-28T13:01:16.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.623+0000 7fab25d9b700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab080682f0 0x7fab0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.624+0000 7fab25d9b700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab080682f0 0x7fab0806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fab280c6f80 tx=0x7fab18009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.628+0000 7fab16ffd700 1 -- 172.21.15.153:0/2912947210 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab1c053310 con 0x7fab2806cdc0 2023-11-28T13:01:16.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.814+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab2806e870 con 0x7fab2806cdc0 2023-11-28T13:01:16.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.815+0000 7fab16ffd700 1 -- 172.21.15.153:0/2912947210 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18027 (secure 0 0 0) 0x7fab1c015090 con 0x7fab2806cdc0 2023-11-28T13:01:16.817 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:16.817 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":39,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:14.078173+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:11.364224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:12.036978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:16.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab080682f0 msgr2=0x7fab0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab080682f0 0x7fab0806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fab280c6f80 tx=0x7fab18009210 comp rx=0 tx=0).stop 2023-11-28T13:01:16.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab2806cdc0 msgr2=0x7fab28159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab2806cdc0 0x7fab28159100 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0090b0 tx=0x7fab1c004750 comp rx=0 tx=0).stop 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 shutdown_connections 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab080682f0 0x7fab0806a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab2806cdc0 0x7fab28159100 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab28071040 0x7fab28155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 --2- 172.21.15.153:0/2912947210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab2806dee0 0x7fab2815b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 >> 172.21.15.153:0/2912947210 conn(0x7fab280d0000 msgr2=0x7fab2806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 shutdown_connections 2023-11-28T13:01:16.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.817+0000 7fab2cbfd700 1 -- 172.21.15.153:0/2912947210 wait complete. 2023-11-28T13:01:16.831 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:01:16.831 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:01:16.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.948+0000 7f3b183fd700 1 Processor -- start 2023-11-28T13:01:16.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.948+0000 7f3b183fd700 1 -- start start 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.949+0000 7f3b183fd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10071040 0x7f3b10071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.949+0000 7f3b183fd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3b10160540 0x7f3b101609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.949+0000 7f3b183fd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b10071950 con 0x7f3b1006dee0 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.949+0000 7f3b183fd700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1015d060 con 0x7f3b10160540 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.949+0000 7f3b183fd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1015d340 con 0x7f3b10071040 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.949+0000 7f3b1699a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3b10160540 0x7f3b101609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b15998700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3b1006dee0 0x7f3b1006e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58404/0 (socket says 172.21.15.153:58404) 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b15998700 1 -- 172.21.15.153:0/517654916 learned_addr learned my addr 172.21.15.153:0/517654916 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:16.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b16199700 1 --2- 172.21.15.153:0/517654916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10071040 0x7f3b10071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b1699a700 1 -- 172.21.15.153:0/517654916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10071040 msgr2=0x7f3b10071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b1699a700 1 --2- 172.21.15.153:0/517654916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10071040 0x7f3b10071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b1699a700 1 -- 172.21.15.153:0/517654916 >> v1:172.21.15.153:6789/0 conn(0x7f3b1006dee0 legacy=0x7f3b1006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b15197700 1 -- 172.21.15.153:0/517654916 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630016444 0 0) 0x7f3b10071950 con 0x7f3b1006dee0 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b1699a700 1 -- 172.21.15.153:0/517654916 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1015d620 con 0x7f3b10160540 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b1699a700 1 --2- 172.21.15.153:0/517654916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b10160540 0x7f3b101609c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009d90 tx=0x7f3b0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6789b91169d60a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b15197700 1 -- 172.21.15.153:0/517654916 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800e070 con 0x7f3b10160540 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b15197700 1 -- 172.21.15.153:0/517654916 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0800bbd0 con 0x7f3b10160540 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.950+0000 7f3b15197700 1 -- 172.21.15.153:0/517654916 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08012430 con 0x7f3b10160540 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 -- 172.21.15.153:0/517654916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b10160540 msgr2=0x7f3b101609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 --2- 172.21.15.153:0/517654916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b10160540 0x7f3b101609c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009d90 tx=0x7f3b0800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 -- 172.21.15.153:0/517654916 shutdown_connections 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 --2- 172.21.15.153:0/517654916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10071040 0x7f3b10071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 --2- 172.21.15.153:0/517654916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b10160540 0x7f3b101609c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 -- 172.21.15.153:0/517654916 >> 172.21.15.153:0/517654916 conn(0x7f3b100d0000 msgr2=0x7f3b100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 -- 172.21.15.153:0/517654916 shutdown_connections 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.951+0000 7f3b183fd700 1 -- 172.21.15.153:0/517654916 wait complete. 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.952+0000 7f3b183fd700 1 Processor -- start 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.952+0000 7f3b183fd700 1 -- start start 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b183fd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b1006dee0 0x7f3b100b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b183fd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b10071040 0x7f3b100ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b183fd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10160540 0x7f3b100ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b183fd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b101f3570 con 0x7f3b1006dee0 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b183fd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100c69b0 con 0x7f3b10071040 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b183fd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b101f63b0 con 0x7f3b10160540 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b1006dee0 0x7f3b100b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b1006dee0 0x7f3b100b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40246/0 (socket says 172.21.15.153:40246) 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 -- 172.21.15.153:0/1791898025 learned_addr learned my addr 172.21.15.153:0/1791898025 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 -- 172.21.15.153:0/1791898025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10160540 msgr2=0x7f3b100ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10160540 0x7f3b100ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b15998700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b10071040 0x7f3b100ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 -- 172.21.15.153:0/1791898025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b10071040 msgr2=0x7f3b100ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b10071040 0x7f3b100ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 -- 172.21.15.153:0/1791898025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009a70 con 0x7f3b1006dee0 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b16199700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b1006dee0 0x7f3b100b5920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d800 tx=0x7f3b0000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.953+0000 7f3b06ffd700 1 -- 172.21.15.153:0/1791898025 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000125f0 con 0x7f3b1006dee0 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.954+0000 7f3b06ffd700 1 -- 172.21.15.153:0/1791898025 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b00012750 con 0x7f3b1006dee0 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.954+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1015ccd0 con 0x7f3b1006dee0 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.954+0000 7f3b06ffd700 1 -- 172.21.15.153:0/1791898025 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00012b00 con 0x7f3b1006dee0 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.954+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1015d900 con 0x7f3b1006dee0 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.954+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1006a270 con 0x7f3b1006dee0 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.955+0000 7f3b06ffd700 1 -- 172.21.15.153:0/1791898025 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b00012c60 con 0x7f3b1006dee0 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.955+0000 7f3b06ffd700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3af40682f0 0x7f3af406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.955+0000 7f3b15998700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3af40682f0 0x7f3af406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.955+0000 7f3b06ffd700 1 -- 172.21.15.153:0/1791898025 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b00086140 con 0x7f3b1006dee0 2023-11-28T13:01:16.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.956+0000 7f3b15998700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3af40682f0 0x7f3af406a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b10157150 tx=0x7f3b0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:16.960+0000 7f3b06ffd700 1 -- 172.21.15.153:0/1791898025 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3b000529a0 con 0x7f3b1006dee0 2023-11-28T13:01:17.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.138+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b1006c360 con 0x7f3af40682f0 2023-11-28T13:01:17.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.147+0000 7f3b06ffd700 1 -- 172.21.15.153:0/1791898025 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309773 (secure 0 0 0) 0x7f3b1006c360 con 0x7f3af40682f0 2023-11-28T13:01:17.152 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:17.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3af40682f0 msgr2=0x7f3af406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3af40682f0 0x7f3af406a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b10157150 tx=0x7f3b0c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:17.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b1006dee0 msgr2=0x7f3b100b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b1006dee0 0x7f3b100b5920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000d800 tx=0x7f3b0000dbc0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 shutdown_connections 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3af40682f0 0x7f3af406a7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b1006dee0 0x7f3b100b5920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b10160540 0x7f3b100ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 --2- 172.21.15.153:0/1791898025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b10071040 0x7f3b100ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 >> 172.21.15.153:0/1791898025 conn(0x7f3b100d0000 msgr2=0x7f3b100cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 shutdown_connections 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.150+0000 7f3b183fd700 1 -- 172.21.15.153:0/1791898025 wait complete. 2023-11-28T13:01:17.156 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:01:17.164 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-11-28T13:01:16.158364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69688,"kb_used_data":4472,"kb_used_omap":0,"kb_used_meta":65152,"kb_avail":754905032,"statfs":{"total":773094113280,"available":773022752768,"internally_reserved":0,"allocated":4579328,"data_stored":2562344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66715648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.478842"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090821+0000","last_change":"2023-11-28T13:01:13.094545+0000","last_active":"2023-11-28T13:01:14.090821+0000","last_peered":"2023-11-28T13:01:14.090821+0000","last_clean":"2023-11-28T13:01:14.090821+0000","last_became_active":"2023-11-28T13:01:13.094296+0000","last_became_peered":"2023-11-28T13:01:13.094296+0000","last_unstale":"2023-11-28T13:01:14.090821+0000","last_undegraded":"2023-11-28T13:01:14.090821+0000","last_fullsized":"2023-11-28T13:01:14.090821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:18.727519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090776+0000","last_change":"2023-11-28T13:01:13.088855+0000","last_active":"2023-11-28T13:01:14.090776+0000","last_peered":"2023-11-28T13:01:14.090776+0000","last_clean":"2023-11-28T13:01:14.090776+0000","last_became_active":"2023-11-28T13:01:13.088631+0000","last_became_peered":"2023-11-28T13:01:13.088631+0000","last_unstale":"2023-11-28T13:01:14.090776+0000","last_undegraded":"2023-11-28T13:01:14.090776+0000","last_fullsized":"2023-11-28T13:01:14.090776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:42.143519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107263+0000","last_change":"2023-11-28T13:01:13.102065+0000","last_active":"2023-11-28T13:01:13.107263+0000","last_peered":"2023-11-28T13:01:13.107263+0000","last_clean":"2023-11-28T13:01:13.107263+0000","last_became_active":"2023-11-28T13:01:13.101822+0000","last_became_peered":"2023-11-28T13:01:13.101822+0000","last_unstale":"2023-11-28T13:01:13.107263+0000","last_undegraded":"2023-11-28T13:01:13.107263+0000","last_fullsized":"2023-11-28T13:01:13.107263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:51.732790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.099900+0000","last_change":"2023-11-28T13:01:13.095907+0000","last_active":"2023-11-28T13:01:13.099900+0000","last_peered":"2023-11-28T13:01:13.099900+0000","last_clean":"2023-11-28T13:01:13.099900+0000","last_became_active":"2023-11-28T13:01:13.095190+0000","last_became_peered":"2023-11-28T13:01:13.095190+0000","last_unstale":"2023-11-28T13:01:13.099900+0000","last_undegraded":"2023-11-28T13:01:13.099900+0000","last_fullsized":"2023-11-28T13:01:13.099900+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:34.713740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088925+0000","last_change":"2023-11-28T13:01:13.103245+0000","last_active":"2023-11-28T13:01:14.088925+0000","last_peered":"2023-11-28T13:01:14.088925+0000","last_clean":"2023-11-28T13:01:14.088925+0000","last_became_active":"2023-11-28T13:01:13.102920+0000","last_became_peered":"2023-11-28T13:01:13.102920+0000","last_unstale":"2023-11-28T13:01:14.088925+0000","last_undegraded":"2023-11-28T13:01:14.088925+0000","last_fullsized":"2023-11-28T13:01:14.088925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:23.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106870+0000","last_change":"2023-11-28T13:01:13.101383+0000","last_active":"2023-11-28T13:01:13.106870+0000","last_peered":"2023-11-28T13:01:13.106870+0000","last_clean":"2023-11-28T13:01:13.106870+0000","last_became_active":"2023-11-28T13:01:13.101093+0000","last_became_peered":"2023-11-28T13:01:13.101093+0000","last_unstale":"2023-11-28T13:01:13.106870+0000","last_undegraded":"2023-11-28T13:01:13.106870+0000","last_fullsized":"2023-11-28T13:01:13.106870+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090405+0000","last_change":"2023-11-28T13:01:13.093176+0000","last_active":"2023-11-28T13:01:14.090405+0000","last_peered":"2023-11-28T13:01:14.090405+0000","last_clean":"2023-11-28T13:01:14.090405+0000","last_became_active":"2023-11-28T13:01:13.092922+0000","last_became_peered":"2023-11-28T13:01:13.092922+0000","last_unstale":"2023-11-28T13:01:14.090405+0000","last_undegraded":"2023-11-28T13:01:14.090405+0000","last_fullsized":"2023-11-28T13:01:14.090405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:31.712364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102138+0000","last_change":"2023-11-28T13:01:13.090026+0000","last_active":"2023-11-28T13:01:13.102138+0000","last_peered":"2023-11-28T13:01:13.102138+0000","last_clean":"2023-11-28T13:01:13.102138+0000","last_became_active":"2023-11-28T13:01:13.089734+0000","last_became_peered":"2023-11-28T13:01:13.089734+0000","last_unstale":"2023-11-28T13:01:13.102138+0000","last_undegraded":"2023-11-28T13:01:13.102138+0000","last_fullsized":"2023-11-28T13:01:13.102138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110483+0000","last_change":"2023-11-28T13:01:13.106317+0000","last_active":"2023-11-28T13:01:13.110483+0000","last_peered":"2023-11-28T13:01:13.110483+0000","last_clean":"2023-11-28T13:01:13.110483+0000","last_became_active":"2023-11-28T13:01:13.106096+0000","last_became_peered":"2023-11-28T13:01:13.106096+0000","last_unstale":"2023-11-28T13:01:13.110483+0000","last_undegraded":"2023-11-28T13:01:13.110483+0000","last_fullsized":"2023-11-28T13:01:13.110483+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102639+0000","last_change":"2023-11-28T13:01:13.098543+0000","last_active":"2023-11-28T13:01:13.102639+0000","last_peered":"2023-11-28T13:01:13.102639+0000","last_clean":"2023-11-28T13:01:13.102639+0000","last_became_active":"2023-11-28T13:01:13.097626+0000","last_became_peered":"2023-11-28T13:01:13.097626+0000","last_unstale":"2023-11-28T13:01:13.102639+0000","last_undegraded":"2023-11-28T13:01:13.102639+0000","last_fullsized":"2023-11-28T13:01:13.102639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104846+0000","last_change":"2023-11-28T13:01:13.097007+0000","last_active":"2023-11-28T13:01:13.104846+0000","last_peered":"2023-11-28T13:01:13.104846+0000","last_clean":"2023-11-28T13:01:13.104846+0000","last_became_active":"2023-11-28T13:01:13.096697+0000","last_became_peered":"2023-11-28T13:01:13.096697+0000","last_unstale":"2023-11-28T13:01:13.104846+0000","last_undegraded":"2023-11-28T13:01:13.104846+0000","last_fullsized":"2023-11-28T13:01:13.104846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102519+0000","last_change":"2023-11-28T13:01:13.092555+0000","last_active":"2023-11-28T13:01:13.102519+0000","last_peered":"2023-11-28T13:01:13.102519+0000","last_clean":"2023-11-28T13:01:13.102519+0000","last_became_active":"2023-11-28T13:01:13.092269+0000","last_became_peered":"2023-11-28T13:01:13.092269+0000","last_unstale":"2023-11-28T13:01:13.102519+0000","last_undegraded":"2023-11-28T13:01:13.102519+0000","last_fullsized":"2023-11-28T13:01:13.102519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110258+0000","last_change":"2023-11-28T13:01:13.106928+0000","last_active":"2023-11-28T13:01:13.110258+0000","last_peered":"2023-11-28T13:01:13.110258+0000","last_clean":"2023-11-28T13:01:13.110258+0000","last_became_active":"2023-11-28T13:01:13.104168+0000","last_became_peered":"2023-11-28T13:01:13.104168+0000","last_unstale":"2023-11-28T13:01:13.110258+0000","last_undegraded":"2023-11-28T13:01:13.110258+0000","last_fullsized":"2023-11-28T13:01:13.110258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110271+0000","last_change":"2023-11-28T13:01:13.106957+0000","last_active":"2023-11-28T13:01:13.110271+0000","last_peered":"2023-11-28T13:01:13.110271+0000","last_clean":"2023-11-28T13:01:13.110271+0000","last_became_active":"2023-11-28T13:01:13.104842+0000","last_became_peered":"2023-11-28T13:01:13.104842+0000","last_unstale":"2023-11-28T13:01:13.110271+0000","last_undegraded":"2023-11-28T13:01:13.110271+0000","last_fullsized":"2023-11-28T13:01:13.110271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105656+0000","last_change":"2023-11-28T13:01:13.102950+0000","last_active":"2023-11-28T13:01:13.105656+0000","last_peered":"2023-11-28T13:01:13.105656+0000","last_clean":"2023-11-28T13:01:13.105656+0000","last_became_active":"2023-11-28T13:01:13.102706+0000","last_became_peered":"2023-11-28T13:01:13.102706+0000","last_unstale":"2023-11-28T13:01:13.105656+0000","last_undegraded":"2023-11-28T13:01:13.105656+0000","last_fullsized":"2023-11-28T13:01:13.105656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:33.362111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.097705+0000","last_change":"2023-11-28T13:01:13.094603+0000","last_active":"2023-11-28T13:01:13.097705+0000","last_peered":"2023-11-28T13:01:13.097705+0000","last_clean":"2023-11-28T13:01:13.097705+0000","last_became_active":"2023-11-28T13:01:13.094377+0000","last_became_peered":"2023-11-28T13:01:13.094377+0000","last_unstale":"2023-11-28T13:01:13.097705+0000","last_undegraded":"2023-11-28T13:01:13.097705+0000","last_fullsized":"2023-11-28T13:01:13.097705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:25.095910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098253+0000","last_change":"2023-11-28T13:01:13.093425+0000","last_active":"2023-11-28T13:01:13.098253+0000","last_peered":"2023-11-28T13:01:13.098253+0000","last_clean":"2023-11-28T13:01:13.098253+0000","last_became_active":"2023-11-28T13:01:13.093128+0000","last_became_peered":"2023-11-28T13:01:13.093128+0000","last_unstale":"2023-11-28T13:01:13.098253+0000","last_undegraded":"2023-11-28T13:01:13.098253+0000","last_fullsized":"2023-11-28T13:01:13.098253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:47.380562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102465+0000","last_change":"2023-11-28T13:01:13.095333+0000","last_active":"2023-11-28T13:01:13.102465+0000","last_peered":"2023-11-28T13:01:13.102465+0000","last_clean":"2023-11-28T13:01:13.102465+0000","last_became_active":"2023-11-28T13:01:13.095050+0000","last_became_peered":"2023-11-28T13:01:13.095050+0000","last_unstale":"2023-11-28T13:01:13.102465+0000","last_undegraded":"2023-11-28T13:01:13.102465+0000","last_fullsized":"2023-11-28T13:01:13.102465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:51:32.434761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107125+0000","last_change":"2023-11-28T13:01:13.105213+0000","last_active":"2023-11-28T13:01:13.107125+0000","last_peered":"2023-11-28T13:01:13.107125+0000","last_clean":"2023-11-28T13:01:13.107125+0000","last_became_active":"2023-11-28T13:01:13.104960+0000","last_became_peered":"2023-11-28T13:01:13.104960+0000","last_unstale":"2023-11-28T13:01:13.107125+0000","last_undegraded":"2023-11-28T13:01:13.107125+0000","last_fullsized":"2023-11-28T13:01:13.107125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110113+0000","last_change":"2023-11-28T13:01:13.102193+0000","last_active":"2023-11-28T13:01:13.110113+0000","last_peered":"2023-11-28T13:01:13.110113+0000","last_clean":"2023-11-28T13:01:13.110113+0000","last_became_active":"2023-11-28T13:01:13.101759+0000","last_became_peered":"2023-11-28T13:01:13.101759+0000","last_unstale":"2023-11-28T13:01:13.110113+0000","last_undegraded":"2023-11-28T13:01:13.110113+0000","last_fullsized":"2023-11-28T13:01:13.110113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:51.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103015+0000","last_change":"2023-11-28T13:01:13.098378+0000","last_active":"2023-11-28T13:01:13.103015+0000","last_peered":"2023-11-28T13:01:13.103015+0000","last_clean":"2023-11-28T13:01:13.103015+0000","last_became_active":"2023-11-28T13:01:13.098101+0000","last_became_peered":"2023-11-28T13:01:13.098101+0000","last_unstale":"2023-11-28T13:01:13.103015+0000","last_undegraded":"2023-11-28T13:01:13.103015+0000","last_fullsized":"2023-11-28T13:01:13.103015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:11.877653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098316+0000","last_change":"2023-11-28T13:01:13.094017+0000","last_active":"2023-11-28T13:01:13.098316+0000","last_peered":"2023-11-28T13:01:13.098316+0000","last_clean":"2023-11-28T13:01:13.098316+0000","last_became_active":"2023-11-28T13:01:13.093753+0000","last_became_peered":"2023-11-28T13:01:13.093753+0000","last_unstale":"2023-11-28T13:01:13.098316+0000","last_undegraded":"2023-11-28T13:01:13.098316+0000","last_fullsized":"2023-11-28T13:01:13.098316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:21.524121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089155+0000","last_change":"2023-11-28T13:01:13.103430+0000","last_active":"2023-11-28T13:01:14.089155+0000","last_peered":"2023-11-28T13:01:14.089155+0000","last_clean":"2023-11-28T13:01:14.089155+0000","last_became_active":"2023-11-28T13:01:13.102784+0000","last_became_peered":"2023-11-28T13:01:13.102784+0000","last_unstale":"2023-11-28T13:01:14.089155+0000","last_undegraded":"2023-11-28T13:01:14.089155+0000","last_fullsized":"2023-11-28T13:01:14.089155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:47.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098350+0000","last_change":"2023-11-28T13:01:13.092484+0000","last_active":"2023-11-28T13:01:13.098350+0000","last_peered":"2023-11-28T13:01:13.098350+0000","last_clean":"2023-11-28T13:01:13.098350+0000","last_became_active":"2023-11-28T13:01:13.092162+0000","last_became_peered":"2023-11-28T13:01:13.092162+0000","last_unstale":"2023-11-28T13:01:13.098350+0000","last_undegraded":"2023-11-28T13:01:13.098350+0000","last_fullsized":"2023-11-28T13:01:13.098350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:03.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107152+0000","last_change":"2023-11-28T13:01:13.100818+0000","last_active":"2023-11-28T13:01:13.107152+0000","last_peered":"2023-11-28T13:01:13.107152+0000","last_clean":"2023-11-28T13:01:13.107152+0000","last_became_active":"2023-11-28T13:01:13.100421+0000","last_became_peered":"2023-11-28T13:01:13.100421+0000","last_unstale":"2023-11-28T13:01:13.107152+0000","last_undegraded":"2023-11-28T13:01:13.107152+0000","last_fullsized":"2023-11-28T13:01:13.107152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098210+0000","last_change":"2023-11-28T13:01:13.093478+0000","last_active":"2023-11-28T13:01:13.098210+0000","last_peered":"2023-11-28T13:01:13.098210+0000","last_clean":"2023-11-28T13:01:13.098210+0000","last_became_active":"2023-11-28T13:01:13.093194+0000","last_became_peered":"2023-11-28T13:01:13.093194+0000","last_unstale":"2023-11-28T13:01:13.098210+0000","last_undegraded":"2023-11-28T13:01:13.098210+0000","last_fullsized":"2023-11-28T13:01:13.098210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:10.435728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098142+0000","last_change":"2023-11-28T13:01:13.088250+0000","last_active":"2023-11-28T13:01:13.098142+0000","last_peered":"2023-11-28T13:01:13.098142+0000","last_clean":"2023-11-28T13:01:13.098142+0000","last_became_active":"2023-11-28T13:01:13.087881+0000","last_became_peered":"2023-11-28T13:01:13.087881+0000","last_unstale":"2023-11-28T13:01:13.098142+0000","last_undegraded":"2023-11-28T13:01:13.098142+0000","last_fullsized":"2023-11-28T13:01:13.098142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:43.126488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110058+0000","last_change":"2023-11-28T13:01:13.102030+0000","last_active":"2023-11-28T13:01:13.110058+0000","last_peered":"2023-11-28T13:01:13.110058+0000","last_clean":"2023-11-28T13:01:13.110058+0000","last_became_active":"2023-11-28T13:01:13.101376+0000","last_became_peered":"2023-11-28T13:01:13.101376+0000","last_unstale":"2023-11-28T13:01:13.110058+0000","last_undegraded":"2023-11-28T13:01:13.110058+0000","last_fullsized":"2023-11-28T13:01:13.110058+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088757+0000","last_change":"2023-11-28T13:01:13.095736+0000","last_active":"2023-11-28T13:01:14.088757+0000","last_peered":"2023-11-28T13:01:14.088757+0000","last_clean":"2023-11-28T13:01:14.088757+0000","last_became_active":"2023-11-28T13:01:13.095469+0000","last_became_peered":"2023-11-28T13:01:13.095469+0000","last_unstale":"2023-11-28T13:01:14.088757+0000","last_undegraded":"2023-11-28T13:01:14.088757+0000","last_fullsized":"2023-11-28T13:01:14.088757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:45.817559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110226+0000","last_change":"2023-11-28T13:01:13.105146+0000","last_active":"2023-11-28T13:01:13.110226+0000","last_peered":"2023-11-28T13:01:13.110226+0000","last_clean":"2023-11-28T13:01:13.110226+0000","last_became_active":"2023-11-28T13:01:13.104852+0000","last_became_peered":"2023-11-28T13:01:13.104852+0000","last_unstale":"2023-11-28T13:01:13.110226+0000","last_undegraded":"2023-11-28T13:01:13.110226+0000","last_fullsized":"2023-11-28T13:01:13.110226+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104391+0000","last_change":"2023-11-28T13:01:12.697633+0000","last_active":"2023-11-28T13:01:13.104391+0000","last_peered":"2023-11-28T13:01:13.104391+0000","last_clean":"2023-11-28T13:01:13.104391+0000","last_became_active":"2023-11-28T13:01:12.697137+0000","last_became_peered":"2023-11-28T13:01:12.697137+0000","last_unstale":"2023-11-28T13:01:13.104391+0000","last_undegraded":"2023-11-28T13:01:13.104391+0000","last_fullsized":"2023-11-28T13:01:13.104391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:59.601679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089117+0000","last_change":"2023-11-28T13:01:13.104026+0000","last_active":"2023-11-28T13:01:14.089117+0000","last_peered":"2023-11-28T13:01:14.089117+0000","last_clean":"2023-11-28T13:01:14.089117+0000","last_became_active":"2023-11-28T13:01:13.103779+0000","last_became_peered":"2023-11-28T13:01:13.103779+0000","last_unstale":"2023-11-28T13:01:14.089117+0000","last_undegraded":"2023-11-28T13:01:14.089117+0000","last_fullsized":"2023-11-28T13:01:14.089117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:13.669167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.685907+0000","last_change":"2023-11-28T13:01:11.685907+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.685907+0000","last_undegraded":"2023-11-28T13:01:11.685907+0000","last_fullsized":"2023-11-28T13:01:11.685907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107203+0000","last_change":"2023-11-28T13:01:13.104225+0000","last_active":"2023-11-28T13:01:13.107203+0000","last_peered":"2023-11-28T13:01:13.107203+0000","last_clean":"2023-11-28T13:01:13.107203+0000","last_became_active":"2023-11-28T13:01:13.103835+0000","last_became_peered":"2023-11-28T13:01:13.103835+0000","last_unstale":"2023-11-28T13:01:13.107203+0000","last_undegraded":"2023-11-28T13:01:13.107203+0000","last_fullsized":"2023-11-28T13:01:13.107203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089210+0000","last_change":"2023-11-28T13:01:12.703691+0000","last_active":"2023-11-28T13:01:14.089210+0000","last_peered":"2023-11-28T13:01:14.089210+0000","last_clean":"2023-11-28T13:01:14.089210+0000","last_became_active":"2023-11-28T13:01:12.703398+0000","last_became_peered":"2023-11-28T13:01:12.703398+0000","last_unstale":"2023-11-28T13:01:14.089210+0000","last_undegraded":"2023-11-28T13:01:14.089210+0000","last_fullsized":"2023-11-28T13:01:14.089210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:24.093954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090818+0000","last_change":"2023-11-28T13:01:13.086251+0000","last_active":"2023-11-28T13:01:14.090818+0000","last_peered":"2023-11-28T13:01:14.090818+0000","last_clean":"2023-11-28T13:01:14.090818+0000","last_became_active":"2023-11-28T13:01:13.085269+0000","last_became_peered":"2023-11-28T13:01:13.085269+0000","last_unstale":"2023-11-28T13:01:14.090818+0000","last_undegraded":"2023-11-28T13:01:14.090818+0000","last_fullsized":"2023-11-28T13:01:14.090818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:04.381796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103953+0000","last_change":"2023-11-28T13:01:12.700070+0000","last_active":"2023-11-28T13:01:13.103953+0000","last_peered":"2023-11-28T13:01:13.103953+0000","last_clean":"2023-11-28T13:01:13.103953+0000","last_became_active":"2023-11-28T13:01:12.699788+0000","last_became_peered":"2023-11-28T13:01:12.699788+0000","last_unstale":"2023-11-28T13:01:13.103953+0000","last_undegraded":"2023-11-28T13:01:13.103953+0000","last_fullsized":"2023-11-28T13:01:13.103953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:37.142758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102494+0000","last_change":"2023-11-28T13:01:12.700665+0000","last_active":"2023-11-28T13:01:13.102494+0000","last_peered":"2023-11-28T13:01:13.102494+0000","last_clean":"2023-11-28T13:01:13.102494+0000","last_became_active":"2023-11-28T13:01:12.700285+0000","last_became_peered":"2023-11-28T13:01:12.700285+0000","last_unstale":"2023-11-28T13:01:13.102494+0000","last_undegraded":"2023-11-28T13:01:13.102494+0000","last_fullsized":"2023-11-28T13:01:13.102494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102539+0000","last_change":"2023-11-28T13:01:13.095159+0000","last_active":"2023-11-28T13:01:13.102539+0000","last_peered":"2023-11-28T13:01:13.102539+0000","last_clean":"2023-11-28T13:01:13.102539+0000","last_became_active":"2023-11-28T13:01:13.094857+0000","last_became_peered":"2023-11-28T13:01:13.094857+0000","last_unstale":"2023-11-28T13:01:13.102539+0000","last_undegraded":"2023-11-28T13:01:13.102539+0000","last_fullsized":"2023-11-28T13:01:13.102539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106011+0000","last_change":"2023-11-28T13:01:12.705296+0000","last_active":"2023-11-28T13:01:13.106011+0000","last_peered":"2023-11-28T13:01:13.106011+0000","last_clean":"2023-11-28T13:01:13.106011+0000","last_became_active":"2023-11-28T13:01:12.705013+0000","last_became_peered":"2023-11-28T13:01:12.705013+0000","last_unstale":"2023-11-28T13:01:13.106011+0000","last_undegraded":"2023-11-28T13:01:13.106011+0000","last_fullsized":"2023-11-28T13:01:13.106011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:46.378606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104868+0000","last_change":"2023-11-28T13:01:13.094747+0000","last_active":"2023-11-28T13:01:13.104868+0000","last_peered":"2023-11-28T13:01:13.104868+0000","last_clean":"2023-11-28T13:01:13.104868+0000","last_became_active":"2023-11-28T13:01:13.094318+0000","last_became_peered":"2023-11-28T13:01:13.094318+0000","last_unstale":"2023-11-28T13:01:13.104868+0000","last_undegraded":"2023-11-28T13:01:13.104868+0000","last_fullsized":"2023-11-28T13:01:13.104868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:27.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089212+0000","last_change":"2023-11-28T13:01:12.706058+0000","last_active":"2023-11-28T13:01:14.089212+0000","last_peered":"2023-11-28T13:01:14.089212+0000","last_clean":"2023-11-28T13:01:14.089212+0000","last_became_active":"2023-11-28T13:01:12.705853+0000","last_became_peered":"2023-11-28T13:01:12.705853+0000","last_unstale":"2023-11-28T13:01:14.089212+0000","last_undegraded":"2023-11-28T13:01:14.089212+0000","last_fullsized":"2023-11-28T13:01:14.089212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:10.875697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090633+0000","last_change":"2023-11-28T13:01:13.099221+0000","last_active":"2023-11-28T13:01:14.090633+0000","last_peered":"2023-11-28T13:01:14.090633+0000","last_clean":"2023-11-28T13:01:14.090633+0000","last_became_active":"2023-11-28T13:01:13.098525+0000","last_became_peered":"2023-11-28T13:01:13.098525+0000","last_unstale":"2023-11-28T13:01:14.090633+0000","last_undegraded":"2023-11-28T13:01:14.090633+0000","last_fullsized":"2023-11-28T13:01:14.090633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:01.070045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092473+0000","last_change":"2023-11-28T13:01:12.700642+0000","last_active":"2023-11-28T13:01:13.092473+0000","last_peered":"2023-11-28T13:01:13.092473+0000","last_clean":"2023-11-28T13:01:13.092473+0000","last_became_active":"2023-11-28T13:01:12.700381+0000","last_became_peered":"2023-11-28T13:01:12.700381+0000","last_unstale":"2023-11-28T13:01:13.092473+0000","last_undegraded":"2023-11-28T13:01:13.092473+0000","last_fullsized":"2023-11-28T13:01:13.092473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:37.268919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110533+0000","last_change":"2023-11-28T13:01:13.107217+0000","last_active":"2023-11-28T13:01:13.110533+0000","last_peered":"2023-11-28T13:01:13.110533+0000","last_clean":"2023-11-28T13:01:13.110533+0000","last_became_active":"2023-11-28T13:01:13.106967+0000","last_became_peered":"2023-11-28T13:01:13.106967+0000","last_unstale":"2023-11-28T13:01:13.110533+0000","last_undegraded":"2023-11-28T13:01:13.110533+0000","last_fullsized":"2023-11-28T13:01:13.110533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:01.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090884+0000","last_change":"2023-11-28T13:01:12.700536+0000","last_active":"2023-11-28T13:01:14.090884+0000","last_peered":"2023-11-28T13:01:14.090884+0000","last_clean":"2023-11-28T13:01:14.090884+0000","last_became_active":"2023-11-28T13:01:12.699782+0000","last_became_peered":"2023-11-28T13:01:12.699782+0000","last_unstale":"2023-11-28T13:01:14.090884+0000","last_undegraded":"2023-11-28T13:01:14.090884+0000","last_fullsized":"2023-11-28T13:01:14.090884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:17.146583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110165+0000","last_change":"2023-11-28T13:01:13.103462+0000","last_active":"2023-11-28T13:01:13.110165+0000","last_peered":"2023-11-28T13:01:13.110165+0000","last_clean":"2023-11-28T13:01:13.110165+0000","last_became_active":"2023-11-28T13:01:13.102855+0000","last_became_peered":"2023-11-28T13:01:13.102855+0000","last_unstale":"2023-11-28T13:01:13.110165+0000","last_undegraded":"2023-11-28T13:01:13.110165+0000","last_fullsized":"2023-11-28T13:01:13.110165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.987104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092356+0000","last_change":"2023-11-28T13:01:12.703891+0000","last_active":"2023-11-28T13:01:13.092356+0000","last_peered":"2023-11-28T13:01:13.092356+0000","last_clean":"2023-11-28T13:01:13.092356+0000","last_became_active":"2023-11-28T13:01:12.703642+0000","last_became_peered":"2023-11-28T13:01:12.703642+0000","last_unstale":"2023-11-28T13:01:13.092356+0000","last_undegraded":"2023-11-28T13:01:13.092356+0000","last_fullsized":"2023-11-28T13:01:13.092356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:32.360155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105022+0000","last_change":"2023-11-28T13:01:12.704636+0000","last_active":"2023-11-28T13:01:13.105022+0000","last_peered":"2023-11-28T13:01:13.105022+0000","last_clean":"2023-11-28T13:01:13.105022+0000","last_became_active":"2023-11-28T13:01:12.704357+0000","last_became_peered":"2023-11-28T13:01:12.704357+0000","last_unstale":"2023-11-28T13:01:13.105022+0000","last_undegraded":"2023-11-28T13:01:13.105022+0000","last_fullsized":"2023-11-28T13:01:13.105022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105277+0000","last_change":"2023-11-28T13:01:13.096971+0000","last_active":"2023-11-28T13:01:13.105277+0000","last_peered":"2023-11-28T13:01:13.105277+0000","last_clean":"2023-11-28T13:01:13.105277+0000","last_became_active":"2023-11-28T13:01:13.096698+0000","last_became_peered":"2023-11-28T13:01:13.096698+0000","last_unstale":"2023-11-28T13:01:13.105277+0000","last_undegraded":"2023-11-28T13:01:13.105277+0000","last_fullsized":"2023-11-28T13:01:13.105277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090728+0000","last_change":"2023-11-28T13:01:12.704846+0000","last_active":"2023-11-28T13:01:14.090728+0000","last_peered":"2023-11-28T13:01:14.090728+0000","last_clean":"2023-11-28T13:01:14.090728+0000","last_became_active":"2023-11-28T13:01:12.704605+0000","last_became_peered":"2023-11-28T13:01:12.704605+0000","last_unstale":"2023-11-28T13:01:14.090728+0000","last_undegraded":"2023-11-28T13:01:14.090728+0000","last_fullsized":"2023-11-28T13:01:14.090728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:43.923726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090672+0000","last_change":"2023-11-28T13:01:12.700930+0000","last_active":"2023-11-28T13:01:14.090672+0000","last_peered":"2023-11-28T13:01:14.090672+0000","last_clean":"2023-11-28T13:01:14.090672+0000","last_became_active":"2023-11-28T13:01:12.700670+0000","last_became_peered":"2023-11-28T13:01:12.700670+0000","last_unstale":"2023-11-28T13:01:14.090672+0000","last_undegraded":"2023-11-28T13:01:14.090672+0000","last_fullsized":"2023-11-28T13:01:14.090672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090509+0000","last_change":"2023-11-28T13:01:13.093993+0000","last_active":"2023-11-28T13:01:14.090509+0000","last_peered":"2023-11-28T13:01:14.090509+0000","last_clean":"2023-11-28T13:01:14.090509+0000","last_became_active":"2023-11-28T13:01:13.093750+0000","last_became_peered":"2023-11-28T13:01:13.093750+0000","last_unstale":"2023-11-28T13:01:14.090509+0000","last_undegraded":"2023-11-28T13:01:14.090509+0000","last_fullsized":"2023-11-28T13:01:14.090509+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:47.159742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088952+0000","last_change":"2023-11-28T13:01:12.706087+0000","last_active":"2023-11-28T13:01:14.088952+0000","last_peered":"2023-11-28T13:01:14.088952+0000","last_clean":"2023-11-28T13:01:14.088952+0000","last_became_active":"2023-11-28T13:01:12.705800+0000","last_became_peered":"2023-11-28T13:01:12.705800+0000","last_unstale":"2023-11-28T13:01:14.088952+0000","last_undegraded":"2023-11-28T13:01:14.088952+0000","last_fullsized":"2023-11-28T13:01:14.088952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102554+0000","last_change":"2023-11-28T13:01:13.101116+0000","last_active":"2023-11-28T13:01:13.102554+0000","last_peered":"2023-11-28T13:01:13.102554+0000","last_clean":"2023-11-28T13:01:13.102554+0000","last_became_active":"2023-11-28T13:01:13.100510+0000","last_became_peered":"2023-11-28T13:01:13.100510+0000","last_unstale":"2023-11-28T13:01:13.102554+0000","last_undegraded":"2023-11-28T13:01:13.102554+0000","last_fullsized":"2023-11-28T13:01:13.102554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:42.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081264+0000","last_change":"2023-11-28T13:01:12.702749+0000","last_active":"2023-11-28T13:01:13.081264+0000","last_peered":"2023-11-28T13:01:13.081264+0000","last_clean":"2023-11-28T13:01:13.081264+0000","last_became_active":"2023-11-28T13:01:12.702450+0000","last_became_peered":"2023-11-28T13:01:12.702450+0000","last_unstale":"2023-11-28T13:01:13.081264+0000","last_undegraded":"2023-11-28T13:01:13.081264+0000","last_fullsized":"2023-11-28T13:01:13.081264+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:38.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090531+0000","last_change":"2023-11-28T13:01:13.095487+0000","last_active":"2023-11-28T13:01:14.090531+0000","last_peered":"2023-11-28T13:01:14.090531+0000","last_clean":"2023-11-28T13:01:14.090531+0000","last_became_active":"2023-11-28T13:01:13.095222+0000","last_became_peered":"2023-11-28T13:01:13.095222+0000","last_unstale":"2023-11-28T13:01:14.090531+0000","last_undegraded":"2023-11-28T13:01:14.090531+0000","last_fullsized":"2023-11-28T13:01:14.090531+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:57.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106265+0000","last_change":"2023-11-28T13:01:12.704714+0000","last_active":"2023-11-28T13:01:13.106265+0000","last_peered":"2023-11-28T13:01:13.106265+0000","last_clean":"2023-11-28T13:01:13.106265+0000","last_became_active":"2023-11-28T13:01:12.704422+0000","last_became_peered":"2023-11-28T13:01:12.704422+0000","last_unstale":"2023-11-28T13:01:13.106265+0000","last_undegraded":"2023-11-28T13:01:13.106265+0000","last_fullsized":"2023-11-28T13:01:13.106265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:42.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090726+0000","last_change":"2023-11-28T13:01:13.088050+0000","last_active":"2023-11-28T13:01:14.090726+0000","last_peered":"2023-11-28T13:01:14.090726+0000","last_clean":"2023-11-28T13:01:14.090726+0000","last_became_active":"2023-11-28T13:01:13.085247+0000","last_became_peered":"2023-11-28T13:01:13.085247+0000","last_unstale":"2023-11-28T13:01:14.090726+0000","last_undegraded":"2023-11-28T13:01:14.090726+0000","last_fullsized":"2023-11-28T13:01:14.090726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:39.035787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107176+0000","last_change":"2023-11-28T13:01:12.702173+0000","last_active":"2023-11-28T13:01:13.107176+0000","last_peered":"2023-11-28T13:01:13.107176+0000","last_clean":"2023-11-28T13:01:13.107176+0000","last_became_active":"2023-11-28T13:01:12.701243+0000","last_became_peered":"2023-11-28T13:01:12.701243+0000","last_unstale":"2023-11-28T13:01:13.107176+0000","last_undegraded":"2023-11-28T13:01:13.107176+0000","last_fullsized":"2023-11-28T13:01:13.107176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102621+0000","last_change":"2023-11-28T13:01:13.099301+0000","last_active":"2023-11-28T13:01:13.102621+0000","last_peered":"2023-11-28T13:01:13.102621+0000","last_clean":"2023-11-28T13:01:13.102621+0000","last_became_active":"2023-11-28T13:01:13.099017+0000","last_became_peered":"2023-11-28T13:01:13.099017+0000","last_unstale":"2023-11-28T13:01:13.102621+0000","last_undegraded":"2023-11-28T13:01:13.102621+0000","last_fullsized":"2023-11-28T13:01:13.102621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104381+0000","last_change":"2023-11-28T13:01:12.706078+0000","last_active":"2023-11-28T13:01:13.104381+0000","last_peered":"2023-11-28T13:01:13.104381+0000","last_clean":"2023-11-28T13:01:13.104381+0000","last_became_active":"2023-11-28T13:01:12.705694+0000","last_became_peered":"2023-11-28T13:01:12.705694+0000","last_unstale":"2023-11-28T13:01:13.104381+0000","last_undegraded":"2023-11-28T13:01:13.104381+0000","last_fullsized":"2023-11-28T13:01:13.104381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:41.987277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105893+0000","last_change":"2023-11-28T13:01:13.098692+0000","last_active":"2023-11-28T13:01:13.105893+0000","last_peered":"2023-11-28T13:01:13.105893+0000","last_clean":"2023-11-28T13:01:13.105893+0000","last_became_active":"2023-11-28T13:01:13.098426+0000","last_became_peered":"2023-11-28T13:01:13.098426+0000","last_unstale":"2023-11-28T13:01:13.105893+0000","last_undegraded":"2023-11-28T13:01:13.105893+0000","last_fullsized":"2023-11-28T13:01:13.105893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.101277+0000","last_change":"2023-11-28T13:01:12.701155+0000","last_active":"2023-11-28T13:01:13.101277+0000","last_peered":"2023-11-28T13:01:13.101277+0000","last_clean":"2023-11-28T13:01:13.101277+0000","last_became_active":"2023-11-28T13:01:12.700897+0000","last_became_peered":"2023-11-28T13:01:12.700897+0000","last_unstale":"2023-11-28T13:01:13.101277+0000","last_undegraded":"2023-11-28T13:01:13.101277+0000","last_fullsized":"2023-11-28T13:01:13.101277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:57.063655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.100622+0000","last_change":"2023-11-28T13:01:13.094885+0000","last_active":"2023-11-28T13:01:13.100622+0000","last_peered":"2023-11-28T13:01:13.100622+0000","last_clean":"2023-11-28T13:01:13.100622+0000","last_became_active":"2023-11-28T13:01:13.094027+0000","last_became_peered":"2023-11-28T13:01:13.094027+0000","last_unstale":"2023-11-28T13:01:13.100622+0000","last_undegraded":"2023-11-28T13:01:13.100622+0000","last_fullsized":"2023-11-28T13:01:13.100622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:43.321762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089094+0000","last_change":"2023-11-28T13:01:12.703499+0000","last_active":"2023-11-28T13:01:14.089094+0000","last_peered":"2023-11-28T13:01:14.089094+0000","last_clean":"2023-11-28T13:01:14.089094+0000","last_became_active":"2023-11-28T13:01:12.703271+0000","last_became_peered":"2023-11-28T13:01:12.703271+0000","last_unstale":"2023-11-28T13:01:14.089094+0000","last_undegraded":"2023-11-28T13:01:14.089094+0000","last_fullsized":"2023-11-28T13:01:14.089094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:03:30.119943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107824+0000","last_change":"2023-11-28T13:01:13.101678+0000","last_active":"2023-11-28T13:01:13.107824+0000","last_peered":"2023-11-28T13:01:13.107824+0000","last_clean":"2023-11-28T13:01:13.107824+0000","last_became_active":"2023-11-28T13:01:13.101386+0000","last_became_peered":"2023-11-28T13:01:13.101386+0000","last_unstale":"2023-11-28T13:01:13.107824+0000","last_undegraded":"2023-11-28T13:01:13.107824+0000","last_fullsized":"2023-11-28T13:01:13.107824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:51.226552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.095695+0000","last_change":"2023-11-28T13:01:12.697001+0000","last_active":"2023-11-28T13:01:13.095695+0000","last_peered":"2023-11-28T13:01:13.095695+0000","last_clean":"2023-11-28T13:01:13.095695+0000","last_became_active":"2023-11-28T13:01:12.696739+0000","last_became_peered":"2023-11-28T13:01:12.696739+0000","last_unstale":"2023-11-28T13:01:13.095695+0000","last_undegraded":"2023-11-28T13:01:13.095695+0000","last_fullsized":"2023-11-28T13:01:13.095695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:37.438564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083496+0000","last_change":"2023-11-28T13:01:12.698523+0000","last_active":"2023-11-28T13:01:13.083496+0000","last_peered":"2023-11-28T13:01:13.083496+0000","last_clean":"2023-11-28T13:01:13.083496+0000","last_became_active":"2023-11-28T13:01:12.698275+0000","last_became_peered":"2023-11-28T13:01:12.698275+0000","last_unstale":"2023-11-28T13:01:13.083496+0000","last_undegraded":"2023-11-28T13:01:13.083496+0000","last_fullsized":"2023-11-28T13:01:13.083496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:02.254485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102691+0000","last_change":"2023-11-28T13:01:13.099007+0000","last_active":"2023-11-28T13:01:13.102691+0000","last_peered":"2023-11-28T13:01:13.102691+0000","last_clean":"2023-11-28T13:01:13.102691+0000","last_became_active":"2023-11-28T13:01:13.098673+0000","last_became_peered":"2023-11-28T13:01:13.098673+0000","last_unstale":"2023-11-28T13:01:13.102691+0000","last_undegraded":"2023-11-28T13:01:13.102691+0000","last_fullsized":"2023-11-28T13:01:13.102691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091666+0000","last_change":"2023-11-28T13:01:12.706237+0000","last_active":"2023-11-28T13:01:13.091666+0000","last_peered":"2023-11-28T13:01:13.091666+0000","last_clean":"2023-11-28T13:01:13.091666+0000","last_became_active":"2023-11-28T13:01:12.705981+0000","last_became_peered":"2023-11-28T13:01:12.705981+0000","last_unstale":"2023-11-28T13:01:13.091666+0000","last_undegraded":"2023-11-28T13:01:13.091666+0000","last_fullsized":"2023-11-28T13:01:13.091666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090578+0000","last_change":"2023-11-28T13:01:13.099382+0000","last_active":"2023-11-28T13:01:14.090578+0000","last_peered":"2023-11-28T13:01:14.090578+0000","last_clean":"2023-11-28T13:01:14.090578+0000","last_became_active":"2023-11-28T13:01:13.099008+0000","last_became_peered":"2023-11-28T13:01:13.099008+0000","last_unstale":"2023-11-28T13:01:14.090578+0000","last_undegraded":"2023-11-28T13:01:14.090578+0000","last_fullsized":"2023-11-28T13:01:14.090578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:35.600930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083382+0000","last_change":"2023-11-28T13:01:12.696728+0000","last_active":"2023-11-28T13:01:13.083382+0000","last_peered":"2023-11-28T13:01:13.083382+0000","last_clean":"2023-11-28T13:01:13.083382+0000","last_became_active":"2023-11-28T13:01:12.696428+0000","last_became_peered":"2023-11-28T13:01:12.696428+0000","last_unstale":"2023-11-28T13:01:13.083382+0000","last_undegraded":"2023-11-28T13:01:13.083382+0000","last_fullsized":"2023-11-28T13:01:13.083382+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:28.781799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.686000+0000","last_change":"2023-11-28T13:01:11.686000+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.686000+0000","last_undegraded":"2023-11-28T13:01:11.686000+0000","last_fullsized":"2023-11-28T13:01:11.686000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091534+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:01:13.091534+0000","last_peered":"2023-11-28T13:01:13.091534+0000","last_clean":"2023-11-28T13:01:13.091534+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:01:13.091534+0000","last_undegraded":"2023-11-28T13:01:13.091534+0000","last_fullsized":"2023-11-28T13:01:13.091534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090805+0000","last_change":"2023-11-28T13:01:13.098873+0000","last_active":"2023-11-28T13:01:14.090805+0000","last_peered":"2023-11-28T13:01:14.090805+0000","last_clean":"2023-11-28T13:01:14.090805+0000","last_became_active":"2023-11-28T13:01:13.098618+0000","last_became_peered":"2023-11-28T13:01:13.098618+0000","last_unstale":"2023-11-28T13:01:14.090805+0000","last_undegraded":"2023-11-28T13:01:14.090805+0000","last_fullsized":"2023-11-28T13:01:14.090805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:49.374140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.686366+0000","last_change":"2023-11-28T13:01:11.686366+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.686366+0000","last_undegraded":"2023-11-28T13:01:11.686366+0000","last_fullsized":"2023-11-28T13:01:11.686366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699954+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:01:12.699954+0000","last_peered":"2023-11-28T13:01:12.699954+0000","last_clean":"2023-11-28T13:01:12.699954+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:01:12.699954+0000","last_undegraded":"2023-11-28T13:01:12.699954+0000","last_fullsized":"2023-11-28T13:01:12.699954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110193+0000","last_change":"2023-11-28T13:01:13.103633+0000","last_active":"2023-11-28T13:01:13.110193+0000","last_peered":"2023-11-28T13:01:13.110193+0000","last_clean":"2023-11-28T13:01:13.110193+0000","last_became_active":"2023-11-28T13:01:13.103198+0000","last_became_peered":"2023-11-28T13:01:13.103198+0000","last_unstale":"2023-11-28T13:01:13.110193+0000","last_undegraded":"2023-11-28T13:01:13.110193+0000","last_fullsized":"2023-11-28T13:01:13.110193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.084615+0000","last_change":"2023-11-28T13:01:12.704388+0000","last_active":"2023-11-28T13:01:13.084615+0000","last_peered":"2023-11-28T13:01:13.084615+0000","last_clean":"2023-11-28T13:01:13.084615+0000","last_became_active":"2023-11-28T13:01:12.704082+0000","last_became_peered":"2023-11-28T13:01:12.704082+0000","last_unstale":"2023-11-28T13:01:13.084615+0000","last_undegraded":"2023-11-28T13:01:13.084615+0000","last_fullsized":"2023-11-28T13:01:13.084615+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:08.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.685641+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:01:12.685641+0000","last_peered":"2023-11-28T13:01:12.685641+0000","last_clean":"2023-11-28T13:01:12.685641+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:01:12.685641+0000","last_undegraded":"2023-11-28T13:01:12.685641+0000","last_fullsized":"2023-11-28T13:01:12.685641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110430+0000","last_change":"2023-11-28T13:01:13.105495+0000","last_active":"2023-11-28T13:01:13.110430+0000","last_peered":"2023-11-28T13:01:13.110430+0000","last_clean":"2023-11-28T13:01:13.110430+0000","last_became_active":"2023-11-28T13:01:13.103141+0000","last_became_peered":"2023-11-28T13:01:13.103141+0000","last_unstale":"2023-11-28T13:01:13.110430+0000","last_undegraded":"2023-11-28T13:01:13.110430+0000","last_fullsized":"2023-11-28T13:01:13.110430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:08.241448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091757+0000","last_change":"2023-11-28T13:01:12.704702+0000","last_active":"2023-11-28T13:01:13.091757+0000","last_peered":"2023-11-28T13:01:13.091757+0000","last_clean":"2023-11-28T13:01:13.091757+0000","last_became_active":"2023-11-28T13:01:12.704416+0000","last_became_peered":"2023-11-28T13:01:12.704416+0000","last_unstale":"2023-11-28T13:01:13.091757+0000","last_undegraded":"2023-11-28T13:01:13.091757+0000","last_fullsized":"2023-11-28T13:01:13.091757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:02.624167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.700208+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:01:12.700208+0000","last_peered":"2023-11-28T13:01:12.700208+0000","last_clean":"2023-11-28T13:01:12.700208+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:01:12.700208+0000","last_undegraded":"2023-11-28T13:01:12.700208+0000","last_fullsized":"2023-11-28T13:01:12.700208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698573+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:01:12.698573+0000","last_peered":"2023-11-28T13:01:12.698573+0000","last_clean":"2023-11-28T13:01:12.698573+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:01:12.698573+0000","last_undegraded":"2023-11-28T13:01:12.698573+0000","last_fullsized":"2023-11-28T13:01:12.698573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088962+0000","last_change":"2023-11-28T13:01:13.104062+0000","last_active":"2023-11-28T13:01:14.088962+0000","last_peered":"2023-11-28T13:01:14.088962+0000","last_clean":"2023-11-28T13:01:14.088962+0000","last_became_active":"2023-11-28T13:01:13.103815+0000","last_became_peered":"2023-11-28T13:01:13.103815+0000","last_unstale":"2023-11-28T13:01:14.088962+0000","last_undegraded":"2023-11-28T13:01:14.088962+0000","last_fullsized":"2023-11-28T13:01:14.088962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:09.350952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106674+0000","last_change":"2023-11-28T13:01:12.706680+0000","last_active":"2023-11-28T13:01:13.106674+0000","last_peered":"2023-11-28T13:01:13.106674+0000","last_clean":"2023-11-28T13:01:13.106674+0000","last_became_active":"2023-11-28T13:01:12.706387+0000","last_became_peered":"2023-11-28T13:01:12.706387+0000","last_unstale":"2023-11-28T13:01:13.106674+0000","last_undegraded":"2023-11-28T13:01:13.106674+0000","last_fullsized":"2023-11-28T13:01:13.106674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698664+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:01:12.698664+0000","last_peered":"2023-11-28T13:01:12.698664+0000","last_clean":"2023-11-28T13:01:12.698664+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:01:12.698664+0000","last_undegraded":"2023-11-28T13:01:12.698664+0000","last_fullsized":"2023-11-28T13:01:12.698664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088854+0000","last_change":"2023-11-28T13:01:13.100011+0000","last_active":"2023-11-28T13:01:14.088854+0000","last_peered":"2023-11-28T13:01:14.088854+0000","last_clean":"2023-11-28T13:01:14.088854+0000","last_became_active":"2023-11-28T13:01:13.099738+0000","last_became_peered":"2023-11-28T13:01:13.099738+0000","last_unstale":"2023-11-28T13:01:14.088854+0000","last_undegraded":"2023-11-28T13:01:14.088854+0000","last_fullsized":"2023-11-28T13:01:14.088854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:52.246680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081223+0000","last_change":"2023-11-28T13:01:12.694952+0000","last_active":"2023-11-28T13:01:13.081223+0000","last_peered":"2023-11-28T13:01:13.081223+0000","last_clean":"2023-11-28T13:01:13.081223+0000","last_became_active":"2023-11-28T13:01:12.694694+0000","last_became_peered":"2023-11-28T13:01:12.694694+0000","last_unstale":"2023-11-28T13:01:13.081223+0000","last_undegraded":"2023-11-28T13:01:13.081223+0000","last_fullsized":"2023-11-28T13:01:13.081223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:54.638737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699297+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:01:12.699297+0000","last_peered":"2023-11-28T13:01:12.699297+0000","last_clean":"2023-11-28T13:01:12.699297+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:01:12.699297+0000","last_undegraded":"2023-11-28T13:01:12.699297+0000","last_fullsized":"2023-11-28T13:01:12.699297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104886+0000","last_change":"2023-11-28T13:01:13.099302+0000","last_active":"2023-11-28T13:01:13.104886+0000","last_peered":"2023-11-28T13:01:13.104886+0000","last_clean":"2023-11-28T13:01:13.104886+0000","last_became_active":"2023-11-28T13:01:13.099047+0000","last_became_peered":"2023-11-28T13:01:13.099047+0000","last_unstale":"2023-11-28T13:01:13.104886+0000","last_undegraded":"2023-11-28T13:01:13.104886+0000","last_fullsized":"2023-11-28T13:01:13.104886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092648+0000","last_change":"2023-11-28T13:01:12.700645+0000","last_active":"2023-11-28T13:01:13.092648+0000","last_peered":"2023-11-28T13:01:13.092648+0000","last_clean":"2023-11-28T13:01:13.092648+0000","last_became_active":"2023-11-28T13:01:12.700349+0000","last_became_peered":"2023-11-28T13:01:12.700349+0000","last_unstale":"2023-11-28T13:01:13.092648+0000","last_undegraded":"2023-11-28T13:01:13.092648+0000","last_fullsized":"2023-11-28T13:01:13.092648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:50.224596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.086932+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:01:13.086932+0000","last_peered":"2023-11-28T13:01:13.086932+0000","last_clean":"2023-11-28T13:01:13.086932+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:01:13.086932+0000","last_undegraded":"2023-11-28T13:01:13.086932+0000","last_fullsized":"2023-11-28T13:01:13.086932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107847+0000","last_change":"2023-11-28T13:01:13.101838+0000","last_active":"2023-11-28T13:01:13.107847+0000","last_peered":"2023-11-28T13:01:13.107847+0000","last_clean":"2023-11-28T13:01:13.107847+0000","last_became_active":"2023-11-28T13:01:13.101565+0000","last_became_peered":"2023-11-28T13:01:13.101565+0000","last_unstale":"2023-11-28T13:01:13.107847+0000","last_undegraded":"2023-11-28T13:01:13.107847+0000","last_fullsized":"2023-11-28T13:01:13.107847+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:08.927902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090444+0000","last_change":"2023-11-28T13:01:12.699934+0000","last_active":"2023-11-28T13:01:14.090444+0000","last_peered":"2023-11-28T13:01:14.090444+0000","last_clean":"2023-11-28T13:01:14.090444+0000","last_became_active":"2023-11-28T13:01:12.699617+0000","last_became_peered":"2023-11-28T13:01:12.699617+0000","last_unstale":"2023-11-28T13:01:14.090444+0000","last_undegraded":"2023-11-28T13:01:14.090444+0000","last_fullsized":"2023-11-28T13:01:14.090444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:02.801702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.690011+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:01:12.690011+0000","last_peered":"2023-11-28T13:01:12.690011+0000","last_clean":"2023-11-28T13:01:12.690011+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:01:12.690011+0000","last_undegraded":"2023-11-28T13:01:12.690011+0000","last_fullsized":"2023-11-28T13:01:12.690011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":184228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.497},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.218,"5min":8.218,"15min":8.218},"last":0.35099999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.439},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":13.384,"5min":13.384,"15min":13.384},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.226,"5min":5.226,"15min":5.226},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.624}]},{"osd":5,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.3209999999999997,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.1029999999999998,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.3849999999999998,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.70799999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8364,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363476,"statfs":{"total":96636764160,"available":96628199424,"internally_reserved":0,"allocated":430080,"data_stored":176063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.27200000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.247},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42799999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.498},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":168865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.438},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9424,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362416,"statfs":{"total":96636764160,"available":96627113984,"internally_reserved":0,"allocated":991232,"data_stored":751836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":14.997999999999999,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.28299999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":19.045999999999999,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.497},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":18.995000000000001,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":19.02,"5min":19.02,"15min":19.02},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.45000000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.438}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.1539999999999999,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":18.957999999999998,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.51300000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9428,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362412,"statfs":{"total":96636764160,"available":96627109888,"internally_reserved":0,"allocated":995328,"data_stored":752605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66300000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":4.7830000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.9510000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":4.8440000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":2.6030000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.87}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":1.25},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":2.5329999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.972},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":2.508}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8496,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363344,"statfs":{"total":96636764160,"available":96628064256,"internally_reserved":0,"allocated":434176,"data_stored":176105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.435},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.67300000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69599999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8428,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363412,"statfs":{"total":96636764160,"available":96628133888,"internally_reserved":0,"allocated":430080,"data_stored":176559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.629}]},{"osd":3,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.503}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.78000000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8492,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363348,"statfs":{"total":96636764160,"available":96628068352,"internally_reserved":0,"allocated":430080,"data_stored":176083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.247}]},{"osd":2,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.37}]},{"osd":3,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:01:17.170 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:01:17.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.323+0000 7fba01932700 1 Processor -- start 2023-11-28T13:01:17.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.323+0000 7fba01932700 1 -- start start 2023-11-28T13:01:17.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fba01932700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 0x7fb9fc06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fba01932700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc1615c0 con 0x7fb9fc15cd30 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fba01932700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0d0840 con 0x7fb9fc06a7b0 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fba01932700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc06ac30 con 0x7fb9fc161150 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9faffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb9fc15cd30 0x7fb9fc0cfda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58418/0 (socket says 172.21.15.153:58418) 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9faffd700 1 -- 172.21.15.153:0/3273999917 learned_addr learned my addr 172.21.15.153:0/3273999917 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995144036 0 0) 0x7fb9f0001110 con 0x7fb9fc15cd30 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9fa7fc700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 0x7fb9fc06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e40035b0 con 0x7fb9fc15cd30 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331704152 0 0) 0x7fb9e40035b0 con 0x7fb9fc15cd30 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 msgr2=0x7fb9fc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 0x7fb9fc06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 >> v1:172.21.15.177:6789/0 conn(0x7fb9fc06a7b0 legacy=0x7fb9fc070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0d0b20 con 0x7fb9fc15cd30 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fb9f0001800 con 0x7fb9fc15cd30 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 >> v1:172.21.15.153:6789/0 conn(0x7fb9fc15cd30 legacy=0x7fb9fc0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e4003a30 0x7fb9e4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4004850 0x7fb9e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 0x7fb9e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc06ac30 con 0x7fb9e4003a30 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e400bac0 con 0x7fb9e4004850 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e400bc80 con 0x7fb9fc161150 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.324+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e4003a30 0x7fb9e4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9fa7fc700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4004850 0x7fb9e4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9fb7fe700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 0x7fb9e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9faffd700 1 -- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 msgr2=0x7fb9e4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 0x7fb9e4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9faffd700 1 -- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4004850 msgr2=0x7fb9e4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4004850 0x7fb9e4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9faffd700 1 -- 172.21.15.153:0/3273999917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e40035b0 con 0x7fb9e4003a30 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9fb7fe700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc161150 0x7fb9e4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e4003a30 0x7fb9e4003e40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c240 tx=0x7fb9e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863a0ca18140a069 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0007db0 con 0x7fb9e4003a30 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f000ce00 con 0x7fb9e4003a30 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9f9ffb700 1 -- 172.21.15.153:0/3273999917 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0005ba0 con 0x7fb9e4003a30 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fba01932700 1 -- 172.21.15.153:0/3273999917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e4003a30 msgr2=0x7fb9e4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fba01932700 1 --2- 172.21.15.153:0/3273999917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9e4003a30 0x7fb9e4003e40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c240 tx=0x7fb9e4005880 comp rx=0 tx=0).stop 2023-11-28T13:01:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.325+0000 7fb9fb7fe700 1 -- 172.21.15.153:0/3273999917 reap_dead start 2023-11-28T13:01:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.326+0000 7fba01932700 1 -- 172.21.15.153:0/3273999917 shutdown_connections 2023-11-28T13:01:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.326+0000 7fba01932700 1 -- 172.21.15.153:0/3273999917 >> 172.21.15.153:0/3273999917 conn(0x7fb9fc06b380 msgr2=0x7fb9fc06b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.326+0000 7fba01932700 1 -- 172.21.15.153:0/3273999917 shutdown_connections 2023-11-28T13:01:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.326+0000 7fba01932700 1 -- 172.21.15.153:0/3273999917 wait complete. 2023-11-28T13:01:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.327+0000 7fba01932700 1 Processor -- start 2023-11-28T13:01:17.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.327+0000 7fba01932700 1 -- start start 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fba01932700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4003a30 0x7fb9fc1f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fba01932700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc1f5b20 0x7fb9fc0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fba01932700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9fc14d1f0 0x7fb9fc14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fba01932700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc1f3510 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fba01932700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc14cac0 con 0x7fb9e4003a30 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fba01932700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc1f6270 con 0x7fb9fc1f5b20 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fb9fb7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9fc14d1f0 0x7fb9fc14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fb9fb7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9fc14d1f0 0x7fb9fc14d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40270/0 (socket says 172.21.15.153:40270) 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fb9fb7fe700 1 -- 172.21.15.153:0/3596078353 learned_addr learned my addr 172.21.15.153:0/3596078353 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fb9fa7fc700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc1f5b20 0x7fb9fc0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.328+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4003a30 0x7fb9fc1f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9fb7fe700 1 -- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc1f5b20 msgr2=0x7fb9fc0b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9fb7fe700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc1f5b20 0x7fb9fc0b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9fb7fe700 1 -- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4003a30 msgr2=0x7fb9fc1f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9fb7fe700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4003a30 0x7fb9fc1f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9fb7fe700 1 -- 172.21.15.153:0/3596078353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc160950 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4003a30 0x7fb9fc1f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9fb7fe700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9fc14d1f0 0x7fb9fc14d660 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0708d0 tx=0x7fb9f400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9e3fff700 1 -- 172.21.15.153:0/3596078353 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4003e10 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9e3fff700 1 -- 172.21.15.153:0/3596078353 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f4005070 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc160670 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fb9e3fff700 1 -- 172.21.15.153:0/3596078353 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4005420 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.329+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc160c30 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.330+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc0745c0 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.330+0000 7fb9e3fff700 1 -- 172.21.15.153:0/3596078353 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9f401a030 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.331+0000 7fb9e3fff700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9e80682f0 0x7fb9e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.331+0000 7fb9e3fff700 1 -- 172.21.15.153:0/3596078353 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb9f4085750 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.331+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9e80682f0 0x7fb9e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.331+0000 7fb9faffd700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9e80682f0 0x7fb9e806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0000c00 tx=0x7fb9f0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.335+0000 7fb9e3fff700 1 -- 172.21.15.153:0/3596078353 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9f4051fb0 con 0x7fb9fc14d1f0 2023-11-28T13:01:17.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.512+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9fc15b990 con 0x7fb9e80682f0 2023-11-28T13:01:17.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.520+0000 7fb9e3fff700 1 -- 172.21.15.153:0/3596078353 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309773 (secure 0 0 0) 0x7fb9fc15b990 con 0x7fb9e80682f0 2023-11-28T13:01:17.525 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:17.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9e80682f0 msgr2=0x7fb9e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9e80682f0 0x7fb9e806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0000c00 tx=0x7fb9f0007000 comp rx=0 tx=0).stop 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9fc14d1f0 msgr2=0x7fb9fc14d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9fc14d1f0 0x7fb9fc14d660 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0708d0 tx=0x7fb9f400de70 comp rx=0 tx=0).stop 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 shutdown_connections 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9e80682f0 0x7fb9e806a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9fc14d1f0 0x7fb9fc14d660 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9e4003a30 0x7fb9fc1f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 --2- 172.21.15.153:0/3596078353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9fc1f5b20 0x7fb9fc0b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 >> 172.21.15.153:0/3596078353 conn(0x7fb9fc06b380 msgr2=0x7fb9fc06b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 shutdown_connections 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.523+0000 7fba01932700 1 -- 172.21.15.153:0/3596078353 wait complete. 2023-11-28T13:01:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:01:17.537 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-11-28T13:01:16.158364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69688,"kb_used_data":4472,"kb_used_omap":0,"kb_used_meta":65152,"kb_avail":754905032,"statfs":{"total":773094113280,"available":773022752768,"internally_reserved":0,"allocated":4579328,"data_stored":2562344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66715648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.478842"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090821+0000","last_change":"2023-11-28T13:01:13.094545+0000","last_active":"2023-11-28T13:01:14.090821+0000","last_peered":"2023-11-28T13:01:14.090821+0000","last_clean":"2023-11-28T13:01:14.090821+0000","last_became_active":"2023-11-28T13:01:13.094296+0000","last_became_peered":"2023-11-28T13:01:13.094296+0000","last_unstale":"2023-11-28T13:01:14.090821+0000","last_undegraded":"2023-11-28T13:01:14.090821+0000","last_fullsized":"2023-11-28T13:01:14.090821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:18.727519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090776+0000","last_change":"2023-11-28T13:01:13.088855+0000","last_active":"2023-11-28T13:01:14.090776+0000","last_peered":"2023-11-28T13:01:14.090776+0000","last_clean":"2023-11-28T13:01:14.090776+0000","last_became_active":"2023-11-28T13:01:13.088631+0000","last_became_peered":"2023-11-28T13:01:13.088631+0000","last_unstale":"2023-11-28T13:01:14.090776+0000","last_undegraded":"2023-11-28T13:01:14.090776+0000","last_fullsized":"2023-11-28T13:01:14.090776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:42.143519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107263+0000","last_change":"2023-11-28T13:01:13.102065+0000","last_active":"2023-11-28T13:01:13.107263+0000","last_peered":"2023-11-28T13:01:13.107263+0000","last_clean":"2023-11-28T13:01:13.107263+0000","last_became_active":"2023-11-28T13:01:13.101822+0000","last_became_peered":"2023-11-28T13:01:13.101822+0000","last_unstale":"2023-11-28T13:01:13.107263+0000","last_undegraded":"2023-11-28T13:01:13.107263+0000","last_fullsized":"2023-11-28T13:01:13.107263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:51.732790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.099900+0000","last_change":"2023-11-28T13:01:13.095907+0000","last_active":"2023-11-28T13:01:13.099900+0000","last_peered":"2023-11-28T13:01:13.099900+0000","last_clean":"2023-11-28T13:01:13.099900+0000","last_became_active":"2023-11-28T13:01:13.095190+0000","last_became_peered":"2023-11-28T13:01:13.095190+0000","last_unstale":"2023-11-28T13:01:13.099900+0000","last_undegraded":"2023-11-28T13:01:13.099900+0000","last_fullsized":"2023-11-28T13:01:13.099900+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:34.713740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088925+0000","last_change":"2023-11-28T13:01:13.103245+0000","last_active":"2023-11-28T13:01:14.088925+0000","last_peered":"2023-11-28T13:01:14.088925+0000","last_clean":"2023-11-28T13:01:14.088925+0000","last_became_active":"2023-11-28T13:01:13.102920+0000","last_became_peered":"2023-11-28T13:01:13.102920+0000","last_unstale":"2023-11-28T13:01:14.088925+0000","last_undegraded":"2023-11-28T13:01:14.088925+0000","last_fullsized":"2023-11-28T13:01:14.088925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:23.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106870+0000","last_change":"2023-11-28T13:01:13.101383+0000","last_active":"2023-11-28T13:01:13.106870+0000","last_peered":"2023-11-28T13:01:13.106870+0000","last_clean":"2023-11-28T13:01:13.106870+0000","last_became_active":"2023-11-28T13:01:13.101093+0000","last_became_peered":"2023-11-28T13:01:13.101093+0000","last_unstale":"2023-11-28T13:01:13.106870+0000","last_undegraded":"2023-11-28T13:01:13.106870+0000","last_fullsized":"2023-11-28T13:01:13.106870+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090405+0000","last_change":"2023-11-28T13:01:13.093176+0000","last_active":"2023-11-28T13:01:14.090405+0000","last_peered":"2023-11-28T13:01:14.090405+0000","last_clean":"2023-11-28T13:01:14.090405+0000","last_became_active":"2023-11-28T13:01:13.092922+0000","last_became_peered":"2023-11-28T13:01:13.092922+0000","last_unstale":"2023-11-28T13:01:14.090405+0000","last_undegraded":"2023-11-28T13:01:14.090405+0000","last_fullsized":"2023-11-28T13:01:14.090405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:31.712364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102138+0000","last_change":"2023-11-28T13:01:13.090026+0000","last_active":"2023-11-28T13:01:13.102138+0000","last_peered":"2023-11-28T13:01:13.102138+0000","last_clean":"2023-11-28T13:01:13.102138+0000","last_became_active":"2023-11-28T13:01:13.089734+0000","last_became_peered":"2023-11-28T13:01:13.089734+0000","last_unstale":"2023-11-28T13:01:13.102138+0000","last_undegraded":"2023-11-28T13:01:13.102138+0000","last_fullsized":"2023-11-28T13:01:13.102138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110483+0000","last_change":"2023-11-28T13:01:13.106317+0000","last_active":"2023-11-28T13:01:13.110483+0000","last_peered":"2023-11-28T13:01:13.110483+0000","last_clean":"2023-11-28T13:01:13.110483+0000","last_became_active":"2023-11-28T13:01:13.106096+0000","last_became_peered":"2023-11-28T13:01:13.106096+0000","last_unstale":"2023-11-28T13:01:13.110483+0000","last_undegraded":"2023-11-28T13:01:13.110483+0000","last_fullsized":"2023-11-28T13:01:13.110483+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102639+0000","last_change":"2023-11-28T13:01:13.098543+0000","last_active":"2023-11-28T13:01:13.102639+0000","last_peered":"2023-11-28T13:01:13.102639+0000","last_clean":"2023-11-28T13:01:13.102639+0000","last_became_active":"2023-11-28T13:01:13.097626+0000","last_became_peered":"2023-11-28T13:01:13.097626+0000","last_unstale":"2023-11-28T13:01:13.102639+0000","last_undegraded":"2023-11-28T13:01:13.102639+0000","last_fullsized":"2023-11-28T13:01:13.102639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104846+0000","last_change":"2023-11-28T13:01:13.097007+0000","last_active":"2023-11-28T13:01:13.104846+0000","last_peered":"2023-11-28T13:01:13.104846+0000","last_clean":"2023-11-28T13:01:13.104846+0000","last_became_active":"2023-11-28T13:01:13.096697+0000","last_became_peered":"2023-11-28T13:01:13.096697+0000","last_unstale":"2023-11-28T13:01:13.104846+0000","last_undegraded":"2023-11-28T13:01:13.104846+0000","last_fullsized":"2023-11-28T13:01:13.104846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102519+0000","last_change":"2023-11-28T13:01:13.092555+0000","last_active":"2023-11-28T13:01:13.102519+0000","last_peered":"2023-11-28T13:01:13.102519+0000","last_clean":"2023-11-28T13:01:13.102519+0000","last_became_active":"2023-11-28T13:01:13.092269+0000","last_became_peered":"2023-11-28T13:01:13.092269+0000","last_unstale":"2023-11-28T13:01:13.102519+0000","last_undegraded":"2023-11-28T13:01:13.102519+0000","last_fullsized":"2023-11-28T13:01:13.102519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110258+0000","last_change":"2023-11-28T13:01:13.106928+0000","last_active":"2023-11-28T13:01:13.110258+0000","last_peered":"2023-11-28T13:01:13.110258+0000","last_clean":"2023-11-28T13:01:13.110258+0000","last_became_active":"2023-11-28T13:01:13.104168+0000","last_became_peered":"2023-11-28T13:01:13.104168+0000","last_unstale":"2023-11-28T13:01:13.110258+0000","last_undegraded":"2023-11-28T13:01:13.110258+0000","last_fullsized":"2023-11-28T13:01:13.110258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110271+0000","last_change":"2023-11-28T13:01:13.106957+0000","last_active":"2023-11-28T13:01:13.110271+0000","last_peered":"2023-11-28T13:01:13.110271+0000","last_clean":"2023-11-28T13:01:13.110271+0000","last_became_active":"2023-11-28T13:01:13.104842+0000","last_became_peered":"2023-11-28T13:01:13.104842+0000","last_unstale":"2023-11-28T13:01:13.110271+0000","last_undegraded":"2023-11-28T13:01:13.110271+0000","last_fullsized":"2023-11-28T13:01:13.110271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105656+0000","last_change":"2023-11-28T13:01:13.102950+0000","last_active":"2023-11-28T13:01:13.105656+0000","last_peered":"2023-11-28T13:01:13.105656+0000","last_clean":"2023-11-28T13:01:13.105656+0000","last_became_active":"2023-11-28T13:01:13.102706+0000","last_became_peered":"2023-11-28T13:01:13.102706+0000","last_unstale":"2023-11-28T13:01:13.105656+0000","last_undegraded":"2023-11-28T13:01:13.105656+0000","last_fullsized":"2023-11-28T13:01:13.105656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:33.362111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.097705+0000","last_change":"2023-11-28T13:01:13.094603+0000","last_active":"2023-11-28T13:01:13.097705+0000","last_peered":"2023-11-28T13:01:13.097705+0000","last_clean":"2023-11-28T13:01:13.097705+0000","last_became_active":"2023-11-28T13:01:13.094377+0000","last_became_peered":"2023-11-28T13:01:13.094377+0000","last_unstale":"2023-11-28T13:01:13.097705+0000","last_undegraded":"2023-11-28T13:01:13.097705+0000","last_fullsized":"2023-11-28T13:01:13.097705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:25.095910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098253+0000","last_change":"2023-11-28T13:01:13.093425+0000","last_active":"2023-11-28T13:01:13.098253+0000","last_peered":"2023-11-28T13:01:13.098253+0000","last_clean":"2023-11-28T13:01:13.098253+0000","last_became_active":"2023-11-28T13:01:13.093128+0000","last_became_peered":"2023-11-28T13:01:13.093128+0000","last_unstale":"2023-11-28T13:01:13.098253+0000","last_undegraded":"2023-11-28T13:01:13.098253+0000","last_fullsized":"2023-11-28T13:01:13.098253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:47.380562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102465+0000","last_change":"2023-11-28T13:01:13.095333+0000","last_active":"2023-11-28T13:01:13.102465+0000","last_peered":"2023-11-28T13:01:13.102465+0000","last_clean":"2023-11-28T13:01:13.102465+0000","last_became_active":"2023-11-28T13:01:13.095050+0000","last_became_peered":"2023-11-28T13:01:13.095050+0000","last_unstale":"2023-11-28T13:01:13.102465+0000","last_undegraded":"2023-11-28T13:01:13.102465+0000","last_fullsized":"2023-11-28T13:01:13.102465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:51:32.434761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107125+0000","last_change":"2023-11-28T13:01:13.105213+0000","last_active":"2023-11-28T13:01:13.107125+0000","last_peered":"2023-11-28T13:01:13.107125+0000","last_clean":"2023-11-28T13:01:13.107125+0000","last_became_active":"2023-11-28T13:01:13.104960+0000","last_became_peered":"2023-11-28T13:01:13.104960+0000","last_unstale":"2023-11-28T13:01:13.107125+0000","last_undegraded":"2023-11-28T13:01:13.107125+0000","last_fullsized":"2023-11-28T13:01:13.107125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110113+0000","last_change":"2023-11-28T13:01:13.102193+0000","last_active":"2023-11-28T13:01:13.110113+0000","last_peered":"2023-11-28T13:01:13.110113+0000","last_clean":"2023-11-28T13:01:13.110113+0000","last_became_active":"2023-11-28T13:01:13.101759+0000","last_became_peered":"2023-11-28T13:01:13.101759+0000","last_unstale":"2023-11-28T13:01:13.110113+0000","last_undegraded":"2023-11-28T13:01:13.110113+0000","last_fullsized":"2023-11-28T13:01:13.110113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:51.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103015+0000","last_change":"2023-11-28T13:01:13.098378+0000","last_active":"2023-11-28T13:01:13.103015+0000","last_peered":"2023-11-28T13:01:13.103015+0000","last_clean":"2023-11-28T13:01:13.103015+0000","last_became_active":"2023-11-28T13:01:13.098101+0000","last_became_peered":"2023-11-28T13:01:13.098101+0000","last_unstale":"2023-11-28T13:01:13.103015+0000","last_undegraded":"2023-11-28T13:01:13.103015+0000","last_fullsized":"2023-11-28T13:01:13.103015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:11.877653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098316+0000","last_change":"2023-11-28T13:01:13.094017+0000","last_active":"2023-11-28T13:01:13.098316+0000","last_peered":"2023-11-28T13:01:13.098316+0000","last_clean":"2023-11-28T13:01:13.098316+0000","last_became_active":"2023-11-28T13:01:13.093753+0000","last_became_peered":"2023-11-28T13:01:13.093753+0000","last_unstale":"2023-11-28T13:01:13.098316+0000","last_undegraded":"2023-11-28T13:01:13.098316+0000","last_fullsized":"2023-11-28T13:01:13.098316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:21.524121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089155+0000","last_change":"2023-11-28T13:01:13.103430+0000","last_active":"2023-11-28T13:01:14.089155+0000","last_peered":"2023-11-28T13:01:14.089155+0000","last_clean":"2023-11-28T13:01:14.089155+0000","last_became_active":"2023-11-28T13:01:13.102784+0000","last_became_peered":"2023-11-28T13:01:13.102784+0000","last_unstale":"2023-11-28T13:01:14.089155+0000","last_undegraded":"2023-11-28T13:01:14.089155+0000","last_fullsized":"2023-11-28T13:01:14.089155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:47.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098350+0000","last_change":"2023-11-28T13:01:13.092484+0000","last_active":"2023-11-28T13:01:13.098350+0000","last_peered":"2023-11-28T13:01:13.098350+0000","last_clean":"2023-11-28T13:01:13.098350+0000","last_became_active":"2023-11-28T13:01:13.092162+0000","last_became_peered":"2023-11-28T13:01:13.092162+0000","last_unstale":"2023-11-28T13:01:13.098350+0000","last_undegraded":"2023-11-28T13:01:13.098350+0000","last_fullsized":"2023-11-28T13:01:13.098350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:03.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107152+0000","last_change":"2023-11-28T13:01:13.100818+0000","last_active":"2023-11-28T13:01:13.107152+0000","last_peered":"2023-11-28T13:01:13.107152+0000","last_clean":"2023-11-28T13:01:13.107152+0000","last_became_active":"2023-11-28T13:01:13.100421+0000","last_became_peered":"2023-11-28T13:01:13.100421+0000","last_unstale":"2023-11-28T13:01:13.107152+0000","last_undegraded":"2023-11-28T13:01:13.107152+0000","last_fullsized":"2023-11-28T13:01:13.107152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098210+0000","last_change":"2023-11-28T13:01:13.093478+0000","last_active":"2023-11-28T13:01:13.098210+0000","last_peered":"2023-11-28T13:01:13.098210+0000","last_clean":"2023-11-28T13:01:13.098210+0000","last_became_active":"2023-11-28T13:01:13.093194+0000","last_became_peered":"2023-11-28T13:01:13.093194+0000","last_unstale":"2023-11-28T13:01:13.098210+0000","last_undegraded":"2023-11-28T13:01:13.098210+0000","last_fullsized":"2023-11-28T13:01:13.098210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:10.435728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098142+0000","last_change":"2023-11-28T13:01:13.088250+0000","last_active":"2023-11-28T13:01:13.098142+0000","last_peered":"2023-11-28T13:01:13.098142+0000","last_clean":"2023-11-28T13:01:13.098142+0000","last_became_active":"2023-11-28T13:01:13.087881+0000","last_became_peered":"2023-11-28T13:01:13.087881+0000","last_unstale":"2023-11-28T13:01:13.098142+0000","last_undegraded":"2023-11-28T13:01:13.098142+0000","last_fullsized":"2023-11-28T13:01:13.098142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:43.126488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110058+0000","last_change":"2023-11-28T13:01:13.102030+0000","last_active":"2023-11-28T13:01:13.110058+0000","last_peered":"2023-11-28T13:01:13.110058+0000","last_clean":"2023-11-28T13:01:13.110058+0000","last_became_active":"2023-11-28T13:01:13.101376+0000","last_became_peered":"2023-11-28T13:01:13.101376+0000","last_unstale":"2023-11-28T13:01:13.110058+0000","last_undegraded":"2023-11-28T13:01:13.110058+0000","last_fullsized":"2023-11-28T13:01:13.110058+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088757+0000","last_change":"2023-11-28T13:01:13.095736+0000","last_active":"2023-11-28T13:01:14.088757+0000","last_peered":"2023-11-28T13:01:14.088757+0000","last_clean":"2023-11-28T13:01:14.088757+0000","last_became_active":"2023-11-28T13:01:13.095469+0000","last_became_peered":"2023-11-28T13:01:13.095469+0000","last_unstale":"2023-11-28T13:01:14.088757+0000","last_undegraded":"2023-11-28T13:01:14.088757+0000","last_fullsized":"2023-11-28T13:01:14.088757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:45.817559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110226+0000","last_change":"2023-11-28T13:01:13.105146+0000","last_active":"2023-11-28T13:01:13.110226+0000","last_peered":"2023-11-28T13:01:13.110226+0000","last_clean":"2023-11-28T13:01:13.110226+0000","last_became_active":"2023-11-28T13:01:13.104852+0000","last_became_peered":"2023-11-28T13:01:13.104852+0000","last_unstale":"2023-11-28T13:01:13.110226+0000","last_undegraded":"2023-11-28T13:01:13.110226+0000","last_fullsized":"2023-11-28T13:01:13.110226+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104391+0000","last_change":"2023-11-28T13:01:12.697633+0000","last_active":"2023-11-28T13:01:13.104391+0000","last_peered":"2023-11-28T13:01:13.104391+0000","last_clean":"2023-11-28T13:01:13.104391+0000","last_became_active":"2023-11-28T13:01:12.697137+0000","last_became_peered":"2023-11-28T13:01:12.697137+0000","last_unstale":"2023-11-28T13:01:13.104391+0000","last_undegraded":"2023-11-28T13:01:13.104391+0000","last_fullsized":"2023-11-28T13:01:13.104391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:59.601679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089117+0000","last_change":"2023-11-28T13:01:13.104026+0000","last_active":"2023-11-28T13:01:14.089117+0000","last_peered":"2023-11-28T13:01:14.089117+0000","last_clean":"2023-11-28T13:01:14.089117+0000","last_became_active":"2023-11-28T13:01:13.103779+0000","last_became_peered":"2023-11-28T13:01:13.103779+0000","last_unstale":"2023-11-28T13:01:14.089117+0000","last_undegraded":"2023-11-28T13:01:14.089117+0000","last_fullsized":"2023-11-28T13:01:14.089117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:13.669167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.685907+0000","last_change":"2023-11-28T13:01:11.685907+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.685907+0000","last_undegraded":"2023-11-28T13:01:11.685907+0000","last_fullsized":"2023-11-28T13:01:11.685907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107203+0000","last_change":"2023-11-28T13:01:13.104225+0000","last_active":"2023-11-28T13:01:13.107203+0000","last_peered":"2023-11-28T13:01:13.107203+0000","last_clean":"2023-11-28T13:01:13.107203+0000","last_became_active":"2023-11-28T13:01:13.103835+0000","last_became_peered":"2023-11-28T13:01:13.103835+0000","last_unstale":"2023-11-28T13:01:13.107203+0000","last_undegraded":"2023-11-28T13:01:13.107203+0000","last_fullsized":"2023-11-28T13:01:13.107203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089210+0000","last_change":"2023-11-28T13:01:12.703691+0000","last_active":"2023-11-28T13:01:14.089210+0000","last_peered":"2023-11-28T13:01:14.089210+0000","last_clean":"2023-11-28T13:01:14.089210+0000","last_became_active":"2023-11-28T13:01:12.703398+0000","last_became_peered":"2023-11-28T13:01:12.703398+0000","last_unstale":"2023-11-28T13:01:14.089210+0000","last_undegraded":"2023-11-28T13:01:14.089210+0000","last_fullsized":"2023-11-28T13:01:14.089210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:24.093954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090818+0000","last_change":"2023-11-28T13:01:13.086251+0000","last_active":"2023-11-28T13:01:14.090818+0000","last_peered":"2023-11-28T13:01:14.090818+0000","last_clean":"2023-11-28T13:01:14.090818+0000","last_became_active":"2023-11-28T13:01:13.085269+0000","last_became_peered":"2023-11-28T13:01:13.085269+0000","last_unstale":"2023-11-28T13:01:14.090818+0000","last_undegraded":"2023-11-28T13:01:14.090818+0000","last_fullsized":"2023-11-28T13:01:14.090818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:04.381796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103953+0000","last_change":"2023-11-28T13:01:12.700070+0000","last_active":"2023-11-28T13:01:13.103953+0000","last_peered":"2023-11-28T13:01:13.103953+0000","last_clean":"2023-11-28T13:01:13.103953+0000","last_became_active":"2023-11-28T13:01:12.699788+0000","last_became_peered":"2023-11-28T13:01:12.699788+0000","last_unstale":"2023-11-28T13:01:13.103953+0000","last_undegraded":"2023-11-28T13:01:13.103953+0000","last_fullsized":"2023-11-28T13:01:13.103953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:37.142758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102494+0000","last_change":"2023-11-28T13:01:12.700665+0000","last_active":"2023-11-28T13:01:13.102494+0000","last_peered":"2023-11-28T13:01:13.102494+0000","last_clean":"2023-11-28T13:01:13.102494+0000","last_became_active":"2023-11-28T13:01:12.700285+0000","last_became_peered":"2023-11-28T13:01:12.700285+0000","last_unstale":"2023-11-28T13:01:13.102494+0000","last_undegraded":"2023-11-28T13:01:13.102494+0000","last_fullsized":"2023-11-28T13:01:13.102494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102539+0000","last_change":"2023-11-28T13:01:13.095159+0000","last_active":"2023-11-28T13:01:13.102539+0000","last_peered":"2023-11-28T13:01:13.102539+0000","last_clean":"2023-11-28T13:01:13.102539+0000","last_became_active":"2023-11-28T13:01:13.094857+0000","last_became_peered":"2023-11-28T13:01:13.094857+0000","last_unstale":"2023-11-28T13:01:13.102539+0000","last_undegraded":"2023-11-28T13:01:13.102539+0000","last_fullsized":"2023-11-28T13:01:13.102539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106011+0000","last_change":"2023-11-28T13:01:12.705296+0000","last_active":"2023-11-28T13:01:13.106011+0000","last_peered":"2023-11-28T13:01:13.106011+0000","last_clean":"2023-11-28T13:01:13.106011+0000","last_became_active":"2023-11-28T13:01:12.705013+0000","last_became_peered":"2023-11-28T13:01:12.705013+0000","last_unstale":"2023-11-28T13:01:13.106011+0000","last_undegraded":"2023-11-28T13:01:13.106011+0000","last_fullsized":"2023-11-28T13:01:13.106011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:46.378606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104868+0000","last_change":"2023-11-28T13:01:13.094747+0000","last_active":"2023-11-28T13:01:13.104868+0000","last_peered":"2023-11-28T13:01:13.104868+0000","last_clean":"2023-11-28T13:01:13.104868+0000","last_became_active":"2023-11-28T13:01:13.094318+0000","last_became_peered":"2023-11-28T13:01:13.094318+0000","last_unstale":"2023-11-28T13:01:13.104868+0000","last_undegraded":"2023-11-28T13:01:13.104868+0000","last_fullsized":"2023-11-28T13:01:13.104868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:27.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089212+0000","last_change":"2023-11-28T13:01:12.706058+0000","last_active":"2023-11-28T13:01:14.089212+0000","last_peered":"2023-11-28T13:01:14.089212+0000","last_clean":"2023-11-28T13:01:14.089212+0000","last_became_active":"2023-11-28T13:01:12.705853+0000","last_became_peered":"2023-11-28T13:01:12.705853+0000","last_unstale":"2023-11-28T13:01:14.089212+0000","last_undegraded":"2023-11-28T13:01:14.089212+0000","last_fullsized":"2023-11-28T13:01:14.089212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:10.875697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090633+0000","last_change":"2023-11-28T13:01:13.099221+0000","last_active":"2023-11-28T13:01:14.090633+0000","last_peered":"2023-11-28T13:01:14.090633+0000","last_clean":"2023-11-28T13:01:14.090633+0000","last_became_active":"2023-11-28T13:01:13.098525+0000","last_became_peered":"2023-11-28T13:01:13.098525+0000","last_unstale":"2023-11-28T13:01:14.090633+0000","last_undegraded":"2023-11-28T13:01:14.090633+0000","last_fullsized":"2023-11-28T13:01:14.090633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:01.070045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092473+0000","last_change":"2023-11-28T13:01:12.700642+0000","last_active":"2023-11-28T13:01:13.092473+0000","last_peered":"2023-11-28T13:01:13.092473+0000","last_clean":"2023-11-28T13:01:13.092473+0000","last_became_active":"2023-11-28T13:01:12.700381+0000","last_became_peered":"2023-11-28T13:01:12.700381+0000","last_unstale":"2023-11-28T13:01:13.092473+0000","last_undegraded":"2023-11-28T13:01:13.092473+0000","last_fullsized":"2023-11-28T13:01:13.092473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:37.268919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110533+0000","last_change":"2023-11-28T13:01:13.107217+0000","last_active":"2023-11-28T13:01:13.110533+0000","last_peered":"2023-11-28T13:01:13.110533+0000","last_clean":"2023-11-28T13:01:13.110533+0000","last_became_active":"2023-11-28T13:01:13.106967+0000","last_became_peered":"2023-11-28T13:01:13.106967+0000","last_unstale":"2023-11-28T13:01:13.110533+0000","last_undegraded":"2023-11-28T13:01:13.110533+0000","last_fullsized":"2023-11-28T13:01:13.110533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:01.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090884+0000","last_change":"2023-11-28T13:01:12.700536+0000","last_active":"2023-11-28T13:01:14.090884+0000","last_peered":"2023-11-28T13:01:14.090884+0000","last_clean":"2023-11-28T13:01:14.090884+0000","last_became_active":"2023-11-28T13:01:12.699782+0000","last_became_peered":"2023-11-28T13:01:12.699782+0000","last_unstale":"2023-11-28T13:01:14.090884+0000","last_undegraded":"2023-11-28T13:01:14.090884+0000","last_fullsized":"2023-11-28T13:01:14.090884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:17.146583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110165+0000","last_change":"2023-11-28T13:01:13.103462+0000","last_active":"2023-11-28T13:01:13.110165+0000","last_peered":"2023-11-28T13:01:13.110165+0000","last_clean":"2023-11-28T13:01:13.110165+0000","last_became_active":"2023-11-28T13:01:13.102855+0000","last_became_peered":"2023-11-28T13:01:13.102855+0000","last_unstale":"2023-11-28T13:01:13.110165+0000","last_undegraded":"2023-11-28T13:01:13.110165+0000","last_fullsized":"2023-11-28T13:01:13.110165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.987104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092356+0000","last_change":"2023-11-28T13:01:12.703891+0000","last_active":"2023-11-28T13:01:13.092356+0000","last_peered":"2023-11-28T13:01:13.092356+0000","last_clean":"2023-11-28T13:01:13.092356+0000","last_became_active":"2023-11-28T13:01:12.703642+0000","last_became_peered":"2023-11-28T13:01:12.703642+0000","last_unstale":"2023-11-28T13:01:13.092356+0000","last_undegraded":"2023-11-28T13:01:13.092356+0000","last_fullsized":"2023-11-28T13:01:13.092356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:32.360155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105022+0000","last_change":"2023-11-28T13:01:12.704636+0000","last_active":"2023-11-28T13:01:13.105022+0000","last_peered":"2023-11-28T13:01:13.105022+0000","last_clean":"2023-11-28T13:01:13.105022+0000","last_became_active":"2023-11-28T13:01:12.704357+0000","last_became_peered":"2023-11-28T13:01:12.704357+0000","last_unstale":"2023-11-28T13:01:13.105022+0000","last_undegraded":"2023-11-28T13:01:13.105022+0000","last_fullsized":"2023-11-28T13:01:13.105022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105277+0000","last_change":"2023-11-28T13:01:13.096971+0000","last_active":"2023-11-28T13:01:13.105277+0000","last_peered":"2023-11-28T13:01:13.105277+0000","last_clean":"2023-11-28T13:01:13.105277+0000","last_became_active":"2023-11-28T13:01:13.096698+0000","last_became_peered":"2023-11-28T13:01:13.096698+0000","last_unstale":"2023-11-28T13:01:13.105277+0000","last_undegraded":"2023-11-28T13:01:13.105277+0000","last_fullsized":"2023-11-28T13:01:13.105277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090728+0000","last_change":"2023-11-28T13:01:12.704846+0000","last_active":"2023-11-28T13:01:14.090728+0000","last_peered":"2023-11-28T13:01:14.090728+0000","last_clean":"2023-11-28T13:01:14.090728+0000","last_became_active":"2023-11-28T13:01:12.704605+0000","last_became_peered":"2023-11-28T13:01:12.704605+0000","last_unstale":"2023-11-28T13:01:14.090728+0000","last_undegraded":"2023-11-28T13:01:14.090728+0000","last_fullsized":"2023-11-28T13:01:14.090728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:43.923726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090672+0000","last_change":"2023-11-28T13:01:12.700930+0000","last_active":"2023-11-28T13:01:14.090672+0000","last_peered":"2023-11-28T13:01:14.090672+0000","last_clean":"2023-11-28T13:01:14.090672+0000","last_became_active":"2023-11-28T13:01:12.700670+0000","last_became_peered":"2023-11-28T13:01:12.700670+0000","last_unstale":"2023-11-28T13:01:14.090672+0000","last_undegraded":"2023-11-28T13:01:14.090672+0000","last_fullsized":"2023-11-28T13:01:14.090672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090509+0000","last_change":"2023-11-28T13:01:13.093993+0000","last_active":"2023-11-28T13:01:14.090509+0000","last_peered":"2023-11-28T13:01:14.090509+0000","last_clean":"2023-11-28T13:01:14.090509+0000","last_became_active":"2023-11-28T13:01:13.093750+0000","last_became_peered":"2023-11-28T13:01:13.093750+0000","last_unstale":"2023-11-28T13:01:14.090509+0000","last_undegraded":"2023-11-28T13:01:14.090509+0000","last_fullsized":"2023-11-28T13:01:14.090509+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:47.159742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088952+0000","last_change":"2023-11-28T13:01:12.706087+0000","last_active":"2023-11-28T13:01:14.088952+0000","last_peered":"2023-11-28T13:01:14.088952+0000","last_clean":"2023-11-28T13:01:14.088952+0000","last_became_active":"2023-11-28T13:01:12.705800+0000","last_became_peered":"2023-11-28T13:01:12.705800+0000","last_unstale":"2023-11-28T13:01:14.088952+0000","last_undegraded":"2023-11-28T13:01:14.088952+0000","last_fullsized":"2023-11-28T13:01:14.088952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102554+0000","last_change":"2023-11-28T13:01:13.101116+0000","last_active":"2023-11-28T13:01:13.102554+0000","last_peered":"2023-11-28T13:01:13.102554+0000","last_clean":"2023-11-28T13:01:13.102554+0000","last_became_active":"2023-11-28T13:01:13.100510+0000","last_became_peered":"2023-11-28T13:01:13.100510+0000","last_unstale":"2023-11-28T13:01:13.102554+0000","last_undegraded":"2023-11-28T13:01:13.102554+0000","last_fullsized":"2023-11-28T13:01:13.102554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:42.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081264+0000","last_change":"2023-11-28T13:01:12.702749+0000","last_active":"2023-11-28T13:01:13.081264+0000","last_peered":"2023-11-28T13:01:13.081264+0000","last_clean":"2023-11-28T13:01:13.081264+0000","last_became_active":"2023-11-28T13:01:12.702450+0000","last_became_peered":"2023-11-28T13:01:12.702450+0000","last_unstale":"2023-11-28T13:01:13.081264+0000","last_undegraded":"2023-11-28T13:01:13.081264+0000","last_fullsized":"2023-11-28T13:01:13.081264+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:38.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090531+0000","last_change":"2023-11-28T13:01:13.095487+0000","last_active":"2023-11-28T13:01:14.090531+0000","last_peered":"2023-11-28T13:01:14.090531+0000","last_clean":"2023-11-28T13:01:14.090531+0000","last_became_active":"2023-11-28T13:01:13.095222+0000","last_became_peered":"2023-11-28T13:01:13.095222+0000","last_unstale":"2023-11-28T13:01:14.090531+0000","last_undegraded":"2023-11-28T13:01:14.090531+0000","last_fullsized":"2023-11-28T13:01:14.090531+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:57.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106265+0000","last_change":"2023-11-28T13:01:12.704714+0000","last_active":"2023-11-28T13:01:13.106265+0000","last_peered":"2023-11-28T13:01:13.106265+0000","last_clean":"2023-11-28T13:01:13.106265+0000","last_became_active":"2023-11-28T13:01:12.704422+0000","last_became_peered":"2023-11-28T13:01:12.704422+0000","last_unstale":"2023-11-28T13:01:13.106265+0000","last_undegraded":"2023-11-28T13:01:13.106265+0000","last_fullsized":"2023-11-28T13:01:13.106265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:42.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090726+0000","last_change":"2023-11-28T13:01:13.088050+0000","last_active":"2023-11-28T13:01:14.090726+0000","last_peered":"2023-11-28T13:01:14.090726+0000","last_clean":"2023-11-28T13:01:14.090726+0000","last_became_active":"2023-11-28T13:01:13.085247+0000","last_became_peered":"2023-11-28T13:01:13.085247+0000","last_unstale":"2023-11-28T13:01:14.090726+0000","last_undegraded":"2023-11-28T13:01:14.090726+0000","last_fullsized":"2023-11-28T13:01:14.090726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:39.035787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107176+0000","last_change":"2023-11-28T13:01:12.702173+0000","last_active":"2023-11-28T13:01:13.107176+0000","last_peered":"2023-11-28T13:01:13.107176+0000","last_clean":"2023-11-28T13:01:13.107176+0000","last_became_active":"2023-11-28T13:01:12.701243+0000","last_became_peered":"2023-11-28T13:01:12.701243+0000","last_unstale":"2023-11-28T13:01:13.107176+0000","last_undegraded":"2023-11-28T13:01:13.107176+0000","last_fullsized":"2023-11-28T13:01:13.107176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102621+0000","last_change":"2023-11-28T13:01:13.099301+0000","last_active":"2023-11-28T13:01:13.102621+0000","last_peered":"2023-11-28T13:01:13.102621+0000","last_clean":"2023-11-28T13:01:13.102621+0000","last_became_active":"2023-11-28T13:01:13.099017+0000","last_became_peered":"2023-11-28T13:01:13.099017+0000","last_unstale":"2023-11-28T13:01:13.102621+0000","last_undegraded":"2023-11-28T13:01:13.102621+0000","last_fullsized":"2023-11-28T13:01:13.102621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104381+0000","last_change":"2023-11-28T13:01:12.706078+0000","last_active":"2023-11-28T13:01:13.104381+0000","last_peered":"2023-11-28T13:01:13.104381+0000","last_clean":"2023-11-28T13:01:13.104381+0000","last_became_active":"2023-11-28T13:01:12.705694+0000","last_became_peered":"2023-11-28T13:01:12.705694+0000","last_unstale":"2023-11-28T13:01:13.104381+0000","last_undegraded":"2023-11-28T13:01:13.104381+0000","last_fullsized":"2023-11-28T13:01:13.104381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:41.987277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105893+0000","last_change":"2023-11-28T13:01:13.098692+0000","last_active":"2023-11-28T13:01:13.105893+0000","last_peered":"2023-11-28T13:01:13.105893+0000","last_clean":"2023-11-28T13:01:13.105893+0000","last_became_active":"2023-11-28T13:01:13.098426+0000","last_became_peered":"2023-11-28T13:01:13.098426+0000","last_unstale":"2023-11-28T13:01:13.105893+0000","last_undegraded":"2023-11-28T13:01:13.105893+0000","last_fullsized":"2023-11-28T13:01:13.105893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.101277+0000","last_change":"2023-11-28T13:01:12.701155+0000","last_active":"2023-11-28T13:01:13.101277+0000","last_peered":"2023-11-28T13:01:13.101277+0000","last_clean":"2023-11-28T13:01:13.101277+0000","last_became_active":"2023-11-28T13:01:12.700897+0000","last_became_peered":"2023-11-28T13:01:12.700897+0000","last_unstale":"2023-11-28T13:01:13.101277+0000","last_undegraded":"2023-11-28T13:01:13.101277+0000","last_fullsized":"2023-11-28T13:01:13.101277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:57.063655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.100622+0000","last_change":"2023-11-28T13:01:13.094885+0000","last_active":"2023-11-28T13:01:13.100622+0000","last_peered":"2023-11-28T13:01:13.100622+0000","last_clean":"2023-11-28T13:01:13.100622+0000","last_became_active":"2023-11-28T13:01:13.094027+0000","last_became_peered":"2023-11-28T13:01:13.094027+0000","last_unstale":"2023-11-28T13:01:13.100622+0000","last_undegraded":"2023-11-28T13:01:13.100622+0000","last_fullsized":"2023-11-28T13:01:13.100622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:43.321762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089094+0000","last_change":"2023-11-28T13:01:12.703499+0000","last_active":"2023-11-28T13:01:14.089094+0000","last_peered":"2023-11-28T13:01:14.089094+0000","last_clean":"2023-11-28T13:01:14.089094+0000","last_became_active":"2023-11-28T13:01:12.703271+0000","last_became_peered":"2023-11-28T13:01:12.703271+0000","last_unstale":"2023-11-28T13:01:14.089094+0000","last_undegraded":"2023-11-28T13:01:14.089094+0000","last_fullsized":"2023-11-28T13:01:14.089094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:03:30.119943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107824+0000","last_change":"2023-11-28T13:01:13.101678+0000","last_active":"2023-11-28T13:01:13.107824+0000","last_peered":"2023-11-28T13:01:13.107824+0000","last_clean":"2023-11-28T13:01:13.107824+0000","last_became_active":"2023-11-28T13:01:13.101386+0000","last_became_peered":"2023-11-28T13:01:13.101386+0000","last_unstale":"2023-11-28T13:01:13.107824+0000","last_undegraded":"2023-11-28T13:01:13.107824+0000","last_fullsized":"2023-11-28T13:01:13.107824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:51.226552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.095695+0000","last_change":"2023-11-28T13:01:12.697001+0000","last_active":"2023-11-28T13:01:13.095695+0000","last_peered":"2023-11-28T13:01:13.095695+0000","last_clean":"2023-11-28T13:01:13.095695+0000","last_became_active":"2023-11-28T13:01:12.696739+0000","last_became_peered":"2023-11-28T13:01:12.696739+0000","last_unstale":"2023-11-28T13:01:13.095695+0000","last_undegraded":"2023-11-28T13:01:13.095695+0000","last_fullsized":"2023-11-28T13:01:13.095695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:37.438564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083496+0000","last_change":"2023-11-28T13:01:12.698523+0000","last_active":"2023-11-28T13:01:13.083496+0000","last_peered":"2023-11-28T13:01:13.083496+0000","last_clean":"2023-11-28T13:01:13.083496+0000","last_became_active":"2023-11-28T13:01:12.698275+0000","last_became_peered":"2023-11-28T13:01:12.698275+0000","last_unstale":"2023-11-28T13:01:13.083496+0000","last_undegraded":"2023-11-28T13:01:13.083496+0000","last_fullsized":"2023-11-28T13:01:13.083496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:02.254485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102691+0000","last_change":"2023-11-28T13:01:13.099007+0000","last_active":"2023-11-28T13:01:13.102691+0000","last_peered":"2023-11-28T13:01:13.102691+0000","last_clean":"2023-11-28T13:01:13.102691+0000","last_became_active":"2023-11-28T13:01:13.098673+0000","last_became_peered":"2023-11-28T13:01:13.098673+0000","last_unstale":"2023-11-28T13:01:13.102691+0000","last_undegraded":"2023-11-28T13:01:13.102691+0000","last_fullsized":"2023-11-28T13:01:13.102691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091666+0000","last_change":"2023-11-28T13:01:12.706237+0000","last_active":"2023-11-28T13:01:13.091666+0000","last_peered":"2023-11-28T13:01:13.091666+0000","last_clean":"2023-11-28T13:01:13.091666+0000","last_became_active":"2023-11-28T13:01:12.705981+0000","last_became_peered":"2023-11-28T13:01:12.705981+0000","last_unstale":"2023-11-28T13:01:13.091666+0000","last_undegraded":"2023-11-28T13:01:13.091666+0000","last_fullsized":"2023-11-28T13:01:13.091666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090578+0000","last_change":"2023-11-28T13:01:13.099382+0000","last_active":"2023-11-28T13:01:14.090578+0000","last_peered":"2023-11-28T13:01:14.090578+0000","last_clean":"2023-11-28T13:01:14.090578+0000","last_became_active":"2023-11-28T13:01:13.099008+0000","last_became_peered":"2023-11-28T13:01:13.099008+0000","last_unstale":"2023-11-28T13:01:14.090578+0000","last_undegraded":"2023-11-28T13:01:14.090578+0000","last_fullsized":"2023-11-28T13:01:14.090578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:35.600930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083382+0000","last_change":"2023-11-28T13:01:12.696728+0000","last_active":"2023-11-28T13:01:13.083382+0000","last_peered":"2023-11-28T13:01:13.083382+0000","last_clean":"2023-11-28T13:01:13.083382+0000","last_became_active":"2023-11-28T13:01:12.696428+0000","last_became_peered":"2023-11-28T13:01:12.696428+0000","last_unstale":"2023-11-28T13:01:13.083382+0000","last_undegraded":"2023-11-28T13:01:13.083382+0000","last_fullsized":"2023-11-28T13:01:13.083382+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:28.781799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.686000+0000","last_change":"2023-11-28T13:01:11.686000+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.686000+0000","last_undegraded":"2023-11-28T13:01:11.686000+0000","last_fullsized":"2023-11-28T13:01:11.686000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091534+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:01:13.091534+0000","last_peered":"2023-11-28T13:01:13.091534+0000","last_clean":"2023-11-28T13:01:13.091534+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:01:13.091534+0000","last_undegraded":"2023-11-28T13:01:13.091534+0000","last_fullsized":"2023-11-28T13:01:13.091534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090805+0000","last_change":"2023-11-28T13:01:13.098873+0000","last_active":"2023-11-28T13:01:14.090805+0000","last_peered":"2023-11-28T13:01:14.090805+0000","last_clean":"2023-11-28T13:01:14.090805+0000","last_became_active":"2023-11-28T13:01:13.098618+0000","last_became_peered":"2023-11-28T13:01:13.098618+0000","last_unstale":"2023-11-28T13:01:14.090805+0000","last_undegraded":"2023-11-28T13:01:14.090805+0000","last_fullsized":"2023-11-28T13:01:14.090805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:49.374140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.686366+0000","last_change":"2023-11-28T13:01:11.686366+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.686366+0000","last_undegraded":"2023-11-28T13:01:11.686366+0000","last_fullsized":"2023-11-28T13:01:11.686366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699954+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:01:12.699954+0000","last_peered":"2023-11-28T13:01:12.699954+0000","last_clean":"2023-11-28T13:01:12.699954+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:01:12.699954+0000","last_undegraded":"2023-11-28T13:01:12.699954+0000","last_fullsized":"2023-11-28T13:01:12.699954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110193+0000","last_change":"2023-11-28T13:01:13.103633+0000","last_active":"2023-11-28T13:01:13.110193+0000","last_peered":"2023-11-28T13:01:13.110193+0000","last_clean":"2023-11-28T13:01:13.110193+0000","last_became_active":"2023-11-28T13:01:13.103198+0000","last_became_peered":"2023-11-28T13:01:13.103198+0000","last_unstale":"2023-11-28T13:01:13.110193+0000","last_undegraded":"2023-11-28T13:01:13.110193+0000","last_fullsized":"2023-11-28T13:01:13.110193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.084615+0000","last_change":"2023-11-28T13:01:12.704388+0000","last_active":"2023-11-28T13:01:13.084615+0000","last_peered":"2023-11-28T13:01:13.084615+0000","last_clean":"2023-11-28T13:01:13.084615+0000","last_became_active":"2023-11-28T13:01:12.704082+0000","last_became_peered":"2023-11-28T13:01:12.704082+0000","last_unstale":"2023-11-28T13:01:13.084615+0000","last_undegraded":"2023-11-28T13:01:13.084615+0000","last_fullsized":"2023-11-28T13:01:13.084615+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:08.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.685641+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:01:12.685641+0000","last_peered":"2023-11-28T13:01:12.685641+0000","last_clean":"2023-11-28T13:01:12.685641+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:01:12.685641+0000","last_undegraded":"2023-11-28T13:01:12.685641+0000","last_fullsized":"2023-11-28T13:01:12.685641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110430+0000","last_change":"2023-11-28T13:01:13.105495+0000","last_active":"2023-11-28T13:01:13.110430+0000","last_peered":"2023-11-28T13:01:13.110430+0000","last_clean":"2023-11-28T13:01:13.110430+0000","last_became_active":"2023-11-28T13:01:13.103141+0000","last_became_peered":"2023-11-28T13:01:13.103141+0000","last_unstale":"2023-11-28T13:01:13.110430+0000","last_undegraded":"2023-11-28T13:01:13.110430+0000","last_fullsized":"2023-11-28T13:01:13.110430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:08.241448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091757+0000","last_change":"2023-11-28T13:01:12.704702+0000","last_active":"2023-11-28T13:01:13.091757+0000","last_peered":"2023-11-28T13:01:13.091757+0000","last_clean":"2023-11-28T13:01:13.091757+0000","last_became_active":"2023-11-28T13:01:12.704416+0000","last_became_peered":"2023-11-28T13:01:12.704416+0000","last_unstale":"2023-11-28T13:01:13.091757+0000","last_undegraded":"2023-11-28T13:01:13.091757+0000","last_fullsized":"2023-11-28T13:01:13.091757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:02.624167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.700208+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:01:12.700208+0000","last_peered":"2023-11-28T13:01:12.700208+0000","last_clean":"2023-11-28T13:01:12.700208+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:01:12.700208+0000","last_undegraded":"2023-11-28T13:01:12.700208+0000","last_fullsized":"2023-11-28T13:01:12.700208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698573+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:01:12.698573+0000","last_peered":"2023-11-28T13:01:12.698573+0000","last_clean":"2023-11-28T13:01:12.698573+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:01:12.698573+0000","last_undegraded":"2023-11-28T13:01:12.698573+0000","last_fullsized":"2023-11-28T13:01:12.698573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088962+0000","last_change":"2023-11-28T13:01:13.104062+0000","last_active":"2023-11-28T13:01:14.088962+0000","last_peered":"2023-11-28T13:01:14.088962+0000","last_clean":"2023-11-28T13:01:14.088962+0000","last_became_active":"2023-11-28T13:01:13.103815+0000","last_became_peered":"2023-11-28T13:01:13.103815+0000","last_unstale":"2023-11-28T13:01:14.088962+0000","last_undegraded":"2023-11-28T13:01:14.088962+0000","last_fullsized":"2023-11-28T13:01:14.088962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:09.350952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106674+0000","last_change":"2023-11-28T13:01:12.706680+0000","last_active":"2023-11-28T13:01:13.106674+0000","last_peered":"2023-11-28T13:01:13.106674+0000","last_clean":"2023-11-28T13:01:13.106674+0000","last_became_active":"2023-11-28T13:01:12.706387+0000","last_became_peered":"2023-11-28T13:01:12.706387+0000","last_unstale":"2023-11-28T13:01:13.106674+0000","last_undegraded":"2023-11-28T13:01:13.106674+0000","last_fullsized":"2023-11-28T13:01:13.106674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698664+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:01:12.698664+0000","last_peered":"2023-11-28T13:01:12.698664+0000","last_clean":"2023-11-28T13:01:12.698664+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:01:12.698664+0000","last_undegraded":"2023-11-28T13:01:12.698664+0000","last_fullsized":"2023-11-28T13:01:12.698664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088854+0000","last_change":"2023-11-28T13:01:13.100011+0000","last_active":"2023-11-28T13:01:14.088854+0000","last_peered":"2023-11-28T13:01:14.088854+0000","last_clean":"2023-11-28T13:01:14.088854+0000","last_became_active":"2023-11-28T13:01:13.099738+0000","last_became_peered":"2023-11-28T13:01:13.099738+0000","last_unstale":"2023-11-28T13:01:14.088854+0000","last_undegraded":"2023-11-28T13:01:14.088854+0000","last_fullsized":"2023-11-28T13:01:14.088854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:52.246680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081223+0000","last_change":"2023-11-28T13:01:12.694952+0000","last_active":"2023-11-28T13:01:13.081223+0000","last_peered":"2023-11-28T13:01:13.081223+0000","last_clean":"2023-11-28T13:01:13.081223+0000","last_became_active":"2023-11-28T13:01:12.694694+0000","last_became_peered":"2023-11-28T13:01:12.694694+0000","last_unstale":"2023-11-28T13:01:13.081223+0000","last_undegraded":"2023-11-28T13:01:13.081223+0000","last_fullsized":"2023-11-28T13:01:13.081223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:54.638737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699297+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:01:12.699297+0000","last_peered":"2023-11-28T13:01:12.699297+0000","last_clean":"2023-11-28T13:01:12.699297+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:01:12.699297+0000","last_undegraded":"2023-11-28T13:01:12.699297+0000","last_fullsized":"2023-11-28T13:01:12.699297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104886+0000","last_change":"2023-11-28T13:01:13.099302+0000","last_active":"2023-11-28T13:01:13.104886+0000","last_peered":"2023-11-28T13:01:13.104886+0000","last_clean":"2023-11-28T13:01:13.104886+0000","last_became_active":"2023-11-28T13:01:13.099047+0000","last_became_peered":"2023-11-28T13:01:13.099047+0000","last_unstale":"2023-11-28T13:01:13.104886+0000","last_undegraded":"2023-11-28T13:01:13.104886+0000","last_fullsized":"2023-11-28T13:01:13.104886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092648+0000","last_change":"2023-11-28T13:01:12.700645+0000","last_active":"2023-11-28T13:01:13.092648+0000","last_peered":"2023-11-28T13:01:13.092648+0000","last_clean":"2023-11-28T13:01:13.092648+0000","last_became_active":"2023-11-28T13:01:12.700349+0000","last_became_peered":"2023-11-28T13:01:12.700349+0000","last_unstale":"2023-11-28T13:01:13.092648+0000","last_undegraded":"2023-11-28T13:01:13.092648+0000","last_fullsized":"2023-11-28T13:01:13.092648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:50.224596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.086932+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:01:13.086932+0000","last_peered":"2023-11-28T13:01:13.086932+0000","last_clean":"2023-11-28T13:01:13.086932+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:01:13.086932+0000","last_undegraded":"2023-11-28T13:01:13.086932+0000","last_fullsized":"2023-11-28T13:01:13.086932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107847+0000","last_change":"2023-11-28T13:01:13.101838+0000","last_active":"2023-11-28T13:01:13.107847+0000","last_peered":"2023-11-28T13:01:13.107847+0000","last_clean":"2023-11-28T13:01:13.107847+0000","last_became_active":"2023-11-28T13:01:13.101565+0000","last_became_peered":"2023-11-28T13:01:13.101565+0000","last_unstale":"2023-11-28T13:01:13.107847+0000","last_undegraded":"2023-11-28T13:01:13.107847+0000","last_fullsized":"2023-11-28T13:01:13.107847+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:08.927902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090444+0000","last_change":"2023-11-28T13:01:12.699934+0000","last_active":"2023-11-28T13:01:14.090444+0000","last_peered":"2023-11-28T13:01:14.090444+0000","last_clean":"2023-11-28T13:01:14.090444+0000","last_became_active":"2023-11-28T13:01:12.699617+0000","last_became_peered":"2023-11-28T13:01:12.699617+0000","last_unstale":"2023-11-28T13:01:14.090444+0000","last_undegraded":"2023-11-28T13:01:14.090444+0000","last_fullsized":"2023-11-28T13:01:14.090444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:02.801702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.690011+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:01:12.690011+0000","last_peered":"2023-11-28T13:01:12.690011+0000","last_clean":"2023-11-28T13:01:12.690011+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:01:12.690011+0000","last_undegraded":"2023-11-28T13:01:12.690011+0000","last_fullsized":"2023-11-28T13:01:12.690011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":184228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.497},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.218,"5min":8.218,"15min":8.218},"last":0.35099999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.439},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":13.384,"5min":13.384,"15min":13.384},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.226,"5min":5.226,"15min":5.226},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.624}]},{"osd":5,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.3209999999999997,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.1029999999999998,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.3849999999999998,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.70799999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8364,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363476,"statfs":{"total":96636764160,"available":96628199424,"internally_reserved":0,"allocated":430080,"data_stored":176063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.27200000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.247},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42799999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.498},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":168865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.438},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9424,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362416,"statfs":{"total":96636764160,"available":96627113984,"internally_reserved":0,"allocated":991232,"data_stored":751836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":14.997999999999999,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.28299999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":19.045999999999999,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.497},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":18.995000000000001,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":19.02,"5min":19.02,"15min":19.02},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.45000000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.438}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.1539999999999999,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":18.957999999999998,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.51300000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9428,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362412,"statfs":{"total":96636764160,"available":96627109888,"internally_reserved":0,"allocated":995328,"data_stored":752605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66300000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":4.7830000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.9510000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":4.8440000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":2.6030000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.87}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":1.25},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":2.5329999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.972},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":2.508}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8496,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363344,"statfs":{"total":96636764160,"available":96628064256,"internally_reserved":0,"allocated":434176,"data_stored":176105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.435},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.67300000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69599999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8428,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363412,"statfs":{"total":96636764160,"available":96628133888,"internally_reserved":0,"allocated":430080,"data_stored":176559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.629}]},{"osd":3,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.503}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.78000000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8492,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363348,"statfs":{"total":96636764160,"available":96628068352,"internally_reserved":0,"allocated":430080,"data_stored":176083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.247}]},{"osd":2,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.37}]},{"osd":3,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:01:17.543 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:01:17.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.696+0000 7f97b83e4700 1 Processor -- start 2023-11-28T13:01:17.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.696+0000 7f97b83e4700 1 -- start start 2023-11-28T13:01:17.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.697+0000 7f97b83e4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b00d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.697+0000 7f97b83e4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97b00b20b0 0x7f97b00d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.697+0000 7f97b83e4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00b0230 con 0x7f97b00b29b0 2023-11-28T13:01:17.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.697+0000 7f97b83e4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b015cec0 con 0x7f97b00b20b0 2023-11-28T13:01:17.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.697+0000 7f97b83e4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00b4570 con 0x7f97b00d01b0 2023-11-28T13:01:17.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97b00b20b0 0x7f97b00d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97b00b20b0 0x7f97b00d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40274/0 (socket says 172.21.15.153:40274) 2023-11-28T13:01:17.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 -- 172.21.15.153:0/3551805043 learned_addr learned my addr 172.21.15.153:0/3551805043 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:17.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b6180700 1 --2- 172.21.15.153:0/3551805043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b00d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 -- 172.21.15.153:0/3551805043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 msgr2=0x7f97b00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 --2- 172.21.15.153:0/3551805043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b00d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 -- 172.21.15.153:0/3551805043 >> v1:172.21.15.177:6789/0 conn(0x7f97b00b29b0 legacy=0x7f97b00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 -- 172.21.15.153:0/3551805043 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00b4850 con 0x7f97b00b20b0 2023-11-28T13:01:17.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b597f700 1 --2- 172.21.15.153:0/3551805043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00d1160 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0098f0 tx=0x7f97ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8442447826f166 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b517e700 1 -- 172.21.15.153:0/3551805043 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00b8a0 con 0x7f97b00b20b0 2023-11-28T13:01:17.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b517e700 1 -- 172.21.15.153:0/3551805043 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac00ba00 con 0x7f97b00b20b0 2023-11-28T13:01:17.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.698+0000 7f97b517e700 1 -- 172.21.15.153:0/3551805043 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00bd90 con 0x7f97b00b20b0 2023-11-28T13:01:17.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 -- 172.21.15.153:0/3551805043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 msgr2=0x7f97b00d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3551805043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00d1160 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0098f0 tx=0x7f97ac009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:17.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 -- 172.21.15.153:0/3551805043 shutdown_connections 2023-11-28T13:01:17.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3551805043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b00d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3551805043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00d1160 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 -- 172.21.15.153:0/3551805043 >> 172.21.15.153:0/3551805043 conn(0x7f97b009c3f0 msgr2=0x7f97b00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 -- 172.21.15.153:0/3551805043 shutdown_connections 2023-11-28T13:01:17.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.699+0000 7f97b83e4700 1 -- 172.21.15.153:0/3551805043 wait complete. 2023-11-28T13:01:17.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.700+0000 7f97b83e4700 1 Processor -- start 2023-11-28T13:01:17.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.700+0000 7f97b83e4700 1 -- start start 2023-11-28T13:01:17.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b83e4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b83e4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97b00b29b0 0x7f97b00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b83e4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b01ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b83e4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00cc7b0 con 0x7f97b00b20b0 2023-11-28T13:01:17.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b83e4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00cd400 con 0x7f97b00b29b0 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b83e4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01f32a0 con 0x7f97b00d01b0 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40280/0 (socket says 172.21.15.153:40280) 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 -- 172.21.15.153:0/3341224740 learned_addr learned my addr 172.21.15.153:0/3341224740 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b597f700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97b00b29b0 0x7f97b00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6981700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b01ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 -- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 msgr2=0x7f97b01ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b01ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 -- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97b00b29b0 msgr2=0x7f97b00cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97b00b29b0 0x7f97b00cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 -- 172.21.15.153:0/3341224740 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac009550 con 0x7f97b00b20b0 2023-11-28T13:01:17.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b597f700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97b00b29b0 0x7f97b00cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:17.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6981700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b01ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:17.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97b6180700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00b8810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c7f0 tx=0x7f97a000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97a6ffd700 1 -- 172.21.15.153:0/3341224740 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0011800 con 0x7f97b00b20b0 2023-11-28T13:01:17.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.701+0000 7f97a6ffd700 1 -- 172.21.15.153:0/3341224740 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a0011960 con 0x7f97b00b20b0 2023-11-28T13:01:17.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.702+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b00afec0 con 0x7f97b00b20b0 2023-11-28T13:01:17.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.702+0000 7f97a6ffd700 1 -- 172.21.15.153:0/3341224740 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0011cf0 con 0x7f97b00b20b0 2023-11-28T13:01:17.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.702+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b00afbe0 con 0x7f97b00b20b0 2023-11-28T13:01:17.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.703+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b00b2600 con 0x7f97b00b20b0 2023-11-28T13:01:17.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.703+0000 7f97a6ffd700 1 -- 172.21.15.153:0/3341224740 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97a0021070 con 0x7f97b00b20b0 2023-11-28T13:01:17.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.703+0000 7f97a6ffd700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f97940682f0 0x7f979406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.703+0000 7f97a6ffd700 1 -- 172.21.15.153:0/3341224740 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97a0085a40 con 0x7f97b00b20b0 2023-11-28T13:01:17.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.703+0000 7f97b597f700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f97940682f0 0x7f979406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.704+0000 7f97b597f700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f97940682f0 0x7f979406a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f97b00a0f30 tx=0x7f97ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.708+0000 7f97a6ffd700 1 -- 172.21.15.153:0/3341224740 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97a00522a0 con 0x7f97b00b20b0 2023-11-28T13:01:17.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.887+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97b0068690 con 0x7f97940682f0 2023-11-28T13:01:17.896 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.895+0000 7f97a6ffd700 1 -- 172.21.15.153:0/3341224740 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309773 (secure 0 0 0) 0x7f97b0068690 con 0x7f97940682f0 2023-11-28T13:01:17.900 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.898+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f97940682f0 msgr2=0x7f979406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.898+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f97940682f0 0x7f979406a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f97b00a0f30 tx=0x7f97ac008030 comp rx=0 tx=0).stop 2023-11-28T13:01:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 msgr2=0x7f97b00b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00b8810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c7f0 tx=0x7f97a000cbb0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 shutdown_connections 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f97940682f0 0x7f979406a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97b00b20b0 0x7f97b00b8810 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97b00d01b0 0x7f97b01ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 --2- 172.21.15.153:0/3341224740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97b00b29b0 0x7f97b00cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 >> 172.21.15.153:0/3341224740 conn(0x7f97b009c3f0 msgr2=0x7f97b00d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 shutdown_connections 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:17.899+0000 7f97b83e4700 1 -- 172.21.15.153:0/3341224740 wait complete. 2023-11-28T13:01:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:01:17.912 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-11-28T13:01:16.158364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69688,"kb_used_data":4472,"kb_used_omap":0,"kb_used_meta":65152,"kb_avail":754905032,"statfs":{"total":773094113280,"available":773022752768,"internally_reserved":0,"allocated":4579328,"data_stored":2562344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66715648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.478842"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090821+0000","last_change":"2023-11-28T13:01:13.094545+0000","last_active":"2023-11-28T13:01:14.090821+0000","last_peered":"2023-11-28T13:01:14.090821+0000","last_clean":"2023-11-28T13:01:14.090821+0000","last_became_active":"2023-11-28T13:01:13.094296+0000","last_became_peered":"2023-11-28T13:01:13.094296+0000","last_unstale":"2023-11-28T13:01:14.090821+0000","last_undegraded":"2023-11-28T13:01:14.090821+0000","last_fullsized":"2023-11-28T13:01:14.090821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:18.727519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090776+0000","last_change":"2023-11-28T13:01:13.088855+0000","last_active":"2023-11-28T13:01:14.090776+0000","last_peered":"2023-11-28T13:01:14.090776+0000","last_clean":"2023-11-28T13:01:14.090776+0000","last_became_active":"2023-11-28T13:01:13.088631+0000","last_became_peered":"2023-11-28T13:01:13.088631+0000","last_unstale":"2023-11-28T13:01:14.090776+0000","last_undegraded":"2023-11-28T13:01:14.090776+0000","last_fullsized":"2023-11-28T13:01:14.090776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:42.143519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107263+0000","last_change":"2023-11-28T13:01:13.102065+0000","last_active":"2023-11-28T13:01:13.107263+0000","last_peered":"2023-11-28T13:01:13.107263+0000","last_clean":"2023-11-28T13:01:13.107263+0000","last_became_active":"2023-11-28T13:01:13.101822+0000","last_became_peered":"2023-11-28T13:01:13.101822+0000","last_unstale":"2023-11-28T13:01:13.107263+0000","last_undegraded":"2023-11-28T13:01:13.107263+0000","last_fullsized":"2023-11-28T13:01:13.107263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:51.732790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.099900+0000","last_change":"2023-11-28T13:01:13.095907+0000","last_active":"2023-11-28T13:01:13.099900+0000","last_peered":"2023-11-28T13:01:13.099900+0000","last_clean":"2023-11-28T13:01:13.099900+0000","last_became_active":"2023-11-28T13:01:13.095190+0000","last_became_peered":"2023-11-28T13:01:13.095190+0000","last_unstale":"2023-11-28T13:01:13.099900+0000","last_undegraded":"2023-11-28T13:01:13.099900+0000","last_fullsized":"2023-11-28T13:01:13.099900+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:34.713740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088925+0000","last_change":"2023-11-28T13:01:13.103245+0000","last_active":"2023-11-28T13:01:14.088925+0000","last_peered":"2023-11-28T13:01:14.088925+0000","last_clean":"2023-11-28T13:01:14.088925+0000","last_became_active":"2023-11-28T13:01:13.102920+0000","last_became_peered":"2023-11-28T13:01:13.102920+0000","last_unstale":"2023-11-28T13:01:14.088925+0000","last_undegraded":"2023-11-28T13:01:14.088925+0000","last_fullsized":"2023-11-28T13:01:14.088925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:23.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106870+0000","last_change":"2023-11-28T13:01:13.101383+0000","last_active":"2023-11-28T13:01:13.106870+0000","last_peered":"2023-11-28T13:01:13.106870+0000","last_clean":"2023-11-28T13:01:13.106870+0000","last_became_active":"2023-11-28T13:01:13.101093+0000","last_became_peered":"2023-11-28T13:01:13.101093+0000","last_unstale":"2023-11-28T13:01:13.106870+0000","last_undegraded":"2023-11-28T13:01:13.106870+0000","last_fullsized":"2023-11-28T13:01:13.106870+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090405+0000","last_change":"2023-11-28T13:01:13.093176+0000","last_active":"2023-11-28T13:01:14.090405+0000","last_peered":"2023-11-28T13:01:14.090405+0000","last_clean":"2023-11-28T13:01:14.090405+0000","last_became_active":"2023-11-28T13:01:13.092922+0000","last_became_peered":"2023-11-28T13:01:13.092922+0000","last_unstale":"2023-11-28T13:01:14.090405+0000","last_undegraded":"2023-11-28T13:01:14.090405+0000","last_fullsized":"2023-11-28T13:01:14.090405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:31.712364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102138+0000","last_change":"2023-11-28T13:01:13.090026+0000","last_active":"2023-11-28T13:01:13.102138+0000","last_peered":"2023-11-28T13:01:13.102138+0000","last_clean":"2023-11-28T13:01:13.102138+0000","last_became_active":"2023-11-28T13:01:13.089734+0000","last_became_peered":"2023-11-28T13:01:13.089734+0000","last_unstale":"2023-11-28T13:01:13.102138+0000","last_undegraded":"2023-11-28T13:01:13.102138+0000","last_fullsized":"2023-11-28T13:01:13.102138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110483+0000","last_change":"2023-11-28T13:01:13.106317+0000","last_active":"2023-11-28T13:01:13.110483+0000","last_peered":"2023-11-28T13:01:13.110483+0000","last_clean":"2023-11-28T13:01:13.110483+0000","last_became_active":"2023-11-28T13:01:13.106096+0000","last_became_peered":"2023-11-28T13:01:13.106096+0000","last_unstale":"2023-11-28T13:01:13.110483+0000","last_undegraded":"2023-11-28T13:01:13.110483+0000","last_fullsized":"2023-11-28T13:01:13.110483+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102639+0000","last_change":"2023-11-28T13:01:13.098543+0000","last_active":"2023-11-28T13:01:13.102639+0000","last_peered":"2023-11-28T13:01:13.102639+0000","last_clean":"2023-11-28T13:01:13.102639+0000","last_became_active":"2023-11-28T13:01:13.097626+0000","last_became_peered":"2023-11-28T13:01:13.097626+0000","last_unstale":"2023-11-28T13:01:13.102639+0000","last_undegraded":"2023-11-28T13:01:13.102639+0000","last_fullsized":"2023-11-28T13:01:13.102639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104846+0000","last_change":"2023-11-28T13:01:13.097007+0000","last_active":"2023-11-28T13:01:13.104846+0000","last_peered":"2023-11-28T13:01:13.104846+0000","last_clean":"2023-11-28T13:01:13.104846+0000","last_became_active":"2023-11-28T13:01:13.096697+0000","last_became_peered":"2023-11-28T13:01:13.096697+0000","last_unstale":"2023-11-28T13:01:13.104846+0000","last_undegraded":"2023-11-28T13:01:13.104846+0000","last_fullsized":"2023-11-28T13:01:13.104846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102519+0000","last_change":"2023-11-28T13:01:13.092555+0000","last_active":"2023-11-28T13:01:13.102519+0000","last_peered":"2023-11-28T13:01:13.102519+0000","last_clean":"2023-11-28T13:01:13.102519+0000","last_became_active":"2023-11-28T13:01:13.092269+0000","last_became_peered":"2023-11-28T13:01:13.092269+0000","last_unstale":"2023-11-28T13:01:13.102519+0000","last_undegraded":"2023-11-28T13:01:13.102519+0000","last_fullsized":"2023-11-28T13:01:13.102519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110258+0000","last_change":"2023-11-28T13:01:13.106928+0000","last_active":"2023-11-28T13:01:13.110258+0000","last_peered":"2023-11-28T13:01:13.110258+0000","last_clean":"2023-11-28T13:01:13.110258+0000","last_became_active":"2023-11-28T13:01:13.104168+0000","last_became_peered":"2023-11-28T13:01:13.104168+0000","last_unstale":"2023-11-28T13:01:13.110258+0000","last_undegraded":"2023-11-28T13:01:13.110258+0000","last_fullsized":"2023-11-28T13:01:13.110258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110271+0000","last_change":"2023-11-28T13:01:13.106957+0000","last_active":"2023-11-28T13:01:13.110271+0000","last_peered":"2023-11-28T13:01:13.110271+0000","last_clean":"2023-11-28T13:01:13.110271+0000","last_became_active":"2023-11-28T13:01:13.104842+0000","last_became_peered":"2023-11-28T13:01:13.104842+0000","last_unstale":"2023-11-28T13:01:13.110271+0000","last_undegraded":"2023-11-28T13:01:13.110271+0000","last_fullsized":"2023-11-28T13:01:13.110271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105656+0000","last_change":"2023-11-28T13:01:13.102950+0000","last_active":"2023-11-28T13:01:13.105656+0000","last_peered":"2023-11-28T13:01:13.105656+0000","last_clean":"2023-11-28T13:01:13.105656+0000","last_became_active":"2023-11-28T13:01:13.102706+0000","last_became_peered":"2023-11-28T13:01:13.102706+0000","last_unstale":"2023-11-28T13:01:13.105656+0000","last_undegraded":"2023-11-28T13:01:13.105656+0000","last_fullsized":"2023-11-28T13:01:13.105656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:33.362111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.097705+0000","last_change":"2023-11-28T13:01:13.094603+0000","last_active":"2023-11-28T13:01:13.097705+0000","last_peered":"2023-11-28T13:01:13.097705+0000","last_clean":"2023-11-28T13:01:13.097705+0000","last_became_active":"2023-11-28T13:01:13.094377+0000","last_became_peered":"2023-11-28T13:01:13.094377+0000","last_unstale":"2023-11-28T13:01:13.097705+0000","last_undegraded":"2023-11-28T13:01:13.097705+0000","last_fullsized":"2023-11-28T13:01:13.097705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:25.095910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098253+0000","last_change":"2023-11-28T13:01:13.093425+0000","last_active":"2023-11-28T13:01:13.098253+0000","last_peered":"2023-11-28T13:01:13.098253+0000","last_clean":"2023-11-28T13:01:13.098253+0000","last_became_active":"2023-11-28T13:01:13.093128+0000","last_became_peered":"2023-11-28T13:01:13.093128+0000","last_unstale":"2023-11-28T13:01:13.098253+0000","last_undegraded":"2023-11-28T13:01:13.098253+0000","last_fullsized":"2023-11-28T13:01:13.098253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:47.380562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102465+0000","last_change":"2023-11-28T13:01:13.095333+0000","last_active":"2023-11-28T13:01:13.102465+0000","last_peered":"2023-11-28T13:01:13.102465+0000","last_clean":"2023-11-28T13:01:13.102465+0000","last_became_active":"2023-11-28T13:01:13.095050+0000","last_became_peered":"2023-11-28T13:01:13.095050+0000","last_unstale":"2023-11-28T13:01:13.102465+0000","last_undegraded":"2023-11-28T13:01:13.102465+0000","last_fullsized":"2023-11-28T13:01:13.102465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:51:32.434761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107125+0000","last_change":"2023-11-28T13:01:13.105213+0000","last_active":"2023-11-28T13:01:13.107125+0000","last_peered":"2023-11-28T13:01:13.107125+0000","last_clean":"2023-11-28T13:01:13.107125+0000","last_became_active":"2023-11-28T13:01:13.104960+0000","last_became_peered":"2023-11-28T13:01:13.104960+0000","last_unstale":"2023-11-28T13:01:13.107125+0000","last_undegraded":"2023-11-28T13:01:13.107125+0000","last_fullsized":"2023-11-28T13:01:13.107125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110113+0000","last_change":"2023-11-28T13:01:13.102193+0000","last_active":"2023-11-28T13:01:13.110113+0000","last_peered":"2023-11-28T13:01:13.110113+0000","last_clean":"2023-11-28T13:01:13.110113+0000","last_became_active":"2023-11-28T13:01:13.101759+0000","last_became_peered":"2023-11-28T13:01:13.101759+0000","last_unstale":"2023-11-28T13:01:13.110113+0000","last_undegraded":"2023-11-28T13:01:13.110113+0000","last_fullsized":"2023-11-28T13:01:13.110113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:51.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103015+0000","last_change":"2023-11-28T13:01:13.098378+0000","last_active":"2023-11-28T13:01:13.103015+0000","last_peered":"2023-11-28T13:01:13.103015+0000","last_clean":"2023-11-28T13:01:13.103015+0000","last_became_active":"2023-11-28T13:01:13.098101+0000","last_became_peered":"2023-11-28T13:01:13.098101+0000","last_unstale":"2023-11-28T13:01:13.103015+0000","last_undegraded":"2023-11-28T13:01:13.103015+0000","last_fullsized":"2023-11-28T13:01:13.103015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:11.877653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098316+0000","last_change":"2023-11-28T13:01:13.094017+0000","last_active":"2023-11-28T13:01:13.098316+0000","last_peered":"2023-11-28T13:01:13.098316+0000","last_clean":"2023-11-28T13:01:13.098316+0000","last_became_active":"2023-11-28T13:01:13.093753+0000","last_became_peered":"2023-11-28T13:01:13.093753+0000","last_unstale":"2023-11-28T13:01:13.098316+0000","last_undegraded":"2023-11-28T13:01:13.098316+0000","last_fullsized":"2023-11-28T13:01:13.098316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:21.524121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089155+0000","last_change":"2023-11-28T13:01:13.103430+0000","last_active":"2023-11-28T13:01:14.089155+0000","last_peered":"2023-11-28T13:01:14.089155+0000","last_clean":"2023-11-28T13:01:14.089155+0000","last_became_active":"2023-11-28T13:01:13.102784+0000","last_became_peered":"2023-11-28T13:01:13.102784+0000","last_unstale":"2023-11-28T13:01:14.089155+0000","last_undegraded":"2023-11-28T13:01:14.089155+0000","last_fullsized":"2023-11-28T13:01:14.089155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:47.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098350+0000","last_change":"2023-11-28T13:01:13.092484+0000","last_active":"2023-11-28T13:01:13.098350+0000","last_peered":"2023-11-28T13:01:13.098350+0000","last_clean":"2023-11-28T13:01:13.098350+0000","last_became_active":"2023-11-28T13:01:13.092162+0000","last_became_peered":"2023-11-28T13:01:13.092162+0000","last_unstale":"2023-11-28T13:01:13.098350+0000","last_undegraded":"2023-11-28T13:01:13.098350+0000","last_fullsized":"2023-11-28T13:01:13.098350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:03.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107152+0000","last_change":"2023-11-28T13:01:13.100818+0000","last_active":"2023-11-28T13:01:13.107152+0000","last_peered":"2023-11-28T13:01:13.107152+0000","last_clean":"2023-11-28T13:01:13.107152+0000","last_became_active":"2023-11-28T13:01:13.100421+0000","last_became_peered":"2023-11-28T13:01:13.100421+0000","last_unstale":"2023-11-28T13:01:13.107152+0000","last_undegraded":"2023-11-28T13:01:13.107152+0000","last_fullsized":"2023-11-28T13:01:13.107152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098210+0000","last_change":"2023-11-28T13:01:13.093478+0000","last_active":"2023-11-28T13:01:13.098210+0000","last_peered":"2023-11-28T13:01:13.098210+0000","last_clean":"2023-11-28T13:01:13.098210+0000","last_became_active":"2023-11-28T13:01:13.093194+0000","last_became_peered":"2023-11-28T13:01:13.093194+0000","last_unstale":"2023-11-28T13:01:13.098210+0000","last_undegraded":"2023-11-28T13:01:13.098210+0000","last_fullsized":"2023-11-28T13:01:13.098210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:10.435728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098142+0000","last_change":"2023-11-28T13:01:13.088250+0000","last_active":"2023-11-28T13:01:13.098142+0000","last_peered":"2023-11-28T13:01:13.098142+0000","last_clean":"2023-11-28T13:01:13.098142+0000","last_became_active":"2023-11-28T13:01:13.087881+0000","last_became_peered":"2023-11-28T13:01:13.087881+0000","last_unstale":"2023-11-28T13:01:13.098142+0000","last_undegraded":"2023-11-28T13:01:13.098142+0000","last_fullsized":"2023-11-28T13:01:13.098142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:43.126488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110058+0000","last_change":"2023-11-28T13:01:13.102030+0000","last_active":"2023-11-28T13:01:13.110058+0000","last_peered":"2023-11-28T13:01:13.110058+0000","last_clean":"2023-11-28T13:01:13.110058+0000","last_became_active":"2023-11-28T13:01:13.101376+0000","last_became_peered":"2023-11-28T13:01:13.101376+0000","last_unstale":"2023-11-28T13:01:13.110058+0000","last_undegraded":"2023-11-28T13:01:13.110058+0000","last_fullsized":"2023-11-28T13:01:13.110058+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088757+0000","last_change":"2023-11-28T13:01:13.095736+0000","last_active":"2023-11-28T13:01:14.088757+0000","last_peered":"2023-11-28T13:01:14.088757+0000","last_clean":"2023-11-28T13:01:14.088757+0000","last_became_active":"2023-11-28T13:01:13.095469+0000","last_became_peered":"2023-11-28T13:01:13.095469+0000","last_unstale":"2023-11-28T13:01:14.088757+0000","last_undegraded":"2023-11-28T13:01:14.088757+0000","last_fullsized":"2023-11-28T13:01:14.088757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:45.817559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110226+0000","last_change":"2023-11-28T13:01:13.105146+0000","last_active":"2023-11-28T13:01:13.110226+0000","last_peered":"2023-11-28T13:01:13.110226+0000","last_clean":"2023-11-28T13:01:13.110226+0000","last_became_active":"2023-11-28T13:01:13.104852+0000","last_became_peered":"2023-11-28T13:01:13.104852+0000","last_unstale":"2023-11-28T13:01:13.110226+0000","last_undegraded":"2023-11-28T13:01:13.110226+0000","last_fullsized":"2023-11-28T13:01:13.110226+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104391+0000","last_change":"2023-11-28T13:01:12.697633+0000","last_active":"2023-11-28T13:01:13.104391+0000","last_peered":"2023-11-28T13:01:13.104391+0000","last_clean":"2023-11-28T13:01:13.104391+0000","last_became_active":"2023-11-28T13:01:12.697137+0000","last_became_peered":"2023-11-28T13:01:12.697137+0000","last_unstale":"2023-11-28T13:01:13.104391+0000","last_undegraded":"2023-11-28T13:01:13.104391+0000","last_fullsized":"2023-11-28T13:01:13.104391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:59.601679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089117+0000","last_change":"2023-11-28T13:01:13.104026+0000","last_active":"2023-11-28T13:01:14.089117+0000","last_peered":"2023-11-28T13:01:14.089117+0000","last_clean":"2023-11-28T13:01:14.089117+0000","last_became_active":"2023-11-28T13:01:13.103779+0000","last_became_peered":"2023-11-28T13:01:13.103779+0000","last_unstale":"2023-11-28T13:01:14.089117+0000","last_undegraded":"2023-11-28T13:01:14.089117+0000","last_fullsized":"2023-11-28T13:01:14.089117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:13.669167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.685907+0000","last_change":"2023-11-28T13:01:11.685907+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.685907+0000","last_undegraded":"2023-11-28T13:01:11.685907+0000","last_fullsized":"2023-11-28T13:01:11.685907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107203+0000","last_change":"2023-11-28T13:01:13.104225+0000","last_active":"2023-11-28T13:01:13.107203+0000","last_peered":"2023-11-28T13:01:13.107203+0000","last_clean":"2023-11-28T13:01:13.107203+0000","last_became_active":"2023-11-28T13:01:13.103835+0000","last_became_peered":"2023-11-28T13:01:13.103835+0000","last_unstale":"2023-11-28T13:01:13.107203+0000","last_undegraded":"2023-11-28T13:01:13.107203+0000","last_fullsized":"2023-11-28T13:01:13.107203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089210+0000","last_change":"2023-11-28T13:01:12.703691+0000","last_active":"2023-11-28T13:01:14.089210+0000","last_peered":"2023-11-28T13:01:14.089210+0000","last_clean":"2023-11-28T13:01:14.089210+0000","last_became_active":"2023-11-28T13:01:12.703398+0000","last_became_peered":"2023-11-28T13:01:12.703398+0000","last_unstale":"2023-11-28T13:01:14.089210+0000","last_undegraded":"2023-11-28T13:01:14.089210+0000","last_fullsized":"2023-11-28T13:01:14.089210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:24.093954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090818+0000","last_change":"2023-11-28T13:01:13.086251+0000","last_active":"2023-11-28T13:01:14.090818+0000","last_peered":"2023-11-28T13:01:14.090818+0000","last_clean":"2023-11-28T13:01:14.090818+0000","last_became_active":"2023-11-28T13:01:13.085269+0000","last_became_peered":"2023-11-28T13:01:13.085269+0000","last_unstale":"2023-11-28T13:01:14.090818+0000","last_undegraded":"2023-11-28T13:01:14.090818+0000","last_fullsized":"2023-11-28T13:01:14.090818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:04.381796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103953+0000","last_change":"2023-11-28T13:01:12.700070+0000","last_active":"2023-11-28T13:01:13.103953+0000","last_peered":"2023-11-28T13:01:13.103953+0000","last_clean":"2023-11-28T13:01:13.103953+0000","last_became_active":"2023-11-28T13:01:12.699788+0000","last_became_peered":"2023-11-28T13:01:12.699788+0000","last_unstale":"2023-11-28T13:01:13.103953+0000","last_undegraded":"2023-11-28T13:01:13.103953+0000","last_fullsized":"2023-11-28T13:01:13.103953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:37.142758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102494+0000","last_change":"2023-11-28T13:01:12.700665+0000","last_active":"2023-11-28T13:01:13.102494+0000","last_peered":"2023-11-28T13:01:13.102494+0000","last_clean":"2023-11-28T13:01:13.102494+0000","last_became_active":"2023-11-28T13:01:12.700285+0000","last_became_peered":"2023-11-28T13:01:12.700285+0000","last_unstale":"2023-11-28T13:01:13.102494+0000","last_undegraded":"2023-11-28T13:01:13.102494+0000","last_fullsized":"2023-11-28T13:01:13.102494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102539+0000","last_change":"2023-11-28T13:01:13.095159+0000","last_active":"2023-11-28T13:01:13.102539+0000","last_peered":"2023-11-28T13:01:13.102539+0000","last_clean":"2023-11-28T13:01:13.102539+0000","last_became_active":"2023-11-28T13:01:13.094857+0000","last_became_peered":"2023-11-28T13:01:13.094857+0000","last_unstale":"2023-11-28T13:01:13.102539+0000","last_undegraded":"2023-11-28T13:01:13.102539+0000","last_fullsized":"2023-11-28T13:01:13.102539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106011+0000","last_change":"2023-11-28T13:01:12.705296+0000","last_active":"2023-11-28T13:01:13.106011+0000","last_peered":"2023-11-28T13:01:13.106011+0000","last_clean":"2023-11-28T13:01:13.106011+0000","last_became_active":"2023-11-28T13:01:12.705013+0000","last_became_peered":"2023-11-28T13:01:12.705013+0000","last_unstale":"2023-11-28T13:01:13.106011+0000","last_undegraded":"2023-11-28T13:01:13.106011+0000","last_fullsized":"2023-11-28T13:01:13.106011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:46.378606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104868+0000","last_change":"2023-11-28T13:01:13.094747+0000","last_active":"2023-11-28T13:01:13.104868+0000","last_peered":"2023-11-28T13:01:13.104868+0000","last_clean":"2023-11-28T13:01:13.104868+0000","last_became_active":"2023-11-28T13:01:13.094318+0000","last_became_peered":"2023-11-28T13:01:13.094318+0000","last_unstale":"2023-11-28T13:01:13.104868+0000","last_undegraded":"2023-11-28T13:01:13.104868+0000","last_fullsized":"2023-11-28T13:01:13.104868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:27.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089212+0000","last_change":"2023-11-28T13:01:12.706058+0000","last_active":"2023-11-28T13:01:14.089212+0000","last_peered":"2023-11-28T13:01:14.089212+0000","last_clean":"2023-11-28T13:01:14.089212+0000","last_became_active":"2023-11-28T13:01:12.705853+0000","last_became_peered":"2023-11-28T13:01:12.705853+0000","last_unstale":"2023-11-28T13:01:14.089212+0000","last_undegraded":"2023-11-28T13:01:14.089212+0000","last_fullsized":"2023-11-28T13:01:14.089212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:10.875697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090633+0000","last_change":"2023-11-28T13:01:13.099221+0000","last_active":"2023-11-28T13:01:14.090633+0000","last_peered":"2023-11-28T13:01:14.090633+0000","last_clean":"2023-11-28T13:01:14.090633+0000","last_became_active":"2023-11-28T13:01:13.098525+0000","last_became_peered":"2023-11-28T13:01:13.098525+0000","last_unstale":"2023-11-28T13:01:14.090633+0000","last_undegraded":"2023-11-28T13:01:14.090633+0000","last_fullsized":"2023-11-28T13:01:14.090633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:01.070045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092473+0000","last_change":"2023-11-28T13:01:12.700642+0000","last_active":"2023-11-28T13:01:13.092473+0000","last_peered":"2023-11-28T13:01:13.092473+0000","last_clean":"2023-11-28T13:01:13.092473+0000","last_became_active":"2023-11-28T13:01:12.700381+0000","last_became_peered":"2023-11-28T13:01:12.700381+0000","last_unstale":"2023-11-28T13:01:13.092473+0000","last_undegraded":"2023-11-28T13:01:13.092473+0000","last_fullsized":"2023-11-28T13:01:13.092473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:37.268919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110533+0000","last_change":"2023-11-28T13:01:13.107217+0000","last_active":"2023-11-28T13:01:13.110533+0000","last_peered":"2023-11-28T13:01:13.110533+0000","last_clean":"2023-11-28T13:01:13.110533+0000","last_became_active":"2023-11-28T13:01:13.106967+0000","last_became_peered":"2023-11-28T13:01:13.106967+0000","last_unstale":"2023-11-28T13:01:13.110533+0000","last_undegraded":"2023-11-28T13:01:13.110533+0000","last_fullsized":"2023-11-28T13:01:13.110533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:01.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090884+0000","last_change":"2023-11-28T13:01:12.700536+0000","last_active":"2023-11-28T13:01:14.090884+0000","last_peered":"2023-11-28T13:01:14.090884+0000","last_clean":"2023-11-28T13:01:14.090884+0000","last_became_active":"2023-11-28T13:01:12.699782+0000","last_became_peered":"2023-11-28T13:01:12.699782+0000","last_unstale":"2023-11-28T13:01:14.090884+0000","last_undegraded":"2023-11-28T13:01:14.090884+0000","last_fullsized":"2023-11-28T13:01:14.090884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:17.146583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110165+0000","last_change":"2023-11-28T13:01:13.103462+0000","last_active":"2023-11-28T13:01:13.110165+0000","last_peered":"2023-11-28T13:01:13.110165+0000","last_clean":"2023-11-28T13:01:13.110165+0000","last_became_active":"2023-11-28T13:01:13.102855+0000","last_became_peered":"2023-11-28T13:01:13.102855+0000","last_unstale":"2023-11-28T13:01:13.110165+0000","last_undegraded":"2023-11-28T13:01:13.110165+0000","last_fullsized":"2023-11-28T13:01:13.110165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.987104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092356+0000","last_change":"2023-11-28T13:01:12.703891+0000","last_active":"2023-11-28T13:01:13.092356+0000","last_peered":"2023-11-28T13:01:13.092356+0000","last_clean":"2023-11-28T13:01:13.092356+0000","last_became_active":"2023-11-28T13:01:12.703642+0000","last_became_peered":"2023-11-28T13:01:12.703642+0000","last_unstale":"2023-11-28T13:01:13.092356+0000","last_undegraded":"2023-11-28T13:01:13.092356+0000","last_fullsized":"2023-11-28T13:01:13.092356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:32.360155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105022+0000","last_change":"2023-11-28T13:01:12.704636+0000","last_active":"2023-11-28T13:01:13.105022+0000","last_peered":"2023-11-28T13:01:13.105022+0000","last_clean":"2023-11-28T13:01:13.105022+0000","last_became_active":"2023-11-28T13:01:12.704357+0000","last_became_peered":"2023-11-28T13:01:12.704357+0000","last_unstale":"2023-11-28T13:01:13.105022+0000","last_undegraded":"2023-11-28T13:01:13.105022+0000","last_fullsized":"2023-11-28T13:01:13.105022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105277+0000","last_change":"2023-11-28T13:01:13.096971+0000","last_active":"2023-11-28T13:01:13.105277+0000","last_peered":"2023-11-28T13:01:13.105277+0000","last_clean":"2023-11-28T13:01:13.105277+0000","last_became_active":"2023-11-28T13:01:13.096698+0000","last_became_peered":"2023-11-28T13:01:13.096698+0000","last_unstale":"2023-11-28T13:01:13.105277+0000","last_undegraded":"2023-11-28T13:01:13.105277+0000","last_fullsized":"2023-11-28T13:01:13.105277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090728+0000","last_change":"2023-11-28T13:01:12.704846+0000","last_active":"2023-11-28T13:01:14.090728+0000","last_peered":"2023-11-28T13:01:14.090728+0000","last_clean":"2023-11-28T13:01:14.090728+0000","last_became_active":"2023-11-28T13:01:12.704605+0000","last_became_peered":"2023-11-28T13:01:12.704605+0000","last_unstale":"2023-11-28T13:01:14.090728+0000","last_undegraded":"2023-11-28T13:01:14.090728+0000","last_fullsized":"2023-11-28T13:01:14.090728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:43.923726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090672+0000","last_change":"2023-11-28T13:01:12.700930+0000","last_active":"2023-11-28T13:01:14.090672+0000","last_peered":"2023-11-28T13:01:14.090672+0000","last_clean":"2023-11-28T13:01:14.090672+0000","last_became_active":"2023-11-28T13:01:12.700670+0000","last_became_peered":"2023-11-28T13:01:12.700670+0000","last_unstale":"2023-11-28T13:01:14.090672+0000","last_undegraded":"2023-11-28T13:01:14.090672+0000","last_fullsized":"2023-11-28T13:01:14.090672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090509+0000","last_change":"2023-11-28T13:01:13.093993+0000","last_active":"2023-11-28T13:01:14.090509+0000","last_peered":"2023-11-28T13:01:14.090509+0000","last_clean":"2023-11-28T13:01:14.090509+0000","last_became_active":"2023-11-28T13:01:13.093750+0000","last_became_peered":"2023-11-28T13:01:13.093750+0000","last_unstale":"2023-11-28T13:01:14.090509+0000","last_undegraded":"2023-11-28T13:01:14.090509+0000","last_fullsized":"2023-11-28T13:01:14.090509+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:47.159742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088952+0000","last_change":"2023-11-28T13:01:12.706087+0000","last_active":"2023-11-28T13:01:14.088952+0000","last_peered":"2023-11-28T13:01:14.088952+0000","last_clean":"2023-11-28T13:01:14.088952+0000","last_became_active":"2023-11-28T13:01:12.705800+0000","last_became_peered":"2023-11-28T13:01:12.705800+0000","last_unstale":"2023-11-28T13:01:14.088952+0000","last_undegraded":"2023-11-28T13:01:14.088952+0000","last_fullsized":"2023-11-28T13:01:14.088952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102554+0000","last_change":"2023-11-28T13:01:13.101116+0000","last_active":"2023-11-28T13:01:13.102554+0000","last_peered":"2023-11-28T13:01:13.102554+0000","last_clean":"2023-11-28T13:01:13.102554+0000","last_became_active":"2023-11-28T13:01:13.100510+0000","last_became_peered":"2023-11-28T13:01:13.100510+0000","last_unstale":"2023-11-28T13:01:13.102554+0000","last_undegraded":"2023-11-28T13:01:13.102554+0000","last_fullsized":"2023-11-28T13:01:13.102554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:42.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081264+0000","last_change":"2023-11-28T13:01:12.702749+0000","last_active":"2023-11-28T13:01:13.081264+0000","last_peered":"2023-11-28T13:01:13.081264+0000","last_clean":"2023-11-28T13:01:13.081264+0000","last_became_active":"2023-11-28T13:01:12.702450+0000","last_became_peered":"2023-11-28T13:01:12.702450+0000","last_unstale":"2023-11-28T13:01:13.081264+0000","last_undegraded":"2023-11-28T13:01:13.081264+0000","last_fullsized":"2023-11-28T13:01:13.081264+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:38.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090531+0000","last_change":"2023-11-28T13:01:13.095487+0000","last_active":"2023-11-28T13:01:14.090531+0000","last_peered":"2023-11-28T13:01:14.090531+0000","last_clean":"2023-11-28T13:01:14.090531+0000","last_became_active":"2023-11-28T13:01:13.095222+0000","last_became_peered":"2023-11-28T13:01:13.095222+0000","last_unstale":"2023-11-28T13:01:14.090531+0000","last_undegraded":"2023-11-28T13:01:14.090531+0000","last_fullsized":"2023-11-28T13:01:14.090531+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:57.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106265+0000","last_change":"2023-11-28T13:01:12.704714+0000","last_active":"2023-11-28T13:01:13.106265+0000","last_peered":"2023-11-28T13:01:13.106265+0000","last_clean":"2023-11-28T13:01:13.106265+0000","last_became_active":"2023-11-28T13:01:12.704422+0000","last_became_peered":"2023-11-28T13:01:12.704422+0000","last_unstale":"2023-11-28T13:01:13.106265+0000","last_undegraded":"2023-11-28T13:01:13.106265+0000","last_fullsized":"2023-11-28T13:01:13.106265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:42.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090726+0000","last_change":"2023-11-28T13:01:13.088050+0000","last_active":"2023-11-28T13:01:14.090726+0000","last_peered":"2023-11-28T13:01:14.090726+0000","last_clean":"2023-11-28T13:01:14.090726+0000","last_became_active":"2023-11-28T13:01:13.085247+0000","last_became_peered":"2023-11-28T13:01:13.085247+0000","last_unstale":"2023-11-28T13:01:14.090726+0000","last_undegraded":"2023-11-28T13:01:14.090726+0000","last_fullsized":"2023-11-28T13:01:14.090726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:39.035787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107176+0000","last_change":"2023-11-28T13:01:12.702173+0000","last_active":"2023-11-28T13:01:13.107176+0000","last_peered":"2023-11-28T13:01:13.107176+0000","last_clean":"2023-11-28T13:01:13.107176+0000","last_became_active":"2023-11-28T13:01:12.701243+0000","last_became_peered":"2023-11-28T13:01:12.701243+0000","last_unstale":"2023-11-28T13:01:13.107176+0000","last_undegraded":"2023-11-28T13:01:13.107176+0000","last_fullsized":"2023-11-28T13:01:13.107176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102621+0000","last_change":"2023-11-28T13:01:13.099301+0000","last_active":"2023-11-28T13:01:13.102621+0000","last_peered":"2023-11-28T13:01:13.102621+0000","last_clean":"2023-11-28T13:01:13.102621+0000","last_became_active":"2023-11-28T13:01:13.099017+0000","last_became_peered":"2023-11-28T13:01:13.099017+0000","last_unstale":"2023-11-28T13:01:13.102621+0000","last_undegraded":"2023-11-28T13:01:13.102621+0000","last_fullsized":"2023-11-28T13:01:13.102621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104381+0000","last_change":"2023-11-28T13:01:12.706078+0000","last_active":"2023-11-28T13:01:13.104381+0000","last_peered":"2023-11-28T13:01:13.104381+0000","last_clean":"2023-11-28T13:01:13.104381+0000","last_became_active":"2023-11-28T13:01:12.705694+0000","last_became_peered":"2023-11-28T13:01:12.705694+0000","last_unstale":"2023-11-28T13:01:13.104381+0000","last_undegraded":"2023-11-28T13:01:13.104381+0000","last_fullsized":"2023-11-28T13:01:13.104381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:41.987277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105893+0000","last_change":"2023-11-28T13:01:13.098692+0000","last_active":"2023-11-28T13:01:13.105893+0000","last_peered":"2023-11-28T13:01:13.105893+0000","last_clean":"2023-11-28T13:01:13.105893+0000","last_became_active":"2023-11-28T13:01:13.098426+0000","last_became_peered":"2023-11-28T13:01:13.098426+0000","last_unstale":"2023-11-28T13:01:13.105893+0000","last_undegraded":"2023-11-28T13:01:13.105893+0000","last_fullsized":"2023-11-28T13:01:13.105893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.101277+0000","last_change":"2023-11-28T13:01:12.701155+0000","last_active":"2023-11-28T13:01:13.101277+0000","last_peered":"2023-11-28T13:01:13.101277+0000","last_clean":"2023-11-28T13:01:13.101277+0000","last_became_active":"2023-11-28T13:01:12.700897+0000","last_became_peered":"2023-11-28T13:01:12.700897+0000","last_unstale":"2023-11-28T13:01:13.101277+0000","last_undegraded":"2023-11-28T13:01:13.101277+0000","last_fullsized":"2023-11-28T13:01:13.101277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:57.063655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.100622+0000","last_change":"2023-11-28T13:01:13.094885+0000","last_active":"2023-11-28T13:01:13.100622+0000","last_peered":"2023-11-28T13:01:13.100622+0000","last_clean":"2023-11-28T13:01:13.100622+0000","last_became_active":"2023-11-28T13:01:13.094027+0000","last_became_peered":"2023-11-28T13:01:13.094027+0000","last_unstale":"2023-11-28T13:01:13.100622+0000","last_undegraded":"2023-11-28T13:01:13.100622+0000","last_fullsized":"2023-11-28T13:01:13.100622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:43.321762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089094+0000","last_change":"2023-11-28T13:01:12.703499+0000","last_active":"2023-11-28T13:01:14.089094+0000","last_peered":"2023-11-28T13:01:14.089094+0000","last_clean":"2023-11-28T13:01:14.089094+0000","last_became_active":"2023-11-28T13:01:12.703271+0000","last_became_peered":"2023-11-28T13:01:12.703271+0000","last_unstale":"2023-11-28T13:01:14.089094+0000","last_undegraded":"2023-11-28T13:01:14.089094+0000","last_fullsized":"2023-11-28T13:01:14.089094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:03:30.119943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107824+0000","last_change":"2023-11-28T13:01:13.101678+0000","last_active":"2023-11-28T13:01:13.107824+0000","last_peered":"2023-11-28T13:01:13.107824+0000","last_clean":"2023-11-28T13:01:13.107824+0000","last_became_active":"2023-11-28T13:01:13.101386+0000","last_became_peered":"2023-11-28T13:01:13.101386+0000","last_unstale":"2023-11-28T13:01:13.107824+0000","last_undegraded":"2023-11-28T13:01:13.107824+0000","last_fullsized":"2023-11-28T13:01:13.107824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:51.226552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.095695+0000","last_change":"2023-11-28T13:01:12.697001+0000","last_active":"2023-11-28T13:01:13.095695+0000","last_peered":"2023-11-28T13:01:13.095695+0000","last_clean":"2023-11-28T13:01:13.095695+0000","last_became_active":"2023-11-28T13:01:12.696739+0000","last_became_peered":"2023-11-28T13:01:12.696739+0000","last_unstale":"2023-11-28T13:01:13.095695+0000","last_undegraded":"2023-11-28T13:01:13.095695+0000","last_fullsized":"2023-11-28T13:01:13.095695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:37.438564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083496+0000","last_change":"2023-11-28T13:01:12.698523+0000","last_active":"2023-11-28T13:01:13.083496+0000","last_peered":"2023-11-28T13:01:13.083496+0000","last_clean":"2023-11-28T13:01:13.083496+0000","last_became_active":"2023-11-28T13:01:12.698275+0000","last_became_peered":"2023-11-28T13:01:12.698275+0000","last_unstale":"2023-11-28T13:01:13.083496+0000","last_undegraded":"2023-11-28T13:01:13.083496+0000","last_fullsized":"2023-11-28T13:01:13.083496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:02.254485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102691+0000","last_change":"2023-11-28T13:01:13.099007+0000","last_active":"2023-11-28T13:01:13.102691+0000","last_peered":"2023-11-28T13:01:13.102691+0000","last_clean":"2023-11-28T13:01:13.102691+0000","last_became_active":"2023-11-28T13:01:13.098673+0000","last_became_peered":"2023-11-28T13:01:13.098673+0000","last_unstale":"2023-11-28T13:01:13.102691+0000","last_undegraded":"2023-11-28T13:01:13.102691+0000","last_fullsized":"2023-11-28T13:01:13.102691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091666+0000","last_change":"2023-11-28T13:01:12.706237+0000","last_active":"2023-11-28T13:01:13.091666+0000","last_peered":"2023-11-28T13:01:13.091666+0000","last_clean":"2023-11-28T13:01:13.091666+0000","last_became_active":"2023-11-28T13:01:12.705981+0000","last_became_peered":"2023-11-28T13:01:12.705981+0000","last_unstale":"2023-11-28T13:01:13.091666+0000","last_undegraded":"2023-11-28T13:01:13.091666+0000","last_fullsized":"2023-11-28T13:01:13.091666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090578+0000","last_change":"2023-11-28T13:01:13.099382+0000","last_active":"2023-11-28T13:01:14.090578+0000","last_peered":"2023-11-28T13:01:14.090578+0000","last_clean":"2023-11-28T13:01:14.090578+0000","last_became_active":"2023-11-28T13:01:13.099008+0000","last_became_peered":"2023-11-28T13:01:13.099008+0000","last_unstale":"2023-11-28T13:01:14.090578+0000","last_undegraded":"2023-11-28T13:01:14.090578+0000","last_fullsized":"2023-11-28T13:01:14.090578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:35.600930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083382+0000","last_change":"2023-11-28T13:01:12.696728+0000","last_active":"2023-11-28T13:01:13.083382+0000","last_peered":"2023-11-28T13:01:13.083382+0000","last_clean":"2023-11-28T13:01:13.083382+0000","last_became_active":"2023-11-28T13:01:12.696428+0000","last_became_peered":"2023-11-28T13:01:12.696428+0000","last_unstale":"2023-11-28T13:01:13.083382+0000","last_undegraded":"2023-11-28T13:01:13.083382+0000","last_fullsized":"2023-11-28T13:01:13.083382+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:28.781799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:01:12.679463+0000","last_change":"2023-11-28T13:01:12.679463+0000","last_active":"2023-11-28T13:01:12.679463+0000","last_peered":"2023-11-28T13:01:12.679463+0000","last_clean":"2023-11-28T13:01:12.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:12.679463+0000","last_undegraded":"2023-11-28T13:01:12.679463+0000","last_fullsized":"2023-11-28T13:01:12.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.686000+0000","last_change":"2023-11-28T13:01:11.686000+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.686000+0000","last_undegraded":"2023-11-28T13:01:11.686000+0000","last_fullsized":"2023-11-28T13:01:11.686000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091534+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:01:13.091534+0000","last_peered":"2023-11-28T13:01:13.091534+0000","last_clean":"2023-11-28T13:01:13.091534+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:01:13.091534+0000","last_undegraded":"2023-11-28T13:01:13.091534+0000","last_fullsized":"2023-11-28T13:01:13.091534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090805+0000","last_change":"2023-11-28T13:01:13.098873+0000","last_active":"2023-11-28T13:01:14.090805+0000","last_peered":"2023-11-28T13:01:14.090805+0000","last_clean":"2023-11-28T13:01:14.090805+0000","last_became_active":"2023-11-28T13:01:13.098618+0000","last_became_peered":"2023-11-28T13:01:13.098618+0000","last_unstale":"2023-11-28T13:01:14.090805+0000","last_undegraded":"2023-11-28T13:01:14.090805+0000","last_fullsized":"2023-11-28T13:01:14.090805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:49.374140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-11-28T13:01:11.686366+0000","last_change":"2023-11-28T13:01:11.686366+0000","last_active":"2023-11-28T13:01:11.677507+0000","last_peered":"2023-11-28T13:01:11.677507+0000","last_clean":"2023-11-28T13:01:11.677507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:01:11.686366+0000","last_undegraded":"2023-11-28T13:01:11.686366+0000","last_fullsized":"2023-11-28T13:01:11.686366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699954+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:01:12.699954+0000","last_peered":"2023-11-28T13:01:12.699954+0000","last_clean":"2023-11-28T13:01:12.699954+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:01:12.699954+0000","last_undegraded":"2023-11-28T13:01:12.699954+0000","last_fullsized":"2023-11-28T13:01:12.699954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110193+0000","last_change":"2023-11-28T13:01:13.103633+0000","last_active":"2023-11-28T13:01:13.110193+0000","last_peered":"2023-11-28T13:01:13.110193+0000","last_clean":"2023-11-28T13:01:13.110193+0000","last_became_active":"2023-11-28T13:01:13.103198+0000","last_became_peered":"2023-11-28T13:01:13.103198+0000","last_unstale":"2023-11-28T13:01:13.110193+0000","last_undegraded":"2023-11-28T13:01:13.110193+0000","last_fullsized":"2023-11-28T13:01:13.110193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.084615+0000","last_change":"2023-11-28T13:01:12.704388+0000","last_active":"2023-11-28T13:01:13.084615+0000","last_peered":"2023-11-28T13:01:13.084615+0000","last_clean":"2023-11-28T13:01:13.084615+0000","last_became_active":"2023-11-28T13:01:12.704082+0000","last_became_peered":"2023-11-28T13:01:12.704082+0000","last_unstale":"2023-11-28T13:01:13.084615+0000","last_undegraded":"2023-11-28T13:01:13.084615+0000","last_fullsized":"2023-11-28T13:01:13.084615+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:08.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.685641+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:01:12.685641+0000","last_peered":"2023-11-28T13:01:12.685641+0000","last_clean":"2023-11-28T13:01:12.685641+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:01:12.685641+0000","last_undegraded":"2023-11-28T13:01:12.685641+0000","last_fullsized":"2023-11-28T13:01:12.685641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110430+0000","last_change":"2023-11-28T13:01:13.105495+0000","last_active":"2023-11-28T13:01:13.110430+0000","last_peered":"2023-11-28T13:01:13.110430+0000","last_clean":"2023-11-28T13:01:13.110430+0000","last_became_active":"2023-11-28T13:01:13.103141+0000","last_became_peered":"2023-11-28T13:01:13.103141+0000","last_unstale":"2023-11-28T13:01:13.110430+0000","last_undegraded":"2023-11-28T13:01:13.110430+0000","last_fullsized":"2023-11-28T13:01:13.110430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:08.241448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091757+0000","last_change":"2023-11-28T13:01:12.704702+0000","last_active":"2023-11-28T13:01:13.091757+0000","last_peered":"2023-11-28T13:01:13.091757+0000","last_clean":"2023-11-28T13:01:13.091757+0000","last_became_active":"2023-11-28T13:01:12.704416+0000","last_became_peered":"2023-11-28T13:01:12.704416+0000","last_unstale":"2023-11-28T13:01:13.091757+0000","last_undegraded":"2023-11-28T13:01:13.091757+0000","last_fullsized":"2023-11-28T13:01:13.091757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:02.624167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.700208+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:01:12.700208+0000","last_peered":"2023-11-28T13:01:12.700208+0000","last_clean":"2023-11-28T13:01:12.700208+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:01:12.700208+0000","last_undegraded":"2023-11-28T13:01:12.700208+0000","last_fullsized":"2023-11-28T13:01:12.700208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698573+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:01:12.698573+0000","last_peered":"2023-11-28T13:01:12.698573+0000","last_clean":"2023-11-28T13:01:12.698573+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:01:12.698573+0000","last_undegraded":"2023-11-28T13:01:12.698573+0000","last_fullsized":"2023-11-28T13:01:12.698573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088962+0000","last_change":"2023-11-28T13:01:13.104062+0000","last_active":"2023-11-28T13:01:14.088962+0000","last_peered":"2023-11-28T13:01:14.088962+0000","last_clean":"2023-11-28T13:01:14.088962+0000","last_became_active":"2023-11-28T13:01:13.103815+0000","last_became_peered":"2023-11-28T13:01:13.103815+0000","last_unstale":"2023-11-28T13:01:14.088962+0000","last_undegraded":"2023-11-28T13:01:14.088962+0000","last_fullsized":"2023-11-28T13:01:14.088962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:09.350952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106674+0000","last_change":"2023-11-28T13:01:12.706680+0000","last_active":"2023-11-28T13:01:13.106674+0000","last_peered":"2023-11-28T13:01:13.106674+0000","last_clean":"2023-11-28T13:01:13.106674+0000","last_became_active":"2023-11-28T13:01:12.706387+0000","last_became_peered":"2023-11-28T13:01:12.706387+0000","last_unstale":"2023-11-28T13:01:13.106674+0000","last_undegraded":"2023-11-28T13:01:13.106674+0000","last_fullsized":"2023-11-28T13:01:13.106674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698664+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:01:12.698664+0000","last_peered":"2023-11-28T13:01:12.698664+0000","last_clean":"2023-11-28T13:01:12.698664+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:01:12.698664+0000","last_undegraded":"2023-11-28T13:01:12.698664+0000","last_fullsized":"2023-11-28T13:01:12.698664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088854+0000","last_change":"2023-11-28T13:01:13.100011+0000","last_active":"2023-11-28T13:01:14.088854+0000","last_peered":"2023-11-28T13:01:14.088854+0000","last_clean":"2023-11-28T13:01:14.088854+0000","last_became_active":"2023-11-28T13:01:13.099738+0000","last_became_peered":"2023-11-28T13:01:13.099738+0000","last_unstale":"2023-11-28T13:01:14.088854+0000","last_undegraded":"2023-11-28T13:01:14.088854+0000","last_fullsized":"2023-11-28T13:01:14.088854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:52.246680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081223+0000","last_change":"2023-11-28T13:01:12.694952+0000","last_active":"2023-11-28T13:01:13.081223+0000","last_peered":"2023-11-28T13:01:13.081223+0000","last_clean":"2023-11-28T13:01:13.081223+0000","last_became_active":"2023-11-28T13:01:12.694694+0000","last_became_peered":"2023-11-28T13:01:12.694694+0000","last_unstale":"2023-11-28T13:01:13.081223+0000","last_undegraded":"2023-11-28T13:01:13.081223+0000","last_fullsized":"2023-11-28T13:01:13.081223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:54.638737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699297+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:01:12.699297+0000","last_peered":"2023-11-28T13:01:12.699297+0000","last_clean":"2023-11-28T13:01:12.699297+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:01:12.699297+0000","last_undegraded":"2023-11-28T13:01:12.699297+0000","last_fullsized":"2023-11-28T13:01:12.699297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104886+0000","last_change":"2023-11-28T13:01:13.099302+0000","last_active":"2023-11-28T13:01:13.104886+0000","last_peered":"2023-11-28T13:01:13.104886+0000","last_clean":"2023-11-28T13:01:13.104886+0000","last_became_active":"2023-11-28T13:01:13.099047+0000","last_became_peered":"2023-11-28T13:01:13.099047+0000","last_unstale":"2023-11-28T13:01:13.104886+0000","last_undegraded":"2023-11-28T13:01:13.104886+0000","last_fullsized":"2023-11-28T13:01:13.104886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092648+0000","last_change":"2023-11-28T13:01:12.700645+0000","last_active":"2023-11-28T13:01:13.092648+0000","last_peered":"2023-11-28T13:01:13.092648+0000","last_clean":"2023-11-28T13:01:13.092648+0000","last_became_active":"2023-11-28T13:01:12.700349+0000","last_became_peered":"2023-11-28T13:01:12.700349+0000","last_unstale":"2023-11-28T13:01:13.092648+0000","last_undegraded":"2023-11-28T13:01:13.092648+0000","last_fullsized":"2023-11-28T13:01:13.092648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:50.224596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.086932+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:01:13.086932+0000","last_peered":"2023-11-28T13:01:13.086932+0000","last_clean":"2023-11-28T13:01:13.086932+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:01:13.086932+0000","last_undegraded":"2023-11-28T13:01:13.086932+0000","last_fullsized":"2023-11-28T13:01:13.086932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107847+0000","last_change":"2023-11-28T13:01:13.101838+0000","last_active":"2023-11-28T13:01:13.107847+0000","last_peered":"2023-11-28T13:01:13.107847+0000","last_clean":"2023-11-28T13:01:13.107847+0000","last_became_active":"2023-11-28T13:01:13.101565+0000","last_became_peered":"2023-11-28T13:01:13.101565+0000","last_unstale":"2023-11-28T13:01:13.107847+0000","last_undegraded":"2023-11-28T13:01:13.107847+0000","last_fullsized":"2023-11-28T13:01:13.107847+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:08.927902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090444+0000","last_change":"2023-11-28T13:01:12.699934+0000","last_active":"2023-11-28T13:01:14.090444+0000","last_peered":"2023-11-28T13:01:14.090444+0000","last_clean":"2023-11-28T13:01:14.090444+0000","last_became_active":"2023-11-28T13:01:12.699617+0000","last_became_peered":"2023-11-28T13:01:12.699617+0000","last_unstale":"2023-11-28T13:01:14.090444+0000","last_undegraded":"2023-11-28T13:01:14.090444+0000","last_fullsized":"2023-11-28T13:01:14.090444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:02.801702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.690011+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:01:12.690011+0000","last_peered":"2023-11-28T13:01:12.690011+0000","last_clean":"2023-11-28T13:01:12.690011+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:01:12.690011+0000","last_undegraded":"2023-11-28T13:01:12.690011+0000","last_fullsized":"2023-11-28T13:01:12.690011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":184228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.497},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.218,"5min":8.218,"15min":8.218},"last":0.35099999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.439},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":13.384,"5min":13.384,"15min":13.384},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.226,"5min":5.226,"15min":5.226},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.624}]},{"osd":5,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.3209999999999997,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.1029999999999998,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.3849999999999998,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.70799999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8364,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363476,"statfs":{"total":96636764160,"available":96628199424,"internally_reserved":0,"allocated":430080,"data_stored":176063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.27200000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.247},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42799999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.498},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":168865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.438},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9424,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362416,"statfs":{"total":96636764160,"available":96627113984,"internally_reserved":0,"allocated":991232,"data_stored":751836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":14.997999999999999,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.28299999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":19.045999999999999,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.497},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":18.995000000000001,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":19.02,"5min":19.02,"15min":19.02},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.45000000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.438}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.1539999999999999,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":18.957999999999998,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.51300000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9428,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362412,"statfs":{"total":96636764160,"available":96627109888,"internally_reserved":0,"allocated":995328,"data_stored":752605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66300000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":4.7830000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.9510000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":4.8440000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":2.6030000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.87}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":1.25},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":2.5329999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.972},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":2.508}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8496,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363344,"statfs":{"total":96636764160,"available":96628064256,"internally_reserved":0,"allocated":434176,"data_stored":176105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.435},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.67300000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69599999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8428,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363412,"statfs":{"total":96636764160,"available":96628133888,"internally_reserved":0,"allocated":430080,"data_stored":176559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.629}]},{"osd":3,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.503}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.78000000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8492,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363348,"statfs":{"total":96636764160,"available":96628068352,"internally_reserved":0,"allocated":430080,"data_stored":176083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.247}]},{"osd":2,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.37}]},{"osd":3,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:01:17.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-11-28T13:01:17.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-11-28T13:01:17.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:01:11.685907+0000', 'last_change': '2023-11-28T13:01:11.685907+0000', 'last_active': '2023-11-28T13:01:11.677507+0000', 'last_peered': '2023-11-28T13:01:11.677507+0000', 'last_clean': '2023-11-28T13:01:11.677507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:11.685907+0000', 'last_undegraded': '2023-11-28T13:01:11.685907+0000', 'last_fullsized': '2023-11-28T13:01:11.685907+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-11-28T13:01:17.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:01:12.679463+0000', 'last_change': '2023-11-28T13:01:12.679463+0000', 'last_active': '2023-11-28T13:01:12.679463+0000', 'last_peered': '2023-11-28T13:01:12.679463+0000', 'last_clean': '2023-11-28T13:01:12.679463+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:12.679463+0000', 'last_undegraded': '2023-11-28T13:01:12.679463+0000', 'last_fullsized': '2023-11-28T13:01:12.679463+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:12.679463+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:01:11.686000+0000', 'last_change': '2023-11-28T13:01:11.686000+0000', 'last_active': '2023-11-28T13:01:11.677507+0000', 'last_peered': '2023-11-28T13:01:11.677507+0000', 'last_clean': '2023-11-28T13:01:11.677507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:11.686000+0000', 'last_undegraded': '2023-11-28T13:01:11.686000+0000', 'last_fullsized': '2023-11-28T13:01:11.686000+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-11-28T13:01:17.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:01:11.686366+0000', 'last_change': '2023-11-28T13:01:11.686366+0000', 'last_active': '2023-11-28T13:01:11.677507+0000', 'last_peered': '2023-11-28T13:01:11.677507+0000', 'last_clean': '2023-11-28T13:01:11.677507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:01:11.686366+0000', 'last_undegraded': '2023-11-28T13:01:11.686366+0000', 'last_fullsized': '2023-11-28T13:01:11.686366+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'last_clean_scrub_stamp': '2023-11-28T13:01:11.677507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T13:01:17.920 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T13:01:18.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.070+0000 7f0e2f390700 1 Processor -- start 2023-11-28T13:01:18.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.070+0000 7f0e2f390700 1 -- start start 2023-11-28T13:01:18.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2f390700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e28071040 0x7f0e28071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2f390700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e28071950 con 0x7f0e28160540 2023-11-28T13:01:18.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2f390700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2815d060 con 0x7f0e2806dee0 2023-11-28T13:01:18.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2f390700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2815d340 con 0x7f0e28071040 2023-11-28T13:01:18.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2d12c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0e28071040 0x7f0e28071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2d92d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0e28160540 0x7f0e28160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58454/0 (socket says 172.21.15.153:58454) 2023-11-28T13:01:18.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2d92d700 1 -- 172.21.15.153:0/1987097148 learned_addr learned my addr 172.21.15.153:0/1987097148 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:18.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2d12c700 1 -- 172.21.15.153:0/1987097148 >> v1:172.21.15.177:6789/0 conn(0x7f0e2806dee0 legacy=0x7f0e2806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2d12c700 1 -- 172.21.15.153:0/1987097148 >> v1:172.21.15.153:6789/0 conn(0x7f0e28160540 legacy=0x7f0e28160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2d12c700 1 -- 172.21.15.153:0/1987097148 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2815d620 con 0x7f0e28071040 2023-11-28T13:01:18.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e1ffff700 1 -- 172.21.15.153:0/1987097148 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011938231 0 0) 0x7f0e28071950 con 0x7f0e28160540 2023-11-28T13:01:18.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.071+0000 7f0e2d12c700 1 --2- 172.21.15.153:0/1987097148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28071040 0x7f0e28071410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009840 tx=0x7f0e24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdac7aa516a1faaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e1ffff700 1 -- 172.21.15.153:0/1987097148 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24011830 con 0x7f0e28071040 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e1ffff700 1 -- 172.21.15.153:0/1987097148 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24011990 con 0x7f0e28071040 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e1ffff700 1 -- 172.21.15.153:0/1987097148 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24011d20 con 0x7f0e28071040 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e2f390700 1 -- 172.21.15.153:0/1987097148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28071040 msgr2=0x7f0e28071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e2f390700 1 --2- 172.21.15.153:0/1987097148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28071040 0x7f0e28071410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009840 tx=0x7f0e24009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e2f390700 1 -- 172.21.15.153:0/1987097148 shutdown_connections 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e2f390700 1 --2- 172.21.15.153:0/1987097148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28071040 0x7f0e28071410 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e2f390700 1 -- 172.21.15.153:0/1987097148 >> 172.21.15.153:0/1987097148 conn(0x7f0e280d0000 msgr2=0x7f0e280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e2f390700 1 -- 172.21.15.153:0/1987097148 shutdown_connections 2023-11-28T13:01:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.072+0000 7f0e2f390700 1 -- 172.21.15.153:0/1987097148 wait complete. 2023-11-28T13:01:18.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.073+0000 7f0e2f390700 1 Processor -- start 2023-11-28T13:01:18.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.073+0000 7f0e2f390700 1 -- start start 2023-11-28T13:01:18.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2f390700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2806dee0 0x7f0e280c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2f390700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e28071040 0x7f0e280c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2f390700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28160540 0x7f0e281f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2f390700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281f35b0 con 0x7f0e28160540 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2f390700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281f2ff0 con 0x7f0e2806dee0 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2f390700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281f32d0 con 0x7f0e28071040 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2d92d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28160540 0x7f0e281f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2d92d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28160540 0x7f0e281f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40296/0 (socket says 172.21.15.153:40296) 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2d92d700 1 -- 172.21.15.153:0/935686727 learned_addr learned my addr 172.21.15.153:0/935686727 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2d12c700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2806dee0 0x7f0e280c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.074+0000 7f0e2c92b700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e28071040 0x7f0e280c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2d92d700 1 -- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e28071040 msgr2=0x7f0e280c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2d92d700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e28071040 0x7f0e280c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2d92d700 1 -- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2806dee0 msgr2=0x7f0e280c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2d92d700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2806dee0 0x7f0e280c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2d92d700 1 -- 172.21.15.153:0/935686727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240094a0 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2d92d700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28160540 0x7f0e281f2680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000cba0 tx=0x7f0e2000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e1dffb700 1 -- 172.21.15.153:0/935686727 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000aaa0 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e1dffb700 1 -- 172.21.15.153:0/935686727 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000ac20 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2815ccd0 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e1dffb700 1 -- 172.21.15.153:0/935686727 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200070d0 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.075+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2815d900 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.076+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2806a270 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.076+0000 7f0e1dffb700 1 -- 172.21.15.153:0/935686727 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e2001a030 con 0x7f0e28160540 2023-11-28T13:01:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.077+0000 7f0e1dffb700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0e0c0682f0 0x7f0e0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.077+0000 7f0e1dffb700 1 -- 172.21.15.153:0/935686727 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e20085810 con 0x7f0e28160540 2023-11-28T13:01:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.077+0000 7f0e2d12c700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0e0c0682f0 0x7f0e0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.077+0000 7f0e2d12c700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0e0c0682f0 0x7f0e0c06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e240090b0 tx=0x7f0e2401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.081+0000 7f0e1dffb700 1 -- 172.21.15.153:0/935686727 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e20052070 con 0x7f0e28160540 2023-11-28T13:01:18.342 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.340+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0e2806b5c0 con 0x7f0e28160540 2023-11-28T13:01:18.343 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.341+0000 7f0e1dffb700 1 -- 172.21.15.153:0/935686727 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1720 (secure 0 0 0) 0x7f0e20055690 con 0x7f0e28160540 2023-11-28T13:01:18.343 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:18.343 INFO:teuthology.orchestra.run.smithi153.stdout:{"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":85,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1701176403,"num_in_osds":8,"osd_in_since":1701176395,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":94},{"state_name":"unknown","count":8},{"state_name":"creating+peering","count":3}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592698,"bytes_used":71360512,"bytes_avail":773022752768,"bytes_total":773094113280,"unknown_pgs_ratio":0.076190479099750519,"inactive_pgs_ratio":0.02857142873108387,"write_bytes_sec":3200,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5073}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-28T13:01:06.156747+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T13:01:18.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0e0c0682f0 msgr2=0x7f0e0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0e0c0682f0 0x7f0e0c06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e240090b0 tx=0x7f0e2401c040 comp rx=0 tx=0).stop 2023-11-28T13:01:18.345 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28160540 msgr2=0x7f0e281f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28160540 0x7f0e281f2680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000cba0 tx=0x7f0e2000ceb0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 shutdown_connections 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0e0c0682f0 0x7f0e0c06a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e28160540 0x7f0e281f2680 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2806dee0 0x7f0e280c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 --2- 172.21.15.153:0/935686727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e28071040 0x7f0e280c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 >> 172.21.15.153:0/935686727 conn(0x7f0e280d0000 msgr2=0x7f0e2815f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 shutdown_connections 2023-11-28T13:01:18.346 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.343+0000 7f0e2f390700 1 -- 172.21.15.153:0/935686727 wait complete. 2023-11-28T13:01:18.357 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T13:01:18.357 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:01:18.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.476+0000 7feb3197a700 1 Processor -- start 2023-11-28T13:01:18.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.476+0000 7feb3197a700 1 -- start start 2023-11-28T13:01:18.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb3197a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feb2c1658c0 0x7feb2c167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb3197a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c15d060 con 0x7feb2c06dee0 2023-11-28T13:01:18.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb3197a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c15d340 con 0x7feb2c071040 2023-11-28T13:01:18.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb3197a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c161550 con 0x7feb2c1658c0 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb2a7fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7feb2c06dee0 0x7feb2c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58464/0 (socket says 172.21.15.153:58464) 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb2a7fc700 1 -- 172.21.15.153:0/2586627942 learned_addr learned my addr 172.21.15.153:0/2586627942 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869547021 0 0) 0x7feb2c15d060 con 0x7feb2c06dee0 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb140035b0 con 0x7feb2c06dee0 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.477+0000 7feb2b7fe700 1 --2- 172.21.15.153:0/2586627942 >> v2:172.21.15.177:3300/0 conn(0x7feb2c1658c0 0x7feb2c167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3754933617 0 0) 0x7feb140035b0 con 0x7feb2c06dee0 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 >> v2:172.21.15.177:3300/0 conn(0x7feb2c1658c0 msgr2=0x7feb2c167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 --2- 172.21.15.153:0/2586627942 >> v2:172.21.15.177:3300/0 conn(0x7feb2c1658c0 0x7feb2c167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 >> v1:172.21.15.177:6789/0 conn(0x7feb2c071040 legacy=0x7feb2c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c15d620 con 0x7feb2c06dee0 2023-11-28T13:01:18.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7feb20001670 con 0x7feb2c06dee0 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 >> v1:172.21.15.153:6789/0 conn(0x7feb2c06dee0 legacy=0x7feb2c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14003a00 0x7feb14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb14004850 0x7feb14004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb14005400 0x7feb1400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c161550 con 0x7feb14004850 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1400c280 con 0x7feb14003a00 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1400c440 con 0x7feb14005400 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2a7fc700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb14004850 0x7feb14004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2affd700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14003a00 0x7feb14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2b7fe700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb14005400 0x7feb1400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2a7fc700 1 -- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb14005400 msgr2=0x7feb1400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2a7fc700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb14005400 0x7feb1400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2a7fc700 1 -- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14003a00 msgr2=0x7feb14003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2a7fc700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14003a00 0x7feb14003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2a7fc700 1 -- 172.21.15.153:0/2586627942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb140035b0 con 0x7feb14004850 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2b7fe700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb14005400 0x7feb1400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2b7fe700 1 -- 172.21.15.153:0/2586627942 reap_dead start 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb2a7fc700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb14004850 0x7feb14004cc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c240 tx=0x7feb14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76c79dfc7bc4836 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20007c10 con 0x7feb14004850 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2000cde0 con 0x7feb14004850 2023-11-28T13:01:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.478+0000 7feb29ffb700 1 -- 172.21.15.153:0/2586627942 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20005ba0 con 0x7feb14004850 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.479+0000 7feb3197a700 1 -- 172.21.15.153:0/2586627942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb14004850 msgr2=0x7feb14004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.479+0000 7feb3197a700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb14004850 0x7feb14004cc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c240 tx=0x7feb14005880 comp rx=0 tx=0).stop 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.479+0000 7feb3197a700 1 -- 172.21.15.153:0/2586627942 shutdown_connections 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.479+0000 7feb3197a700 1 --2- 172.21.15.153:0/2586627942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb14004850 0x7feb14004cc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.479+0000 7feb3197a700 1 -- 172.21.15.153:0/2586627942 >> 172.21.15.153:0/2586627942 conn(0x7feb2c0d0000 msgr2=0x7feb2c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.479+0000 7feb3197a700 1 -- 172.21.15.153:0/2586627942 shutdown_connections 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.479+0000 7feb3197a700 1 -- 172.21.15.153:0/2586627942 wait complete. 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.480+0000 7feb3197a700 1 Processor -- start 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.480+0000 7feb3197a700 1 -- start start 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb3197a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14004850 0x7feb2c0c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb3197a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2c1510d0 0x7feb2c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb3197a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb2c15e360 0x7feb2c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb3197a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c155470 con 0x7feb2c1510d0 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb3197a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0ae870 con 0x7feb14004850 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb3197a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c155c40 con 0x7feb2c15e360 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb2a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2c1510d0 0x7feb2c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb2a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2c1510d0 0x7feb2c1570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40314/0 (socket says 172.21.15.153:40314) 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb2a7fc700 1 -- 172.21.15.153:0/1169932576 learned_addr learned my addr 172.21.15.153:0/1169932576 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb2affd700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14004850 0x7feb2c0c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.481+0000 7feb2b7fe700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb2c15e360 0x7feb2c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb2a7fc700 1 -- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb2c15e360 msgr2=0x7feb2c150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb2a7fc700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb2c15e360 0x7feb2c150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb2a7fc700 1 -- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14004850 msgr2=0x7feb2c0c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb2a7fc700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14004850 0x7feb2c0c3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb2a7fc700 1 -- 172.21.15.153:0/1169932576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c15ccd0 con 0x7feb2c1510d0 2023-11-28T13:01:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb2a7fc700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2c1510d0 0x7feb2c1570e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c250 tx=0x7feb20001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb13fff700 1 -- 172.21.15.153:0/1169932576 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20001630 con 0x7feb2c1510d0 2023-11-28T13:01:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb13fff700 1 -- 172.21.15.153:0/1169932576 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200070d0 con 0x7feb2c1510d0 2023-11-28T13:01:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb13fff700 1 -- 172.21.15.153:0/1169932576 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20007460 con 0x7feb2c1510d0 2023-11-28T13:01:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c15d900 con 0x7feb2c1510d0 2023-11-28T13:01:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.482+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c06a270 con 0x7feb2c1510d0 2023-11-28T13:01:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.483+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c0ae6d0 con 0x7feb2c1510d0 2023-11-28T13:01:18.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.483+0000 7feb13fff700 1 -- 172.21.15.153:0/1169932576 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb20024030 con 0x7feb2c1510d0 2023-11-28T13:01:18.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.484+0000 7feb13fff700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feb180682f0 0x7feb1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.484+0000 7feb13fff700 1 -- 172.21.15.153:0/1169932576 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb20087060 con 0x7feb2c1510d0 2023-11-28T13:01:18.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.484+0000 7feb2affd700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feb180682f0 0x7feb1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.484+0000 7feb2affd700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feb180682f0 0x7feb1806a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000c00 tx=0x7feb1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.488+0000 7feb13fff700 1 -- 172.21.15.153:0/1169932576 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb200b85e0 con 0x7feb2c1510d0 2023-11-28T13:01:18.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.670+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb2c155470 con 0x7feb180682f0 2023-11-28T13:01:18.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.678+0000 7feb13fff700 1 -- 172.21.15.153:0/1169932576 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311089 (secure 0 0 0) 0x7feb2c155470 con 0x7feb180682f0 2023-11-28T13:01:18.684 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:18.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feb180682f0 msgr2=0x7feb1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feb180682f0 0x7feb1806a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000c00 tx=0x7feb1c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:18.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2c1510d0 msgr2=0x7feb2c1570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2c1510d0 0x7feb2c1570e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c250 tx=0x7feb20001a00 comp rx=0 tx=0).stop 2023-11-28T13:01:18.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 shutdown_connections 2023-11-28T13:01:18.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb2c15e360 0x7feb2c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb14004850 0x7feb2c0c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feb180682f0 0x7feb1806a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 --2- 172.21.15.153:0/1169932576 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb2c1510d0 0x7feb2c1570e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.681+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 >> 172.21.15.153:0/1169932576 conn(0x7feb2c0d0000 msgr2=0x7feb2c15f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.682+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 shutdown_connections 2023-11-28T13:01:18.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:18.682+0000 7feb3197a700 1 -- 172.21.15.153:0/1169932576 wait complete. 2023-11-28T13:01:18.687 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:01:18.695 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-11-28T13:01:18.158757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69856,"kb_used_data":4512,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":754904864,"statfs":{"total":773094113280,"available":773022580736,"internally_reserved":0,"allocated":4620288,"data_stored":2582787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001023"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090821+0000","last_change":"2023-11-28T13:01:13.094545+0000","last_active":"2023-11-28T13:01:14.090821+0000","last_peered":"2023-11-28T13:01:14.090821+0000","last_clean":"2023-11-28T13:01:14.090821+0000","last_became_active":"2023-11-28T13:01:13.094296+0000","last_became_peered":"2023-11-28T13:01:13.094296+0000","last_unstale":"2023-11-28T13:01:14.090821+0000","last_undegraded":"2023-11-28T13:01:14.090821+0000","last_fullsized":"2023-11-28T13:01:14.090821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:18.727519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090776+0000","last_change":"2023-11-28T13:01:13.088855+0000","last_active":"2023-11-28T13:01:14.090776+0000","last_peered":"2023-11-28T13:01:14.090776+0000","last_clean":"2023-11-28T13:01:14.090776+0000","last_became_active":"2023-11-28T13:01:13.088631+0000","last_became_peered":"2023-11-28T13:01:13.088631+0000","last_unstale":"2023-11-28T13:01:14.090776+0000","last_undegraded":"2023-11-28T13:01:14.090776+0000","last_fullsized":"2023-11-28T13:01:14.090776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:42.143519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325409+0000","last_change":"2023-11-28T13:01:13.103414+0000","last_active":"2023-11-28T13:01:14.325409+0000","last_peered":"2023-11-28T13:01:14.325409+0000","last_clean":"2023-11-28T13:01:14.325409+0000","last_became_active":"2023-11-28T13:01:13.103175+0000","last_became_peered":"2023-11-28T13:01:13.103175+0000","last_unstale":"2023-11-28T13:01:14.325409+0000","last_undegraded":"2023-11-28T13:01:14.325409+0000","last_fullsized":"2023-11-28T13:01:14.325409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107263+0000","last_change":"2023-11-28T13:01:13.102065+0000","last_active":"2023-11-28T13:01:13.107263+0000","last_peered":"2023-11-28T13:01:13.107263+0000","last_clean":"2023-11-28T13:01:13.107263+0000","last_became_active":"2023-11-28T13:01:13.101822+0000","last_became_peered":"2023-11-28T13:01:13.101822+0000","last_unstale":"2023-11-28T13:01:13.107263+0000","last_undegraded":"2023-11-28T13:01:13.107263+0000","last_fullsized":"2023-11-28T13:01:13.107263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:51.732790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.099900+0000","last_change":"2023-11-28T13:01:13.095907+0000","last_active":"2023-11-28T13:01:13.099900+0000","last_peered":"2023-11-28T13:01:13.099900+0000","last_clean":"2023-11-28T13:01:13.099900+0000","last_became_active":"2023-11-28T13:01:13.095190+0000","last_became_peered":"2023-11-28T13:01:13.095190+0000","last_unstale":"2023-11-28T13:01:13.099900+0000","last_undegraded":"2023-11-28T13:01:13.099900+0000","last_fullsized":"2023-11-28T13:01:13.099900+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:34.713740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088925+0000","last_change":"2023-11-28T13:01:13.103245+0000","last_active":"2023-11-28T13:01:14.088925+0000","last_peered":"2023-11-28T13:01:14.088925+0000","last_clean":"2023-11-28T13:01:14.088925+0000","last_became_active":"2023-11-28T13:01:13.102920+0000","last_became_peered":"2023-11-28T13:01:13.102920+0000","last_unstale":"2023-11-28T13:01:14.088925+0000","last_undegraded":"2023-11-28T13:01:14.088925+0000","last_fullsized":"2023-11-28T13:01:14.088925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:23.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106870+0000","last_change":"2023-11-28T13:01:13.101383+0000","last_active":"2023-11-28T13:01:13.106870+0000","last_peered":"2023-11-28T13:01:13.106870+0000","last_clean":"2023-11-28T13:01:13.106870+0000","last_became_active":"2023-11-28T13:01:13.101093+0000","last_became_peered":"2023-11-28T13:01:13.101093+0000","last_unstale":"2023-11-28T13:01:13.106870+0000","last_undegraded":"2023-11-28T13:01:13.106870+0000","last_fullsized":"2023-11-28T13:01:13.106870+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090405+0000","last_change":"2023-11-28T13:01:13.093176+0000","last_active":"2023-11-28T13:01:14.090405+0000","last_peered":"2023-11-28T13:01:14.090405+0000","last_clean":"2023-11-28T13:01:14.090405+0000","last_became_active":"2023-11-28T13:01:13.092922+0000","last_became_peered":"2023-11-28T13:01:13.092922+0000","last_unstale":"2023-11-28T13:01:14.090405+0000","last_undegraded":"2023-11-28T13:01:14.090405+0000","last_fullsized":"2023-11-28T13:01:14.090405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:31.712364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325497+0000","last_change":"2023-11-28T13:01:13.105535+0000","last_active":"2023-11-28T13:01:14.325497+0000","last_peered":"2023-11-28T13:01:14.325497+0000","last_clean":"2023-11-28T13:01:14.325497+0000","last_became_active":"2023-11-28T13:01:13.105306+0000","last_became_peered":"2023-11-28T13:01:13.105306+0000","last_unstale":"2023-11-28T13:01:14.325497+0000","last_undegraded":"2023-11-28T13:01:14.325497+0000","last_fullsized":"2023-11-28T13:01:14.325497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102138+0000","last_change":"2023-11-28T13:01:13.090026+0000","last_active":"2023-11-28T13:01:13.102138+0000","last_peered":"2023-11-28T13:01:13.102138+0000","last_clean":"2023-11-28T13:01:13.102138+0000","last_became_active":"2023-11-28T13:01:13.089734+0000","last_became_peered":"2023-11-28T13:01:13.089734+0000","last_unstale":"2023-11-28T13:01:13.102138+0000","last_undegraded":"2023-11-28T13:01:13.102138+0000","last_fullsized":"2023-11-28T13:01:13.102138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110483+0000","last_change":"2023-11-28T13:01:13.106317+0000","last_active":"2023-11-28T13:01:13.110483+0000","last_peered":"2023-11-28T13:01:13.110483+0000","last_clean":"2023-11-28T13:01:13.110483+0000","last_became_active":"2023-11-28T13:01:13.106096+0000","last_became_peered":"2023-11-28T13:01:13.106096+0000","last_unstale":"2023-11-28T13:01:13.110483+0000","last_undegraded":"2023-11-28T13:01:13.110483+0000","last_fullsized":"2023-11-28T13:01:13.110483+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325360+0000","last_change":"2023-11-28T13:01:13.095524+0000","last_active":"2023-11-28T13:01:14.325360+0000","last_peered":"2023-11-28T13:01:14.325360+0000","last_clean":"2023-11-28T13:01:14.325360+0000","last_became_active":"2023-11-28T13:01:13.095284+0000","last_became_peered":"2023-11-28T13:01:13.095284+0000","last_unstale":"2023-11-28T13:01:14.325360+0000","last_undegraded":"2023-11-28T13:01:14.325360+0000","last_fullsized":"2023-11-28T13:01:14.325360+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:51.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102639+0000","last_change":"2023-11-28T13:01:13.098543+0000","last_active":"2023-11-28T13:01:13.102639+0000","last_peered":"2023-11-28T13:01:13.102639+0000","last_clean":"2023-11-28T13:01:13.102639+0000","last_became_active":"2023-11-28T13:01:13.097626+0000","last_became_peered":"2023-11-28T13:01:13.097626+0000","last_unstale":"2023-11-28T13:01:13.102639+0000","last_undegraded":"2023-11-28T13:01:13.102639+0000","last_fullsized":"2023-11-28T13:01:13.102639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104846+0000","last_change":"2023-11-28T13:01:13.097007+0000","last_active":"2023-11-28T13:01:13.104846+0000","last_peered":"2023-11-28T13:01:13.104846+0000","last_clean":"2023-11-28T13:01:13.104846+0000","last_became_active":"2023-11-28T13:01:13.096697+0000","last_became_peered":"2023-11-28T13:01:13.096697+0000","last_unstale":"2023-11-28T13:01:13.104846+0000","last_undegraded":"2023-11-28T13:01:13.104846+0000","last_fullsized":"2023-11-28T13:01:13.104846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102519+0000","last_change":"2023-11-28T13:01:13.092555+0000","last_active":"2023-11-28T13:01:13.102519+0000","last_peered":"2023-11-28T13:01:13.102519+0000","last_clean":"2023-11-28T13:01:13.102519+0000","last_became_active":"2023-11-28T13:01:13.092269+0000","last_became_peered":"2023-11-28T13:01:13.092269+0000","last_unstale":"2023-11-28T13:01:13.102519+0000","last_undegraded":"2023-11-28T13:01:13.102519+0000","last_fullsized":"2023-11-28T13:01:13.102519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110258+0000","last_change":"2023-11-28T13:01:13.106928+0000","last_active":"2023-11-28T13:01:13.110258+0000","last_peered":"2023-11-28T13:01:13.110258+0000","last_clean":"2023-11-28T13:01:13.110258+0000","last_became_active":"2023-11-28T13:01:13.104168+0000","last_became_peered":"2023-11-28T13:01:13.104168+0000","last_unstale":"2023-11-28T13:01:13.110258+0000","last_undegraded":"2023-11-28T13:01:13.110258+0000","last_fullsized":"2023-11-28T13:01:13.110258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110271+0000","last_change":"2023-11-28T13:01:13.106957+0000","last_active":"2023-11-28T13:01:13.110271+0000","last_peered":"2023-11-28T13:01:13.110271+0000","last_clean":"2023-11-28T13:01:13.110271+0000","last_became_active":"2023-11-28T13:01:13.104842+0000","last_became_peered":"2023-11-28T13:01:13.104842+0000","last_unstale":"2023-11-28T13:01:13.110271+0000","last_undegraded":"2023-11-28T13:01:13.110271+0000","last_fullsized":"2023-11-28T13:01:13.110271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105656+0000","last_change":"2023-11-28T13:01:13.102950+0000","last_active":"2023-11-28T13:01:13.105656+0000","last_peered":"2023-11-28T13:01:13.105656+0000","last_clean":"2023-11-28T13:01:13.105656+0000","last_became_active":"2023-11-28T13:01:13.102706+0000","last_became_peered":"2023-11-28T13:01:13.102706+0000","last_unstale":"2023-11-28T13:01:13.105656+0000","last_undegraded":"2023-11-28T13:01:13.105656+0000","last_fullsized":"2023-11-28T13:01:13.105656+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:33.362111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.097705+0000","last_change":"2023-11-28T13:01:13.094603+0000","last_active":"2023-11-28T13:01:13.097705+0000","last_peered":"2023-11-28T13:01:13.097705+0000","last_clean":"2023-11-28T13:01:13.097705+0000","last_became_active":"2023-11-28T13:01:13.094377+0000","last_became_peered":"2023-11-28T13:01:13.094377+0000","last_unstale":"2023-11-28T13:01:13.097705+0000","last_undegraded":"2023-11-28T13:01:13.097705+0000","last_fullsized":"2023-11-28T13:01:13.097705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:25.095910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098253+0000","last_change":"2023-11-28T13:01:13.093425+0000","last_active":"2023-11-28T13:01:13.098253+0000","last_peered":"2023-11-28T13:01:13.098253+0000","last_clean":"2023-11-28T13:01:13.098253+0000","last_became_active":"2023-11-28T13:01:13.093128+0000","last_became_peered":"2023-11-28T13:01:13.093128+0000","last_unstale":"2023-11-28T13:01:13.098253+0000","last_undegraded":"2023-11-28T13:01:13.098253+0000","last_fullsized":"2023-11-28T13:01:13.098253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:47.380562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102465+0000","last_change":"2023-11-28T13:01:13.095333+0000","last_active":"2023-11-28T13:01:13.102465+0000","last_peered":"2023-11-28T13:01:13.102465+0000","last_clean":"2023-11-28T13:01:13.102465+0000","last_became_active":"2023-11-28T13:01:13.095050+0000","last_became_peered":"2023-11-28T13:01:13.095050+0000","last_unstale":"2023-11-28T13:01:13.102465+0000","last_undegraded":"2023-11-28T13:01:13.102465+0000","last_fullsized":"2023-11-28T13:01:13.102465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:51:32.434761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107125+0000","last_change":"2023-11-28T13:01:13.105213+0000","last_active":"2023-11-28T13:01:13.107125+0000","last_peered":"2023-11-28T13:01:13.107125+0000","last_clean":"2023-11-28T13:01:13.107125+0000","last_became_active":"2023-11-28T13:01:13.104960+0000","last_became_peered":"2023-11-28T13:01:13.104960+0000","last_unstale":"2023-11-28T13:01:13.107125+0000","last_undegraded":"2023-11-28T13:01:13.107125+0000","last_fullsized":"2023-11-28T13:01:13.107125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110113+0000","last_change":"2023-11-28T13:01:13.102193+0000","last_active":"2023-11-28T13:01:13.110113+0000","last_peered":"2023-11-28T13:01:13.110113+0000","last_clean":"2023-11-28T13:01:13.110113+0000","last_became_active":"2023-11-28T13:01:13.101759+0000","last_became_peered":"2023-11-28T13:01:13.101759+0000","last_unstale":"2023-11-28T13:01:13.110113+0000","last_undegraded":"2023-11-28T13:01:13.110113+0000","last_fullsized":"2023-11-28T13:01:13.110113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:51.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103015+0000","last_change":"2023-11-28T13:01:13.098378+0000","last_active":"2023-11-28T13:01:13.103015+0000","last_peered":"2023-11-28T13:01:13.103015+0000","last_clean":"2023-11-28T13:01:13.103015+0000","last_became_active":"2023-11-28T13:01:13.098101+0000","last_became_peered":"2023-11-28T13:01:13.098101+0000","last_unstale":"2023-11-28T13:01:13.103015+0000","last_undegraded":"2023-11-28T13:01:13.103015+0000","last_fullsized":"2023-11-28T13:01:13.103015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:11.877653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098316+0000","last_change":"2023-11-28T13:01:13.094017+0000","last_active":"2023-11-28T13:01:13.098316+0000","last_peered":"2023-11-28T13:01:13.098316+0000","last_clean":"2023-11-28T13:01:13.098316+0000","last_became_active":"2023-11-28T13:01:13.093753+0000","last_became_peered":"2023-11-28T13:01:13.093753+0000","last_unstale":"2023-11-28T13:01:13.098316+0000","last_undegraded":"2023-11-28T13:01:13.098316+0000","last_fullsized":"2023-11-28T13:01:13.098316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:21.524121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089155+0000","last_change":"2023-11-28T13:01:13.103430+0000","last_active":"2023-11-28T13:01:14.089155+0000","last_peered":"2023-11-28T13:01:14.089155+0000","last_clean":"2023-11-28T13:01:14.089155+0000","last_became_active":"2023-11-28T13:01:13.102784+0000","last_became_peered":"2023-11-28T13:01:13.102784+0000","last_unstale":"2023-11-28T13:01:14.089155+0000","last_undegraded":"2023-11-28T13:01:14.089155+0000","last_fullsized":"2023-11-28T13:01:14.089155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:47.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098350+0000","last_change":"2023-11-28T13:01:13.092484+0000","last_active":"2023-11-28T13:01:13.098350+0000","last_peered":"2023-11-28T13:01:13.098350+0000","last_clean":"2023-11-28T13:01:13.098350+0000","last_became_active":"2023-11-28T13:01:13.092162+0000","last_became_peered":"2023-11-28T13:01:13.092162+0000","last_unstale":"2023-11-28T13:01:13.098350+0000","last_undegraded":"2023-11-28T13:01:13.098350+0000","last_fullsized":"2023-11-28T13:01:13.098350+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:03.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107152+0000","last_change":"2023-11-28T13:01:13.100818+0000","last_active":"2023-11-28T13:01:13.107152+0000","last_peered":"2023-11-28T13:01:13.107152+0000","last_clean":"2023-11-28T13:01:13.107152+0000","last_became_active":"2023-11-28T13:01:13.100421+0000","last_became_peered":"2023-11-28T13:01:13.100421+0000","last_unstale":"2023-11-28T13:01:13.107152+0000","last_undegraded":"2023-11-28T13:01:13.107152+0000","last_fullsized":"2023-11-28T13:01:13.107152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098210+0000","last_change":"2023-11-28T13:01:13.093478+0000","last_active":"2023-11-28T13:01:13.098210+0000","last_peered":"2023-11-28T13:01:13.098210+0000","last_clean":"2023-11-28T13:01:13.098210+0000","last_became_active":"2023-11-28T13:01:13.093194+0000","last_became_peered":"2023-11-28T13:01:13.093194+0000","last_unstale":"2023-11-28T13:01:13.098210+0000","last_undegraded":"2023-11-28T13:01:13.098210+0000","last_fullsized":"2023-11-28T13:01:13.098210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:10.435728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.098142+0000","last_change":"2023-11-28T13:01:13.088250+0000","last_active":"2023-11-28T13:01:13.098142+0000","last_peered":"2023-11-28T13:01:13.098142+0000","last_clean":"2023-11-28T13:01:13.098142+0000","last_became_active":"2023-11-28T13:01:13.087881+0000","last_became_peered":"2023-11-28T13:01:13.087881+0000","last_unstale":"2023-11-28T13:01:13.098142+0000","last_undegraded":"2023-11-28T13:01:13.098142+0000","last_fullsized":"2023-11-28T13:01:13.098142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:43.126488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110058+0000","last_change":"2023-11-28T13:01:13.102030+0000","last_active":"2023-11-28T13:01:13.110058+0000","last_peered":"2023-11-28T13:01:13.110058+0000","last_clean":"2023-11-28T13:01:13.110058+0000","last_became_active":"2023-11-28T13:01:13.101376+0000","last_became_peered":"2023-11-28T13:01:13.101376+0000","last_unstale":"2023-11-28T13:01:13.110058+0000","last_undegraded":"2023-11-28T13:01:13.110058+0000","last_fullsized":"2023-11-28T13:01:13.110058+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088757+0000","last_change":"2023-11-28T13:01:13.095736+0000","last_active":"2023-11-28T13:01:14.088757+0000","last_peered":"2023-11-28T13:01:14.088757+0000","last_clean":"2023-11-28T13:01:14.088757+0000","last_became_active":"2023-11-28T13:01:13.095469+0000","last_became_peered":"2023-11-28T13:01:13.095469+0000","last_unstale":"2023-11-28T13:01:14.088757+0000","last_undegraded":"2023-11-28T13:01:14.088757+0000","last_fullsized":"2023-11-28T13:01:14.088757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:45.817559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110226+0000","last_change":"2023-11-28T13:01:13.105146+0000","last_active":"2023-11-28T13:01:13.110226+0000","last_peered":"2023-11-28T13:01:13.110226+0000","last_clean":"2023-11-28T13:01:13.110226+0000","last_became_active":"2023-11-28T13:01:13.104852+0000","last_became_peered":"2023-11-28T13:01:13.104852+0000","last_unstale":"2023-11-28T13:01:13.110226+0000","last_undegraded":"2023-11-28T13:01:13.110226+0000","last_fullsized":"2023-11-28T13:01:13.110226+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104391+0000","last_change":"2023-11-28T13:01:12.697633+0000","last_active":"2023-11-28T13:01:13.104391+0000","last_peered":"2023-11-28T13:01:13.104391+0000","last_clean":"2023-11-28T13:01:13.104391+0000","last_became_active":"2023-11-28T13:01:12.697137+0000","last_became_peered":"2023-11-28T13:01:12.697137+0000","last_unstale":"2023-11-28T13:01:13.104391+0000","last_undegraded":"2023-11-28T13:01:13.104391+0000","last_fullsized":"2023-11-28T13:01:13.104391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:59.601679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089117+0000","last_change":"2023-11-28T13:01:13.104026+0000","last_active":"2023-11-28T13:01:14.089117+0000","last_peered":"2023-11-28T13:01:14.089117+0000","last_clean":"2023-11-28T13:01:14.089117+0000","last_became_active":"2023-11-28T13:01:13.103779+0000","last_became_peered":"2023-11-28T13:01:13.103779+0000","last_unstale":"2023-11-28T13:01:14.089117+0000","last_undegraded":"2023-11-28T13:01:14.089117+0000","last_fullsized":"2023-11-28T13:01:14.089117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:13.669167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.096308+0000","last_change":"2023-11-28T13:01:12.700381+0000","last_active":"2023-11-28T13:01:13.096308+0000","last_peered":"2023-11-28T13:01:13.096308+0000","last_clean":"2023-11-28T13:01:13.096308+0000","last_became_active":"2023-11-28T13:01:12.700088+0000","last_became_peered":"2023-11-28T13:01:12.700088+0000","last_unstale":"2023-11-28T13:01:13.096308+0000","last_undegraded":"2023-11-28T13:01:13.096308+0000","last_fullsized":"2023-11-28T13:01:13.096308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107203+0000","last_change":"2023-11-28T13:01:13.104225+0000","last_active":"2023-11-28T13:01:13.107203+0000","last_peered":"2023-11-28T13:01:13.107203+0000","last_clean":"2023-11-28T13:01:13.107203+0000","last_became_active":"2023-11-28T13:01:13.103835+0000","last_became_peered":"2023-11-28T13:01:13.103835+0000","last_unstale":"2023-11-28T13:01:13.107203+0000","last_undegraded":"2023-11-28T13:01:13.107203+0000","last_fullsized":"2023-11-28T13:01:13.107203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089210+0000","last_change":"2023-11-28T13:01:12.703691+0000","last_active":"2023-11-28T13:01:14.089210+0000","last_peered":"2023-11-28T13:01:14.089210+0000","last_clean":"2023-11-28T13:01:14.089210+0000","last_became_active":"2023-11-28T13:01:12.703398+0000","last_became_peered":"2023-11-28T13:01:12.703398+0000","last_unstale":"2023-11-28T13:01:14.089210+0000","last_undegraded":"2023-11-28T13:01:14.089210+0000","last_fullsized":"2023-11-28T13:01:14.089210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:24.093954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090818+0000","last_change":"2023-11-28T13:01:13.086251+0000","last_active":"2023-11-28T13:01:14.090818+0000","last_peered":"2023-11-28T13:01:14.090818+0000","last_clean":"2023-11-28T13:01:14.090818+0000","last_became_active":"2023-11-28T13:01:13.085269+0000","last_became_peered":"2023-11-28T13:01:13.085269+0000","last_unstale":"2023-11-28T13:01:14.090818+0000","last_undegraded":"2023-11-28T13:01:14.090818+0000","last_fullsized":"2023-11-28T13:01:14.090818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:04.381796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.103953+0000","last_change":"2023-11-28T13:01:12.700070+0000","last_active":"2023-11-28T13:01:13.103953+0000","last_peered":"2023-11-28T13:01:13.103953+0000","last_clean":"2023-11-28T13:01:13.103953+0000","last_became_active":"2023-11-28T13:01:12.699788+0000","last_became_peered":"2023-11-28T13:01:12.699788+0000","last_unstale":"2023-11-28T13:01:13.103953+0000","last_undegraded":"2023-11-28T13:01:13.103953+0000","last_fullsized":"2023-11-28T13:01:13.103953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:37.142758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325299+0000","last_change":"2023-11-28T13:01:13.093841+0000","last_active":"2023-11-28T13:01:14.325299+0000","last_peered":"2023-11-28T13:01:14.325299+0000","last_clean":"2023-11-28T13:01:14.325299+0000","last_became_active":"2023-11-28T13:01:13.093572+0000","last_became_peered":"2023-11-28T13:01:13.093572+0000","last_unstale":"2023-11-28T13:01:14.325299+0000","last_undegraded":"2023-11-28T13:01:14.325299+0000","last_fullsized":"2023-11-28T13:01:14.325299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102494+0000","last_change":"2023-11-28T13:01:12.700665+0000","last_active":"2023-11-28T13:01:13.102494+0000","last_peered":"2023-11-28T13:01:13.102494+0000","last_clean":"2023-11-28T13:01:13.102494+0000","last_became_active":"2023-11-28T13:01:12.700285+0000","last_became_peered":"2023-11-28T13:01:12.700285+0000","last_unstale":"2023-11-28T13:01:13.102494+0000","last_undegraded":"2023-11-28T13:01:13.102494+0000","last_fullsized":"2023-11-28T13:01:13.102494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102539+0000","last_change":"2023-11-28T13:01:13.095159+0000","last_active":"2023-11-28T13:01:13.102539+0000","last_peered":"2023-11-28T13:01:13.102539+0000","last_clean":"2023-11-28T13:01:13.102539+0000","last_became_active":"2023-11-28T13:01:13.094857+0000","last_became_peered":"2023-11-28T13:01:13.094857+0000","last_unstale":"2023-11-28T13:01:13.102539+0000","last_undegraded":"2023-11-28T13:01:13.102539+0000","last_fullsized":"2023-11-28T13:01:13.102539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106011+0000","last_change":"2023-11-28T13:01:12.705296+0000","last_active":"2023-11-28T13:01:13.106011+0000","last_peered":"2023-11-28T13:01:13.106011+0000","last_clean":"2023-11-28T13:01:13.106011+0000","last_became_active":"2023-11-28T13:01:12.705013+0000","last_became_peered":"2023-11-28T13:01:12.705013+0000","last_unstale":"2023-11-28T13:01:13.106011+0000","last_undegraded":"2023-11-28T13:01:13.106011+0000","last_fullsized":"2023-11-28T13:01:13.106011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:46.378606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104868+0000","last_change":"2023-11-28T13:01:13.094747+0000","last_active":"2023-11-28T13:01:13.104868+0000","last_peered":"2023-11-28T13:01:13.104868+0000","last_clean":"2023-11-28T13:01:13.104868+0000","last_became_active":"2023-11-28T13:01:13.094318+0000","last_became_peered":"2023-11-28T13:01:13.094318+0000","last_unstale":"2023-11-28T13:01:13.104868+0000","last_undegraded":"2023-11-28T13:01:13.104868+0000","last_fullsized":"2023-11-28T13:01:13.104868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:27.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089212+0000","last_change":"2023-11-28T13:01:12.706058+0000","last_active":"2023-11-28T13:01:14.089212+0000","last_peered":"2023-11-28T13:01:14.089212+0000","last_clean":"2023-11-28T13:01:14.089212+0000","last_became_active":"2023-11-28T13:01:12.705853+0000","last_became_peered":"2023-11-28T13:01:12.705853+0000","last_unstale":"2023-11-28T13:01:14.089212+0000","last_undegraded":"2023-11-28T13:01:14.089212+0000","last_fullsized":"2023-11-28T13:01:14.089212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:10.875697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090633+0000","last_change":"2023-11-28T13:01:13.099221+0000","last_active":"2023-11-28T13:01:14.090633+0000","last_peered":"2023-11-28T13:01:14.090633+0000","last_clean":"2023-11-28T13:01:14.090633+0000","last_became_active":"2023-11-28T13:01:13.098525+0000","last_became_peered":"2023-11-28T13:01:13.098525+0000","last_unstale":"2023-11-28T13:01:14.090633+0000","last_undegraded":"2023-11-28T13:01:14.090633+0000","last_fullsized":"2023-11-28T13:01:14.090633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:01.070045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092473+0000","last_change":"2023-11-28T13:01:12.700642+0000","last_active":"2023-11-28T13:01:13.092473+0000","last_peered":"2023-11-28T13:01:13.092473+0000","last_clean":"2023-11-28T13:01:13.092473+0000","last_became_active":"2023-11-28T13:01:12.700381+0000","last_became_peered":"2023-11-28T13:01:12.700381+0000","last_unstale":"2023-11-28T13:01:13.092473+0000","last_undegraded":"2023-11-28T13:01:13.092473+0000","last_fullsized":"2023-11-28T13:01:13.092473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:37.268919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110533+0000","last_change":"2023-11-28T13:01:13.107217+0000","last_active":"2023-11-28T13:01:13.110533+0000","last_peered":"2023-11-28T13:01:13.110533+0000","last_clean":"2023-11-28T13:01:13.110533+0000","last_became_active":"2023-11-28T13:01:13.106967+0000","last_became_peered":"2023-11-28T13:01:13.106967+0000","last_unstale":"2023-11-28T13:01:13.110533+0000","last_undegraded":"2023-11-28T13:01:13.110533+0000","last_fullsized":"2023-11-28T13:01:13.110533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:01.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090884+0000","last_change":"2023-11-28T13:01:12.700536+0000","last_active":"2023-11-28T13:01:14.090884+0000","last_peered":"2023-11-28T13:01:14.090884+0000","last_clean":"2023-11-28T13:01:14.090884+0000","last_became_active":"2023-11-28T13:01:12.699782+0000","last_became_peered":"2023-11-28T13:01:12.699782+0000","last_unstale":"2023-11-28T13:01:14.090884+0000","last_undegraded":"2023-11-28T13:01:14.090884+0000","last_fullsized":"2023-11-28T13:01:14.090884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:17.146583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110165+0000","last_change":"2023-11-28T13:01:13.103462+0000","last_active":"2023-11-28T13:01:13.110165+0000","last_peered":"2023-11-28T13:01:13.110165+0000","last_clean":"2023-11-28T13:01:13.110165+0000","last_became_active":"2023-11-28T13:01:13.102855+0000","last_became_peered":"2023-11-28T13:01:13.102855+0000","last_unstale":"2023-11-28T13:01:13.110165+0000","last_undegraded":"2023-11-28T13:01:13.110165+0000","last_fullsized":"2023-11-28T13:01:13.110165+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.987104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092356+0000","last_change":"2023-11-28T13:01:12.703891+0000","last_active":"2023-11-28T13:01:13.092356+0000","last_peered":"2023-11-28T13:01:13.092356+0000","last_clean":"2023-11-28T13:01:13.092356+0000","last_became_active":"2023-11-28T13:01:12.703642+0000","last_became_peered":"2023-11-28T13:01:12.703642+0000","last_unstale":"2023-11-28T13:01:13.092356+0000","last_undegraded":"2023-11-28T13:01:13.092356+0000","last_fullsized":"2023-11-28T13:01:13.092356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:32.360155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325431+0000","last_change":"2023-11-28T13:01:13.096906+0000","last_active":"2023-11-28T13:01:14.325431+0000","last_peered":"2023-11-28T13:01:14.325431+0000","last_clean":"2023-11-28T13:01:14.325431+0000","last_became_active":"2023-11-28T13:01:13.096606+0000","last_became_peered":"2023-11-28T13:01:13.096606+0000","last_unstale":"2023-11-28T13:01:14.325431+0000","last_undegraded":"2023-11-28T13:01:14.325431+0000","last_fullsized":"2023-11-28T13:01:14.325431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105022+0000","last_change":"2023-11-28T13:01:12.704636+0000","last_active":"2023-11-28T13:01:13.105022+0000","last_peered":"2023-11-28T13:01:13.105022+0000","last_clean":"2023-11-28T13:01:13.105022+0000","last_became_active":"2023-11-28T13:01:12.704357+0000","last_became_peered":"2023-11-28T13:01:12.704357+0000","last_unstale":"2023-11-28T13:01:13.105022+0000","last_undegraded":"2023-11-28T13:01:13.105022+0000","last_fullsized":"2023-11-28T13:01:13.105022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105277+0000","last_change":"2023-11-28T13:01:13.096971+0000","last_active":"2023-11-28T13:01:13.105277+0000","last_peered":"2023-11-28T13:01:13.105277+0000","last_clean":"2023-11-28T13:01:13.105277+0000","last_became_active":"2023-11-28T13:01:13.096698+0000","last_became_peered":"2023-11-28T13:01:13.096698+0000","last_unstale":"2023-11-28T13:01:13.105277+0000","last_undegraded":"2023-11-28T13:01:13.105277+0000","last_fullsized":"2023-11-28T13:01:13.105277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090728+0000","last_change":"2023-11-28T13:01:12.704846+0000","last_active":"2023-11-28T13:01:14.090728+0000","last_peered":"2023-11-28T13:01:14.090728+0000","last_clean":"2023-11-28T13:01:14.090728+0000","last_became_active":"2023-11-28T13:01:12.704605+0000","last_became_peered":"2023-11-28T13:01:12.704605+0000","last_unstale":"2023-11-28T13:01:14.090728+0000","last_undegraded":"2023-11-28T13:01:14.090728+0000","last_fullsized":"2023-11-28T13:01:14.090728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:43.923726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325464+0000","last_change":"2023-11-28T13:01:13.101854+0000","last_active":"2023-11-28T13:01:14.325464+0000","last_peered":"2023-11-28T13:01:14.325464+0000","last_clean":"2023-11-28T13:01:14.325464+0000","last_became_active":"2023-11-28T13:01:13.101608+0000","last_became_peered":"2023-11-28T13:01:13.101608+0000","last_unstale":"2023-11-28T13:01:14.325464+0000","last_undegraded":"2023-11-28T13:01:14.325464+0000","last_fullsized":"2023-11-28T13:01:14.325464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090672+0000","last_change":"2023-11-28T13:01:12.700930+0000","last_active":"2023-11-28T13:01:14.090672+0000","last_peered":"2023-11-28T13:01:14.090672+0000","last_clean":"2023-11-28T13:01:14.090672+0000","last_became_active":"2023-11-28T13:01:12.700670+0000","last_became_peered":"2023-11-28T13:01:12.700670+0000","last_unstale":"2023-11-28T13:01:14.090672+0000","last_undegraded":"2023-11-28T13:01:14.090672+0000","last_fullsized":"2023-11-28T13:01:14.090672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090509+0000","last_change":"2023-11-28T13:01:13.093993+0000","last_active":"2023-11-28T13:01:14.090509+0000","last_peered":"2023-11-28T13:01:14.090509+0000","last_clean":"2023-11-28T13:01:14.090509+0000","last_became_active":"2023-11-28T13:01:13.093750+0000","last_became_peered":"2023-11-28T13:01:13.093750+0000","last_unstale":"2023-11-28T13:01:14.090509+0000","last_undegraded":"2023-11-28T13:01:14.090509+0000","last_fullsized":"2023-11-28T13:01:14.090509+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:47.159742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088952+0000","last_change":"2023-11-28T13:01:12.706087+0000","last_active":"2023-11-28T13:01:14.088952+0000","last_peered":"2023-11-28T13:01:14.088952+0000","last_clean":"2023-11-28T13:01:14.088952+0000","last_became_active":"2023-11-28T13:01:12.705800+0000","last_became_peered":"2023-11-28T13:01:12.705800+0000","last_unstale":"2023-11-28T13:01:14.088952+0000","last_undegraded":"2023-11-28T13:01:14.088952+0000","last_fullsized":"2023-11-28T13:01:14.088952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102554+0000","last_change":"2023-11-28T13:01:13.101116+0000","last_active":"2023-11-28T13:01:13.102554+0000","last_peered":"2023-11-28T13:01:13.102554+0000","last_clean":"2023-11-28T13:01:13.102554+0000","last_became_active":"2023-11-28T13:01:13.100510+0000","last_became_peered":"2023-11-28T13:01:13.100510+0000","last_unstale":"2023-11-28T13:01:13.102554+0000","last_undegraded":"2023-11-28T13:01:13.102554+0000","last_fullsized":"2023-11-28T13:01:13.102554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:42.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081264+0000","last_change":"2023-11-28T13:01:12.702749+0000","last_active":"2023-11-28T13:01:13.081264+0000","last_peered":"2023-11-28T13:01:13.081264+0000","last_clean":"2023-11-28T13:01:13.081264+0000","last_became_active":"2023-11-28T13:01:12.702450+0000","last_became_peered":"2023-11-28T13:01:12.702450+0000","last_unstale":"2023-11-28T13:01:13.081264+0000","last_undegraded":"2023-11-28T13:01:13.081264+0000","last_fullsized":"2023-11-28T13:01:13.081264+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:38.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090531+0000","last_change":"2023-11-28T13:01:13.095487+0000","last_active":"2023-11-28T13:01:14.090531+0000","last_peered":"2023-11-28T13:01:14.090531+0000","last_clean":"2023-11-28T13:01:14.090531+0000","last_became_active":"2023-11-28T13:01:13.095222+0000","last_became_peered":"2023-11-28T13:01:13.095222+0000","last_unstale":"2023-11-28T13:01:14.090531+0000","last_undegraded":"2023-11-28T13:01:14.090531+0000","last_fullsized":"2023-11-28T13:01:14.090531+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:57.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106265+0000","last_change":"2023-11-28T13:01:12.704714+0000","last_active":"2023-11-28T13:01:13.106265+0000","last_peered":"2023-11-28T13:01:13.106265+0000","last_clean":"2023-11-28T13:01:13.106265+0000","last_became_active":"2023-11-28T13:01:12.704422+0000","last_became_peered":"2023-11-28T13:01:12.704422+0000","last_unstale":"2023-11-28T13:01:13.106265+0000","last_undegraded":"2023-11-28T13:01:13.106265+0000","last_fullsized":"2023-11-28T13:01:13.106265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:42.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090726+0000","last_change":"2023-11-28T13:01:13.088050+0000","last_active":"2023-11-28T13:01:14.090726+0000","last_peered":"2023-11-28T13:01:14.090726+0000","last_clean":"2023-11-28T13:01:14.090726+0000","last_became_active":"2023-11-28T13:01:13.085247+0000","last_became_peered":"2023-11-28T13:01:13.085247+0000","last_unstale":"2023-11-28T13:01:14.090726+0000","last_undegraded":"2023-11-28T13:01:14.090726+0000","last_fullsized":"2023-11-28T13:01:14.090726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:39.035787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107176+0000","last_change":"2023-11-28T13:01:12.702173+0000","last_active":"2023-11-28T13:01:13.107176+0000","last_peered":"2023-11-28T13:01:13.107176+0000","last_clean":"2023-11-28T13:01:13.107176+0000","last_became_active":"2023-11-28T13:01:12.701243+0000","last_became_peered":"2023-11-28T13:01:12.701243+0000","last_unstale":"2023-11-28T13:01:13.107176+0000","last_undegraded":"2023-11-28T13:01:13.107176+0000","last_fullsized":"2023-11-28T13:01:13.107176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102621+0000","last_change":"2023-11-28T13:01:13.099301+0000","last_active":"2023-11-28T13:01:13.102621+0000","last_peered":"2023-11-28T13:01:13.102621+0000","last_clean":"2023-11-28T13:01:13.102621+0000","last_became_active":"2023-11-28T13:01:13.099017+0000","last_became_peered":"2023-11-28T13:01:13.099017+0000","last_unstale":"2023-11-28T13:01:13.102621+0000","last_undegraded":"2023-11-28T13:01:13.102621+0000","last_fullsized":"2023-11-28T13:01:13.102621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104381+0000","last_change":"2023-11-28T13:01:12.706078+0000","last_active":"2023-11-28T13:01:13.104381+0000","last_peered":"2023-11-28T13:01:13.104381+0000","last_clean":"2023-11-28T13:01:13.104381+0000","last_became_active":"2023-11-28T13:01:12.705694+0000","last_became_peered":"2023-11-28T13:01:12.705694+0000","last_unstale":"2023-11-28T13:01:13.104381+0000","last_undegraded":"2023-11-28T13:01:13.104381+0000","last_fullsized":"2023-11-28T13:01:13.104381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:41.987277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.105893+0000","last_change":"2023-11-28T13:01:13.098692+0000","last_active":"2023-11-28T13:01:13.105893+0000","last_peered":"2023-11-28T13:01:13.105893+0000","last_clean":"2023-11-28T13:01:13.105893+0000","last_became_active":"2023-11-28T13:01:13.098426+0000","last_became_peered":"2023-11-28T13:01:13.098426+0000","last_unstale":"2023-11-28T13:01:13.105893+0000","last_undegraded":"2023-11-28T13:01:13.105893+0000","last_fullsized":"2023-11-28T13:01:13.105893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.101277+0000","last_change":"2023-11-28T13:01:12.701155+0000","last_active":"2023-11-28T13:01:13.101277+0000","last_peered":"2023-11-28T13:01:13.101277+0000","last_clean":"2023-11-28T13:01:13.101277+0000","last_became_active":"2023-11-28T13:01:12.700897+0000","last_became_peered":"2023-11-28T13:01:12.700897+0000","last_unstale":"2023-11-28T13:01:13.101277+0000","last_undegraded":"2023-11-28T13:01:13.101277+0000","last_fullsized":"2023-11-28T13:01:13.101277+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:57.063655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.100622+0000","last_change":"2023-11-28T13:01:13.094885+0000","last_active":"2023-11-28T13:01:13.100622+0000","last_peered":"2023-11-28T13:01:13.100622+0000","last_clean":"2023-11-28T13:01:13.100622+0000","last_became_active":"2023-11-28T13:01:13.094027+0000","last_became_peered":"2023-11-28T13:01:13.094027+0000","last_unstale":"2023-11-28T13:01:13.100622+0000","last_undegraded":"2023-11-28T13:01:13.100622+0000","last_fullsized":"2023-11-28T13:01:13.100622+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:43.321762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089094+0000","last_change":"2023-11-28T13:01:12.703499+0000","last_active":"2023-11-28T13:01:14.089094+0000","last_peered":"2023-11-28T13:01:14.089094+0000","last_clean":"2023-11-28T13:01:14.089094+0000","last_became_active":"2023-11-28T13:01:12.703271+0000","last_became_peered":"2023-11-28T13:01:12.703271+0000","last_unstale":"2023-11-28T13:01:14.089094+0000","last_undegraded":"2023-11-28T13:01:14.089094+0000","last_fullsized":"2023-11-28T13:01:14.089094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:03:30.119943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107824+0000","last_change":"2023-11-28T13:01:13.101678+0000","last_active":"2023-11-28T13:01:13.107824+0000","last_peered":"2023-11-28T13:01:13.107824+0000","last_clean":"2023-11-28T13:01:13.107824+0000","last_became_active":"2023-11-28T13:01:13.101386+0000","last_became_peered":"2023-11-28T13:01:13.101386+0000","last_unstale":"2023-11-28T13:01:13.107824+0000","last_undegraded":"2023-11-28T13:01:13.107824+0000","last_fullsized":"2023-11-28T13:01:13.107824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:51.226552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.095695+0000","last_change":"2023-11-28T13:01:12.697001+0000","last_active":"2023-11-28T13:01:13.095695+0000","last_peered":"2023-11-28T13:01:13.095695+0000","last_clean":"2023-11-28T13:01:13.095695+0000","last_became_active":"2023-11-28T13:01:12.696739+0000","last_became_peered":"2023-11-28T13:01:12.696739+0000","last_unstale":"2023-11-28T13:01:13.095695+0000","last_undegraded":"2023-11-28T13:01:13.095695+0000","last_fullsized":"2023-11-28T13:01:13.095695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:37.438564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325346+0000","last_change":"2023-11-28T13:01:13.096745+0000","last_active":"2023-11-28T13:01:14.325346+0000","last_peered":"2023-11-28T13:01:14.325346+0000","last_clean":"2023-11-28T13:01:14.325346+0000","last_became_active":"2023-11-28T13:01:13.096461+0000","last_became_peered":"2023-11-28T13:01:13.096461+0000","last_unstale":"2023-11-28T13:01:14.325346+0000","last_undegraded":"2023-11-28T13:01:14.325346+0000","last_fullsized":"2023-11-28T13:01:14.325346+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.987104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083496+0000","last_change":"2023-11-28T13:01:12.698523+0000","last_active":"2023-11-28T13:01:13.083496+0000","last_peered":"2023-11-28T13:01:13.083496+0000","last_clean":"2023-11-28T13:01:13.083496+0000","last_became_active":"2023-11-28T13:01:12.698275+0000","last_became_peered":"2023-11-28T13:01:12.698275+0000","last_unstale":"2023-11-28T13:01:13.083496+0000","last_undegraded":"2023-11-28T13:01:13.083496+0000","last_fullsized":"2023-11-28T13:01:13.083496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:02.254485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.102691+0000","last_change":"2023-11-28T13:01:13.099007+0000","last_active":"2023-11-28T13:01:13.102691+0000","last_peered":"2023-11-28T13:01:13.102691+0000","last_clean":"2023-11-28T13:01:13.102691+0000","last_became_active":"2023-11-28T13:01:13.098673+0000","last_became_peered":"2023-11-28T13:01:13.098673+0000","last_unstale":"2023-11-28T13:01:13.102691+0000","last_undegraded":"2023-11-28T13:01:13.102691+0000","last_fullsized":"2023-11-28T13:01:13.102691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091666+0000","last_change":"2023-11-28T13:01:12.706237+0000","last_active":"2023-11-28T13:01:13.091666+0000","last_peered":"2023-11-28T13:01:13.091666+0000","last_clean":"2023-11-28T13:01:13.091666+0000","last_became_active":"2023-11-28T13:01:12.705981+0000","last_became_peered":"2023-11-28T13:01:12.705981+0000","last_unstale":"2023-11-28T13:01:13.091666+0000","last_undegraded":"2023-11-28T13:01:13.091666+0000","last_fullsized":"2023-11-28T13:01:13.091666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090578+0000","last_change":"2023-11-28T13:01:13.099382+0000","last_active":"2023-11-28T13:01:14.090578+0000","last_peered":"2023-11-28T13:01:14.090578+0000","last_clean":"2023-11-28T13:01:14.090578+0000","last_became_active":"2023-11-28T13:01:13.099008+0000","last_became_peered":"2023-11-28T13:01:13.099008+0000","last_unstale":"2023-11-28T13:01:14.090578+0000","last_undegraded":"2023-11-28T13:01:14.090578+0000","last_fullsized":"2023-11-28T13:01:14.090578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:35.600930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083382+0000","last_change":"2023-11-28T13:01:12.696728+0000","last_active":"2023-11-28T13:01:13.083382+0000","last_peered":"2023-11-28T13:01:13.083382+0000","last_clean":"2023-11-28T13:01:13.083382+0000","last_became_active":"2023-11-28T13:01:12.696428+0000","last_became_peered":"2023-11-28T13:01:12.696428+0000","last_unstale":"2023-11-28T13:01:13.083382+0000","last_undegraded":"2023-11-28T13:01:13.083382+0000","last_fullsized":"2023-11-28T13:01:13.083382+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:28.781799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325302+0000","last_change":"2023-11-28T13:01:13.099016+0000","last_active":"2023-11-28T13:01:14.325302+0000","last_peered":"2023-11-28T13:01:14.325302+0000","last_clean":"2023-11-28T13:01:14.325302+0000","last_became_active":"2023-11-28T13:01:13.098565+0000","last_became_peered":"2023-11-28T13:01:13.098565+0000","last_unstale":"2023-11-28T13:01:14.325302+0000","last_undegraded":"2023-11-28T13:01:14.325302+0000","last_fullsized":"2023-11-28T13:01:14.325302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.089564+0000","last_change":"2023-11-28T13:01:12.698962+0000","last_active":"2023-11-28T13:01:13.089564+0000","last_peered":"2023-11-28T13:01:13.089564+0000","last_clean":"2023-11-28T13:01:13.089564+0000","last_became_active":"2023-11-28T13:01:12.698648+0000","last_became_peered":"2023-11-28T13:01:12.698648+0000","last_unstale":"2023-11-28T13:01:13.089564+0000","last_undegraded":"2023-11-28T13:01:13.089564+0000","last_fullsized":"2023-11-28T13:01:13.089564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091534+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:01:13.091534+0000","last_peered":"2023-11-28T13:01:13.091534+0000","last_clean":"2023-11-28T13:01:13.091534+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:01:13.091534+0000","last_undegraded":"2023-11-28T13:01:13.091534+0000","last_fullsized":"2023-11-28T13:01:13.091534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090805+0000","last_change":"2023-11-28T13:01:13.098873+0000","last_active":"2023-11-28T13:01:14.090805+0000","last_peered":"2023-11-28T13:01:14.090805+0000","last_clean":"2023-11-28T13:01:14.090805+0000","last_became_active":"2023-11-28T13:01:13.098618+0000","last_became_peered":"2023-11-28T13:01:13.098618+0000","last_unstale":"2023-11-28T13:01:14.090805+0000","last_undegraded":"2023-11-28T13:01:14.090805+0000","last_fullsized":"2023-11-28T13:01:14.090805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:49.374140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.089151+0000","last_change":"2023-11-28T13:01:12.699988+0000","last_active":"2023-11-28T13:01:13.089151+0000","last_peered":"2023-11-28T13:01:13.089151+0000","last_clean":"2023-11-28T13:01:13.089151+0000","last_became_active":"2023-11-28T13:01:12.699721+0000","last_became_peered":"2023-11-28T13:01:12.699721+0000","last_unstale":"2023-11-28T13:01:13.089151+0000","last_undegraded":"2023-11-28T13:01:13.089151+0000","last_fullsized":"2023-11-28T13:01:13.089151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699954+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:01:12.699954+0000","last_peered":"2023-11-28T13:01:12.699954+0000","last_clean":"2023-11-28T13:01:12.699954+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:01:12.699954+0000","last_undegraded":"2023-11-28T13:01:12.699954+0000","last_fullsized":"2023-11-28T13:01:12.699954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110193+0000","last_change":"2023-11-28T13:01:13.103633+0000","last_active":"2023-11-28T13:01:13.110193+0000","last_peered":"2023-11-28T13:01:13.110193+0000","last_clean":"2023-11-28T13:01:13.110193+0000","last_became_active":"2023-11-28T13:01:13.103198+0000","last_became_peered":"2023-11-28T13:01:13.103198+0000","last_unstale":"2023-11-28T13:01:13.110193+0000","last_undegraded":"2023-11-28T13:01:13.110193+0000","last_fullsized":"2023-11-28T13:01:13.110193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.084615+0000","last_change":"2023-11-28T13:01:12.704388+0000","last_active":"2023-11-28T13:01:13.084615+0000","last_peered":"2023-11-28T13:01:13.084615+0000","last_clean":"2023-11-28T13:01:13.084615+0000","last_became_active":"2023-11-28T13:01:12.704082+0000","last_became_peered":"2023-11-28T13:01:12.704082+0000","last_unstale":"2023-11-28T13:01:13.084615+0000","last_undegraded":"2023-11-28T13:01:13.084615+0000","last_fullsized":"2023-11-28T13:01:13.084615+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:08.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.685641+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:01:12.685641+0000","last_peered":"2023-11-28T13:01:12.685641+0000","last_clean":"2023-11-28T13:01:12.685641+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:01:12.685641+0000","last_undegraded":"2023-11-28T13:01:12.685641+0000","last_fullsized":"2023-11-28T13:01:12.685641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.110430+0000","last_change":"2023-11-28T13:01:13.105495+0000","last_active":"2023-11-28T13:01:13.110430+0000","last_peered":"2023-11-28T13:01:13.110430+0000","last_clean":"2023-11-28T13:01:13.110430+0000","last_became_active":"2023-11-28T13:01:13.103141+0000","last_became_peered":"2023-11-28T13:01:13.103141+0000","last_unstale":"2023-11-28T13:01:13.110430+0000","last_undegraded":"2023-11-28T13:01:13.110430+0000","last_fullsized":"2023-11-28T13:01:13.110430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:08.241448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091757+0000","last_change":"2023-11-28T13:01:12.704702+0000","last_active":"2023-11-28T13:01:13.091757+0000","last_peered":"2023-11-28T13:01:13.091757+0000","last_clean":"2023-11-28T13:01:13.091757+0000","last_became_active":"2023-11-28T13:01:12.704416+0000","last_became_peered":"2023-11-28T13:01:12.704416+0000","last_unstale":"2023-11-28T13:01:13.091757+0000","last_undegraded":"2023-11-28T13:01:13.091757+0000","last_fullsized":"2023-11-28T13:01:13.091757+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:02.624167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.700208+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:01:12.700208+0000","last_peered":"2023-11-28T13:01:12.700208+0000","last_clean":"2023-11-28T13:01:12.700208+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:01:12.700208+0000","last_undegraded":"2023-11-28T13:01:12.700208+0000","last_fullsized":"2023-11-28T13:01:12.700208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698573+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:01:12.698573+0000","last_peered":"2023-11-28T13:01:12.698573+0000","last_clean":"2023-11-28T13:01:12.698573+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:01:12.698573+0000","last_undegraded":"2023-11-28T13:01:12.698573+0000","last_fullsized":"2023-11-28T13:01:12.698573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088962+0000","last_change":"2023-11-28T13:01:13.104062+0000","last_active":"2023-11-28T13:01:14.088962+0000","last_peered":"2023-11-28T13:01:14.088962+0000","last_clean":"2023-11-28T13:01:14.088962+0000","last_became_active":"2023-11-28T13:01:13.103815+0000","last_became_peered":"2023-11-28T13:01:13.103815+0000","last_unstale":"2023-11-28T13:01:14.088962+0000","last_undegraded":"2023-11-28T13:01:14.088962+0000","last_fullsized":"2023-11-28T13:01:14.088962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:09.350952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.106674+0000","last_change":"2023-11-28T13:01:12.706680+0000","last_active":"2023-11-28T13:01:13.106674+0000","last_peered":"2023-11-28T13:01:13.106674+0000","last_clean":"2023-11-28T13:01:13.106674+0000","last_became_active":"2023-11-28T13:01:12.706387+0000","last_became_peered":"2023-11-28T13:01:12.706387+0000","last_unstale":"2023-11-28T13:01:13.106674+0000","last_undegraded":"2023-11-28T13:01:13.106674+0000","last_fullsized":"2023-11-28T13:01:13.106674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.698664+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:01:12.698664+0000","last_peered":"2023-11-28T13:01:12.698664+0000","last_clean":"2023-11-28T13:01:12.698664+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:01:12.698664+0000","last_undegraded":"2023-11-28T13:01:12.698664+0000","last_fullsized":"2023-11-28T13:01:12.698664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088854+0000","last_change":"2023-11-28T13:01:13.100011+0000","last_active":"2023-11-28T13:01:14.088854+0000","last_peered":"2023-11-28T13:01:14.088854+0000","last_clean":"2023-11-28T13:01:14.088854+0000","last_became_active":"2023-11-28T13:01:13.099738+0000","last_became_peered":"2023-11-28T13:01:13.099738+0000","last_unstale":"2023-11-28T13:01:14.088854+0000","last_undegraded":"2023-11-28T13:01:14.088854+0000","last_fullsized":"2023-11-28T13:01:14.088854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:52.246680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081223+0000","last_change":"2023-11-28T13:01:12.694952+0000","last_active":"2023-11-28T13:01:13.081223+0000","last_peered":"2023-11-28T13:01:13.081223+0000","last_clean":"2023-11-28T13:01:13.081223+0000","last_became_active":"2023-11-28T13:01:12.694694+0000","last_became_peered":"2023-11-28T13:01:12.694694+0000","last_unstale":"2023-11-28T13:01:13.081223+0000","last_undegraded":"2023-11-28T13:01:13.081223+0000","last_fullsized":"2023-11-28T13:01:13.081223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:54.638737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.699297+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:01:12.699297+0000","last_peered":"2023-11-28T13:01:12.699297+0000","last_clean":"2023-11-28T13:01:12.699297+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:01:12.699297+0000","last_undegraded":"2023-11-28T13:01:12.699297+0000","last_fullsized":"2023-11-28T13:01:12.699297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.104886+0000","last_change":"2023-11-28T13:01:13.099302+0000","last_active":"2023-11-28T13:01:13.104886+0000","last_peered":"2023-11-28T13:01:13.104886+0000","last_clean":"2023-11-28T13:01:13.104886+0000","last_became_active":"2023-11-28T13:01:13.099047+0000","last_became_peered":"2023-11-28T13:01:13.099047+0000","last_unstale":"2023-11-28T13:01:13.104886+0000","last_undegraded":"2023-11-28T13:01:13.104886+0000","last_fullsized":"2023-11-28T13:01:13.104886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.092648+0000","last_change":"2023-11-28T13:01:12.700645+0000","last_active":"2023-11-28T13:01:13.092648+0000","last_peered":"2023-11-28T13:01:13.092648+0000","last_clean":"2023-11-28T13:01:13.092648+0000","last_became_active":"2023-11-28T13:01:12.700349+0000","last_became_peered":"2023-11-28T13:01:12.700349+0000","last_unstale":"2023-11-28T13:01:13.092648+0000","last_undegraded":"2023-11-28T13:01:13.092648+0000","last_fullsized":"2023-11-28T13:01:13.092648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:50.224596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.086932+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:01:13.086932+0000","last_peered":"2023-11-28T13:01:13.086932+0000","last_clean":"2023-11-28T13:01:13.086932+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:01:13.086932+0000","last_undegraded":"2023-11-28T13:01:13.086932+0000","last_fullsized":"2023-11-28T13:01:13.086932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.107847+0000","last_change":"2023-11-28T13:01:13.101838+0000","last_active":"2023-11-28T13:01:13.107847+0000","last_peered":"2023-11-28T13:01:13.107847+0000","last_clean":"2023-11-28T13:01:13.107847+0000","last_became_active":"2023-11-28T13:01:13.101565+0000","last_became_peered":"2023-11-28T13:01:13.101565+0000","last_unstale":"2023-11-28T13:01:13.107847+0000","last_undegraded":"2023-11-28T13:01:13.107847+0000","last_fullsized":"2023-11-28T13:01:13.107847+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:08.927902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090444+0000","last_change":"2023-11-28T13:01:12.699934+0000","last_active":"2023-11-28T13:01:14.090444+0000","last_peered":"2023-11-28T13:01:14.090444+0000","last_clean":"2023-11-28T13:01:14.090444+0000","last_became_active":"2023-11-28T13:01:12.699617+0000","last_became_peered":"2023-11-28T13:01:12.699617+0000","last_unstale":"2023-11-28T13:01:14.090444+0000","last_undegraded":"2023-11-28T13:01:14.090444+0000","last_fullsized":"2023-11-28T13:01:14.090444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:02.801702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-28T13:01:12.690011+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:01:12.690011+0000","last_peered":"2023-11-28T13:01:12.690011+0000","last_clean":"2023-11-28T13:01:12.690011+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:01:12.690011+0000","last_undegraded":"2023-11-28T13:01:12.690011+0000","last_fullsized":"2023-11-28T13:01:12.690011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":184228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.497},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.218,"5min":8.218,"15min":8.218},"last":0.35099999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.439},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":13.384,"5min":13.384,"15min":13.384},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.226,"5min":5.226,"15min":5.226},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.624}]},{"osd":5,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.3209999999999997,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.1029999999999998,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.3849999999999998,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.70799999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8364,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363476,"statfs":{"total":96636764160,"available":96628199424,"internally_reserved":0,"allocated":430080,"data_stored":176063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.27200000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.247},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.42799999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.498},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":168865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.438},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":1032192,"data_stored":772279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":14.997999999999999,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":19.045999999999999,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.377}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":18.995000000000001,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":19.02,"5min":19.02,"15min":19.02},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.1539999999999999,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":18.957999999999998,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.59299999999999997}]}]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9428,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362412,"statfs":{"total":96636764160,"available":96627109888,"internally_reserved":0,"allocated":995328,"data_stored":752605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66300000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":4.7830000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.9510000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":4.8440000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":2.6030000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.87}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":1.25},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":2.5329999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.972},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":2.508}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8496,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363344,"statfs":{"total":96636764160,"available":96628064256,"internally_reserved":0,"allocated":434176,"data_stored":176105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.435},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.67300000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69599999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8428,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363412,"statfs":{"total":96636764160,"available":96628133888,"internally_reserved":0,"allocated":430080,"data_stored":176559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.629}]},{"osd":3,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.503}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56299999999999994}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.78000000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8492,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363348,"statfs":{"total":96636764160,"available":96628068352,"internally_reserved":0,"allocated":430080,"data_stored":176083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.247}]},{"osd":2,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.37}]},{"osd":3,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.64100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:01:21.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:01:21.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.833+0000 7fbb6f227700 1 Processor -- start 2023-11-28T13:01:21.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.833+0000 7fbb6f227700 1 -- start start 2023-11-28T13:01:21.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb6f227700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbb680cd8c0 0x7fbb68140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb6f227700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbb680cdc90 0x7fbb680b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb6f227700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb6806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb6f227700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6813ff00 con 0x7fbb680cdc90 2023-11-28T13:01:21.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb6f227700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb681404c0 con 0x7fbb680cd8c0 2023-11-28T13:01:21.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb6f227700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6815aa80 con 0x7fbb680b2e70 2023-11-28T13:01:21.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb67fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbb680cdc90 0x7fbb680b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb67fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbb680cdc90 0x7fbb680b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40322/0 (socket says 172.21.15.153:40322) 2023-11-28T13:01:21.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.834+0000 7fbb67fff700 1 -- 172.21.15.153:0/4198902682 learned_addr learned my addr 172.21.15.153:0/4198902682 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:21.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb6cfc3700 1 --2- 172.21.15.153:0/4198902682 >> v2:172.21.15.177:3300/0 conn(0x7fbb680cd8c0 0x7fbb68140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb6d7c4700 1 --2- 172.21.15.153:0/4198902682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb6806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb67fff700 1 -- 172.21.15.153:0/4198902682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 msgr2=0x7fbb6806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb67fff700 1 --2- 172.21.15.153:0/4198902682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb6806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb67fff700 1 -- 172.21.15.153:0/4198902682 >> v2:172.21.15.177:3300/0 conn(0x7fbb680cd8c0 msgr2=0x7fbb68140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb67fff700 1 --2- 172.21.15.153:0/4198902682 >> v2:172.21.15.177:3300/0 conn(0x7fbb680cd8c0 0x7fbb68140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb67fff700 1 -- 172.21.15.153:0/4198902682 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680b3310 con 0x7fbb680cdc90 2023-11-28T13:01:21.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb6d7c4700 1 --2- 172.21.15.153:0/4198902682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb6806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:21.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb67fff700 1 --2- 172.21.15.153:0/4198902682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cdc90 0x7fbb680b2930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbb68070a20 tx=0x7fbb54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222cf779e3a35bf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb677fe700 1 -- 172.21.15.153:0/4198902682 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400b870 con 0x7fbb680cdc90 2023-11-28T13:01:21.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb677fe700 1 -- 172.21.15.153:0/4198902682 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5400b9d0 con 0x7fbb680cdc90 2023-11-28T13:01:21.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb677fe700 1 -- 172.21.15.153:0/4198902682 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400bd60 con 0x7fbb680cdc90 2023-11-28T13:01:21.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb6f227700 1 -- 172.21.15.153:0/4198902682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cdc90 msgr2=0x7fbb680b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb6f227700 1 --2- 172.21.15.153:0/4198902682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cdc90 0x7fbb680b2930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbb68070a20 tx=0x7fbb54009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb6f227700 1 -- 172.21.15.153:0/4198902682 shutdown_connections 2023-11-28T13:01:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.835+0000 7fbb6f227700 1 --2- 172.21.15.153:0/4198902682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb6806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.836+0000 7fbb6f227700 1 --2- 172.21.15.153:0/4198902682 >> v2:172.21.15.177:3300/0 conn(0x7fbb680cd8c0 0x7fbb68140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.836+0000 7fbb6f227700 1 --2- 172.21.15.153:0/4198902682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cdc90 0x7fbb680b2930 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.836+0000 7fbb6f227700 1 -- 172.21.15.153:0/4198902682 >> 172.21.15.153:0/4198902682 conn(0x7fbb6806aed0 msgr2=0x7fbb6806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.836+0000 7fbb6f227700 1 -- 172.21.15.153:0/4198902682 shutdown_connections 2023-11-28T13:01:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.836+0000 7fbb6f227700 1 -- 172.21.15.153:0/4198902682 wait complete. 2023-11-28T13:01:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.837+0000 7fbb6f227700 1 Processor -- start 2023-11-28T13:01:21.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.837+0000 7fbb6f227700 1 -- start start 2023-11-28T13:01:21.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb6f227700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb680b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb6f227700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cd8c0 0x7fbb681efca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb6f227700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb680cdc90 0x7fbb680ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb6f227700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680b53e0 con 0x7fbb680cd8c0 2023-11-28T13:01:21.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb6f227700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680b56c0 con 0x7fbb680cdc90 2023-11-28T13:01:21.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb6f227700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680b59a0 con 0x7fbb680b2e70 2023-11-28T13:01:21.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cd8c0 0x7fbb681efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb67fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cd8c0 0x7fbb681efca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40328/0 (socket says 172.21.15.153:40328) 2023-11-28T13:01:21.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.838+0000 7fbb67fff700 1 -- 172.21.15.153:0/1880718635 learned_addr learned my addr 172.21.15.153:0/1880718635 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:21.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb6d7c4700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb680cdc90 0x7fbb680ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb6cfc3700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb680b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb67fff700 1 -- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 msgr2=0x7fbb680b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb67fff700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb680b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb67fff700 1 -- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb680cdc90 msgr2=0x7fbb680ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb67fff700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb680cdc90 0x7fbb680ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb67fff700 1 -- 172.21.15.153:0/1880718635 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb54009550 con 0x7fbb680cd8c0 2023-11-28T13:01:21.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb67fff700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cd8c0 0x7fbb681efca0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbb54000c00 tx=0x7fbb54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb657fa700 1 -- 172.21.15.153:0/1880718635 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54004480 con 0x7fbb680cd8c0 2023-11-28T13:01:21.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb657fa700 1 -- 172.21.15.153:0/1880718635 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb540045e0 con 0x7fbb680cd8c0 2023-11-28T13:01:21.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb657fa700 1 -- 172.21.15.153:0/1880718635 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb540038e0 con 0x7fbb680cd8c0 2023-11-28T13:01:21.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb68140150 con 0x7fbb680cd8c0 2023-11-28T13:01:21.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.839+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb680b35f0 con 0x7fbb680cd8c0 2023-11-28T13:01:21.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.840+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb680ce100 con 0x7fbb680cd8c0 2023-11-28T13:01:21.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.841+0000 7fbb657fa700 1 -- 172.21.15.153:0/1880718635 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb5400f050 con 0x7fbb680cd8c0 2023-11-28T13:01:21.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.841+0000 7fbb657fa700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbb4c0682f0 0x7fbb4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.841+0000 7fbb657fa700 1 -- 172.21.15.153:0/1880718635 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb540873c0 con 0x7fbb680cd8c0 2023-11-28T13:01:21.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.841+0000 7fbb6cfc3700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbb4c0682f0 0x7fbb4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.841+0000 7fbb6cfc3700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbb4c0682f0 0x7fbb4c06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c006fd0 tx=0x7fbb5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:21.845+0000 7fbb657fa700 1 -- 172.21.15.153:0/1880718635 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb54053c20 con 0x7fbb680cd8c0 2023-11-28T13:01:22.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.022+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb680b5830 con 0x7fbb4c0682f0 2023-11-28T13:01:22.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.031+0000 7fbb657fa700 1 -- 172.21.15.153:0/1880718635 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311076 (secure 0 0 0) 0x7fbb680b5830 con 0x7fbb4c0682f0 2023-11-28T13:01:22.036 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:22.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbb4c0682f0 msgr2=0x7fbb4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbb4c0682f0 0x7fbb4c06a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c006fd0 tx=0x7fbb5c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:22.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cd8c0 msgr2=0x7fbb681efca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cd8c0 0x7fbb681efca0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbb54000c00 tx=0x7fbb54004750 comp rx=0 tx=0).stop 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 shutdown_connections 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb680cdc90 0x7fbb680ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb680b2e70 0x7fbb680b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbb4c0682f0 0x7fbb4c06a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 --2- 172.21.15.153:0/1880718635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb680cd8c0 0x7fbb681efca0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 >> 172.21.15.153:0/1880718635 conn(0x7fbb6806aed0 msgr2=0x7fbb680cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 shutdown_connections 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.034+0000 7fbb6f227700 1 -- 172.21.15.153:0/1880718635 wait complete. 2023-11-28T13:01:22.039 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:01:22.048 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-11-28T13:01:20.159250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":70244,"kb_used_data":4644,"kb_used_omap":0,"kb_used_meta":65536,"kb_avail":754904476,"statfs":{"total":773094113280,"available":773022183424,"internally_reserved":0,"allocated":4755456,"data_stored":2645795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":67108864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.475126"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090821+0000","last_change":"2023-11-28T13:01:13.094545+0000","last_active":"2023-11-28T13:01:14.090821+0000","last_peered":"2023-11-28T13:01:14.090821+0000","last_clean":"2023-11-28T13:01:14.090821+0000","last_became_active":"2023-11-28T13:01:13.094296+0000","last_became_peered":"2023-11-28T13:01:13.094296+0000","last_unstale":"2023-11-28T13:01:14.090821+0000","last_undegraded":"2023-11-28T13:01:14.090821+0000","last_fullsized":"2023-11-28T13:01:14.090821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:18.727519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090776+0000","last_change":"2023-11-28T13:01:13.088855+0000","last_active":"2023-11-28T13:01:14.090776+0000","last_peered":"2023-11-28T13:01:14.090776+0000","last_clean":"2023-11-28T13:01:14.090776+0000","last_became_active":"2023-11-28T13:01:13.088631+0000","last_became_peered":"2023-11-28T13:01:13.088631+0000","last_unstale":"2023-11-28T13:01:14.090776+0000","last_undegraded":"2023-11-28T13:01:14.090776+0000","last_fullsized":"2023-11-28T13:01:14.090776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:01:42.143519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325409+0000","last_change":"2023-11-28T13:01:13.103414+0000","last_active":"2023-11-28T13:01:14.325409+0000","last_peered":"2023-11-28T13:01:14.325409+0000","last_clean":"2023-11-28T13:01:14.325409+0000","last_became_active":"2023-11-28T13:01:13.103175+0000","last_became_peered":"2023-11-28T13:01:13.103175+0000","last_unstale":"2023-11-28T13:01:14.325409+0000","last_undegraded":"2023-11-28T13:01:14.325409+0000","last_fullsized":"2023-11-28T13:01:14.325409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536698+0000","last_change":"2023-11-28T13:01:13.102065+0000","last_active":"2023-11-28T13:01:14.536698+0000","last_peered":"2023-11-28T13:01:14.536698+0000","last_clean":"2023-11-28T13:01:14.536698+0000","last_became_active":"2023-11-28T13:01:13.101822+0000","last_became_peered":"2023-11-28T13:01:13.101822+0000","last_unstale":"2023-11-28T13:01:14.536698+0000","last_undegraded":"2023-11-28T13:01:14.536698+0000","last_fullsized":"2023-11-28T13:01:14.536698+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:51.732790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536658+0000","last_change":"2023-11-28T13:01:13.095907+0000","last_active":"2023-11-28T13:01:14.536658+0000","last_peered":"2023-11-28T13:01:14.536658+0000","last_clean":"2023-11-28T13:01:14.536658+0000","last_became_active":"2023-11-28T13:01:13.095190+0000","last_became_peered":"2023-11-28T13:01:13.095190+0000","last_unstale":"2023-11-28T13:01:14.536658+0000","last_undegraded":"2023-11-28T13:01:14.536658+0000","last_fullsized":"2023-11-28T13:01:14.536658+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:34.713740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088925+0000","last_change":"2023-11-28T13:01:13.103245+0000","last_active":"2023-11-28T13:01:14.088925+0000","last_peered":"2023-11-28T13:01:14.088925+0000","last_clean":"2023-11-28T13:01:14.088925+0000","last_became_active":"2023-11-28T13:01:13.102920+0000","last_became_peered":"2023-11-28T13:01:13.102920+0000","last_unstale":"2023-11-28T13:01:14.088925+0000","last_undegraded":"2023-11-28T13:01:14.088925+0000","last_fullsized":"2023-11-28T13:01:14.088925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:52:23.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344746+0000","last_change":"2023-11-28T13:01:13.101383+0000","last_active":"2023-11-28T13:01:13.344746+0000","last_peered":"2023-11-28T13:01:13.344746+0000","last_clean":"2023-11-28T13:01:13.344746+0000","last_became_active":"2023-11-28T13:01:13.101093+0000","last_became_peered":"2023-11-28T13:01:13.101093+0000","last_unstale":"2023-11-28T13:01:13.344746+0000","last_undegraded":"2023-11-28T13:01:13.344746+0000","last_fullsized":"2023-11-28T13:01:13.344746+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090405+0000","last_change":"2023-11-28T13:01:13.093176+0000","last_active":"2023-11-28T13:01:14.090405+0000","last_peered":"2023-11-28T13:01:14.090405+0000","last_clean":"2023-11-28T13:01:14.090405+0000","last_became_active":"2023-11-28T13:01:13.092922+0000","last_became_peered":"2023-11-28T13:01:13.092922+0000","last_unstale":"2023-11-28T13:01:14.090405+0000","last_undegraded":"2023-11-28T13:01:14.090405+0000","last_fullsized":"2023-11-28T13:01:14.090405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:31.712364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325497+0000","last_change":"2023-11-28T13:01:13.105535+0000","last_active":"2023-11-28T13:01:14.325497+0000","last_peered":"2023-11-28T13:01:14.325497+0000","last_clean":"2023-11-28T13:01:14.325497+0000","last_became_active":"2023-11-28T13:01:13.105306+0000","last_became_peered":"2023-11-28T13:01:13.105306+0000","last_unstale":"2023-11-28T13:01:14.325497+0000","last_undegraded":"2023-11-28T13:01:14.325497+0000","last_fullsized":"2023-11-28T13:01:14.325497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344910+0000","last_change":"2023-11-28T13:01:13.090026+0000","last_active":"2023-11-28T13:01:13.344910+0000","last_peered":"2023-11-28T13:01:13.344910+0000","last_clean":"2023-11-28T13:01:13.344910+0000","last_became_active":"2023-11-28T13:01:13.089734+0000","last_became_peered":"2023-11-28T13:01:13.089734+0000","last_unstale":"2023-11-28T13:01:13.344910+0000","last_undegraded":"2023-11-28T13:01:13.344910+0000","last_fullsized":"2023-11-28T13:01:13.344910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344887+0000","last_change":"2023-11-28T13:01:13.106317+0000","last_active":"2023-11-28T13:01:13.344887+0000","last_peered":"2023-11-28T13:01:13.344887+0000","last_clean":"2023-11-28T13:01:13.344887+0000","last_became_active":"2023-11-28T13:01:13.106096+0000","last_became_peered":"2023-11-28T13:01:13.106096+0000","last_unstale":"2023-11-28T13:01:13.344887+0000","last_undegraded":"2023-11-28T13:01:13.344887+0000","last_fullsized":"2023-11-28T13:01:13.344887+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325360+0000","last_change":"2023-11-28T13:01:13.095524+0000","last_active":"2023-11-28T13:01:14.325360+0000","last_peered":"2023-11-28T13:01:14.325360+0000","last_clean":"2023-11-28T13:01:14.325360+0000","last_became_active":"2023-11-28T13:01:13.095284+0000","last_became_peered":"2023-11-28T13:01:13.095284+0000","last_unstale":"2023-11-28T13:01:14.325360+0000","last_undegraded":"2023-11-28T13:01:14.325360+0000","last_fullsized":"2023-11-28T13:01:14.325360+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:51.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344809+0000","last_change":"2023-11-28T13:01:13.098543+0000","last_active":"2023-11-28T13:01:13.344809+0000","last_peered":"2023-11-28T13:01:13.344809+0000","last_clean":"2023-11-28T13:01:13.344809+0000","last_became_active":"2023-11-28T13:01:13.097626+0000","last_became_peered":"2023-11-28T13:01:13.097626+0000","last_unstale":"2023-11-28T13:01:13.344809+0000","last_undegraded":"2023-11-28T13:01:13.344809+0000","last_fullsized":"2023-11-28T13:01:13.344809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503294+0000","last_change":"2023-11-28T13:01:13.097007+0000","last_active":"2023-11-28T13:01:13.503294+0000","last_peered":"2023-11-28T13:01:13.503294+0000","last_clean":"2023-11-28T13:01:13.503294+0000","last_became_active":"2023-11-28T13:01:13.096697+0000","last_became_peered":"2023-11-28T13:01:13.096697+0000","last_unstale":"2023-11-28T13:01:13.503294+0000","last_undegraded":"2023-11-28T13:01:13.503294+0000","last_fullsized":"2023-11-28T13:01:13.503294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:25.643565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344780+0000","last_change":"2023-11-28T13:01:13.092555+0000","last_active":"2023-11-28T13:01:13.344780+0000","last_peered":"2023-11-28T13:01:13.344780+0000","last_clean":"2023-11-28T13:01:13.344780+0000","last_became_active":"2023-11-28T13:01:13.092269+0000","last_became_peered":"2023-11-28T13:01:13.092269+0000","last_unstale":"2023-11-28T13:01:13.344780+0000","last_undegraded":"2023-11-28T13:01:13.344780+0000","last_fullsized":"2023-11-28T13:01:13.344780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337014+0000","last_change":"2023-11-28T13:01:13.106928+0000","last_active":"2023-11-28T13:01:13.337014+0000","last_peered":"2023-11-28T13:01:13.337014+0000","last_clean":"2023-11-28T13:01:13.337014+0000","last_became_active":"2023-11-28T13:01:13.104168+0000","last_became_peered":"2023-11-28T13:01:13.104168+0000","last_unstale":"2023-11-28T13:01:13.337014+0000","last_undegraded":"2023-11-28T13:01:13.337014+0000","last_fullsized":"2023-11-28T13:01:13.337014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337298+0000","last_change":"2023-11-28T13:01:13.106957+0000","last_active":"2023-11-28T13:01:13.337298+0000","last_peered":"2023-11-28T13:01:13.337298+0000","last_clean":"2023-11-28T13:01:13.337298+0000","last_became_active":"2023-11-28T13:01:13.104842+0000","last_became_peered":"2023-11-28T13:01:13.104842+0000","last_unstale":"2023-11-28T13:01:13.337298+0000","last_undegraded":"2023-11-28T13:01:13.337298+0000","last_fullsized":"2023-11-28T13:01:13.337298+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336873+0000","last_change":"2023-11-28T13:01:13.102950+0000","last_active":"2023-11-28T13:01:13.336873+0000","last_peered":"2023-11-28T13:01:13.336873+0000","last_clean":"2023-11-28T13:01:13.336873+0000","last_became_active":"2023-11-28T13:01:13.102706+0000","last_became_peered":"2023-11-28T13:01:13.102706+0000","last_unstale":"2023-11-28T13:01:13.336873+0000","last_undegraded":"2023-11-28T13:01:13.336873+0000","last_fullsized":"2023-11-28T13:01:13.336873+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:33.362111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536780+0000","last_change":"2023-11-28T13:01:13.094603+0000","last_active":"2023-11-28T13:01:14.536780+0000","last_peered":"2023-11-28T13:01:14.536780+0000","last_clean":"2023-11-28T13:01:14.536780+0000","last_became_active":"2023-11-28T13:01:13.094377+0000","last_became_peered":"2023-11-28T13:01:13.094377+0000","last_unstale":"2023-11-28T13:01:14.536780+0000","last_undegraded":"2023-11-28T13:01:14.536780+0000","last_fullsized":"2023-11-28T13:01:14.536780+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:25.095910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336852+0000","last_change":"2023-11-28T13:01:13.093425+0000","last_active":"2023-11-28T13:01:13.336852+0000","last_peered":"2023-11-28T13:01:13.336852+0000","last_clean":"2023-11-28T13:01:13.336852+0000","last_became_active":"2023-11-28T13:01:13.093128+0000","last_became_peered":"2023-11-28T13:01:13.093128+0000","last_unstale":"2023-11-28T13:01:13.336852+0000","last_undegraded":"2023-11-28T13:01:13.336852+0000","last_fullsized":"2023-11-28T13:01:13.336852+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:47.380562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344823+0000","last_change":"2023-11-28T13:01:13.095333+0000","last_active":"2023-11-28T13:01:13.344823+0000","last_peered":"2023-11-28T13:01:13.344823+0000","last_clean":"2023-11-28T13:01:13.344823+0000","last_became_active":"2023-11-28T13:01:13.095050+0000","last_became_peered":"2023-11-28T13:01:13.095050+0000","last_unstale":"2023-11-28T13:01:13.344823+0000","last_undegraded":"2023-11-28T13:01:13.344823+0000","last_fullsized":"2023-11-28T13:01:13.344823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:51:32.434761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503304+0000","last_change":"2023-11-28T13:01:13.105213+0000","last_active":"2023-11-28T13:01:13.503304+0000","last_peered":"2023-11-28T13:01:13.503304+0000","last_clean":"2023-11-28T13:01:13.503304+0000","last_became_active":"2023-11-28T13:01:13.104960+0000","last_became_peered":"2023-11-28T13:01:13.104960+0000","last_unstale":"2023-11-28T13:01:13.503304+0000","last_undegraded":"2023-11-28T13:01:13.503304+0000","last_fullsized":"2023-11-28T13:01:13.503304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337115+0000","last_change":"2023-11-28T13:01:13.102193+0000","last_active":"2023-11-28T13:01:13.337115+0000","last_peered":"2023-11-28T13:01:13.337115+0000","last_clean":"2023-11-28T13:01:13.337115+0000","last_became_active":"2023-11-28T13:01:13.101759+0000","last_became_peered":"2023-11-28T13:01:13.101759+0000","last_unstale":"2023-11-28T13:01:13.337115+0000","last_undegraded":"2023-11-28T13:01:13.337115+0000","last_fullsized":"2023-11-28T13:01:13.337115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:51.025874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536576+0000","last_change":"2023-11-28T13:01:13.098378+0000","last_active":"2023-11-28T13:01:14.536576+0000","last_peered":"2023-11-28T13:01:14.536576+0000","last_clean":"2023-11-28T13:01:14.536576+0000","last_became_active":"2023-11-28T13:01:13.098101+0000","last_became_peered":"2023-11-28T13:01:13.098101+0000","last_unstale":"2023-11-28T13:01:14.536576+0000","last_undegraded":"2023-11-28T13:01:14.536576+0000","last_fullsized":"2023-11-28T13:01:14.536576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:11.877653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336884+0000","last_change":"2023-11-28T13:01:13.094017+0000","last_active":"2023-11-28T13:01:13.336884+0000","last_peered":"2023-11-28T13:01:13.336884+0000","last_clean":"2023-11-28T13:01:13.336884+0000","last_became_active":"2023-11-28T13:01:13.093753+0000","last_became_peered":"2023-11-28T13:01:13.093753+0000","last_unstale":"2023-11-28T13:01:13.336884+0000","last_undegraded":"2023-11-28T13:01:13.336884+0000","last_fullsized":"2023-11-28T13:01:13.336884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:21.524121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089155+0000","last_change":"2023-11-28T13:01:13.103430+0000","last_active":"2023-11-28T13:01:14.089155+0000","last_peered":"2023-11-28T13:01:14.089155+0000","last_clean":"2023-11-28T13:01:14.089155+0000","last_became_active":"2023-11-28T13:01:13.102784+0000","last_became_peered":"2023-11-28T13:01:13.102784+0000","last_unstale":"2023-11-28T13:01:14.089155+0000","last_undegraded":"2023-11-28T13:01:14.089155+0000","last_fullsized":"2023-11-28T13:01:14.089155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:47.288068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336863+0000","last_change":"2023-11-28T13:01:13.092484+0000","last_active":"2023-11-28T13:01:13.336863+0000","last_peered":"2023-11-28T13:01:13.336863+0000","last_clean":"2023-11-28T13:01:13.336863+0000","last_became_active":"2023-11-28T13:01:13.092162+0000","last_became_peered":"2023-11-28T13:01:13.092162+0000","last_unstale":"2023-11-28T13:01:13.336863+0000","last_undegraded":"2023-11-28T13:01:13.336863+0000","last_fullsized":"2023-11-28T13:01:13.336863+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:03.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344864+0000","last_change":"2023-11-28T13:01:13.100818+0000","last_active":"2023-11-28T13:01:13.344864+0000","last_peered":"2023-11-28T13:01:13.344864+0000","last_clean":"2023-11-28T13:01:13.344864+0000","last_became_active":"2023-11-28T13:01:13.100421+0000","last_became_peered":"2023-11-28T13:01:13.100421+0000","last_unstale":"2023-11-28T13:01:13.344864+0000","last_undegraded":"2023-11-28T13:01:13.344864+0000","last_fullsized":"2023-11-28T13:01:13.344864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336841+0000","last_change":"2023-11-28T13:01:13.093478+0000","last_active":"2023-11-28T13:01:13.336841+0000","last_peered":"2023-11-28T13:01:13.336841+0000","last_clean":"2023-11-28T13:01:13.336841+0000","last_became_active":"2023-11-28T13:01:13.093194+0000","last_became_peered":"2023-11-28T13:01:13.093194+0000","last_unstale":"2023-11-28T13:01:13.336841+0000","last_undegraded":"2023-11-28T13:01:13.336841+0000","last_fullsized":"2023-11-28T13:01:13.336841+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:10.435728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336817+0000","last_change":"2023-11-28T13:01:13.088250+0000","last_active":"2023-11-28T13:01:13.336817+0000","last_peered":"2023-11-28T13:01:13.336817+0000","last_clean":"2023-11-28T13:01:13.336817+0000","last_became_active":"2023-11-28T13:01:13.087881+0000","last_became_peered":"2023-11-28T13:01:13.087881+0000","last_unstale":"2023-11-28T13:01:13.336817+0000","last_undegraded":"2023-11-28T13:01:13.336817+0000","last_fullsized":"2023-11-28T13:01:13.336817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:43.126488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337098+0000","last_change":"2023-11-28T13:01:13.102030+0000","last_active":"2023-11-28T13:01:13.337098+0000","last_peered":"2023-11-28T13:01:13.337098+0000","last_clean":"2023-11-28T13:01:13.337098+0000","last_became_active":"2023-11-28T13:01:13.101376+0000","last_became_peered":"2023-11-28T13:01:13.101376+0000","last_unstale":"2023-11-28T13:01:13.337098+0000","last_undegraded":"2023-11-28T13:01:13.337098+0000","last_fullsized":"2023-11-28T13:01:13.337098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088757+0000","last_change":"2023-11-28T13:01:13.095736+0000","last_active":"2023-11-28T13:01:14.088757+0000","last_peered":"2023-11-28T13:01:14.088757+0000","last_clean":"2023-11-28T13:01:14.088757+0000","last_became_active":"2023-11-28T13:01:13.095469+0000","last_became_peered":"2023-11-28T13:01:13.095469+0000","last_unstale":"2023-11-28T13:01:14.088757+0000","last_undegraded":"2023-11-28T13:01:14.088757+0000","last_fullsized":"2023-11-28T13:01:14.088757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:45.817559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337205+0000","last_change":"2023-11-28T13:01:13.105146+0000","last_active":"2023-11-28T13:01:13.337205+0000","last_peered":"2023-11-28T13:01:13.337205+0000","last_clean":"2023-11-28T13:01:13.337205+0000","last_became_active":"2023-11-28T13:01:13.104852+0000","last_became_peered":"2023-11-28T13:01:13.104852+0000","last_unstale":"2023-11-28T13:01:13.337205+0000","last_undegraded":"2023-11-28T13:01:13.337205+0000","last_fullsized":"2023-11-28T13:01:13.337205+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336897+0000","last_change":"2023-11-28T13:01:12.697633+0000","last_active":"2023-11-28T13:01:13.336897+0000","last_peered":"2023-11-28T13:01:13.336897+0000","last_clean":"2023-11-28T13:01:13.336897+0000","last_became_active":"2023-11-28T13:01:12.697137+0000","last_became_peered":"2023-11-28T13:01:12.697137+0000","last_unstale":"2023-11-28T13:01:13.336897+0000","last_undegraded":"2023-11-28T13:01:13.336897+0000","last_fullsized":"2023-11-28T13:01:13.336897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:31:59.601679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089117+0000","last_change":"2023-11-28T13:01:13.104026+0000","last_active":"2023-11-28T13:01:14.089117+0000","last_peered":"2023-11-28T13:01:14.089117+0000","last_clean":"2023-11-28T13:01:14.089117+0000","last_became_active":"2023-11-28T13:01:13.103779+0000","last_became_peered":"2023-11-28T13:01:13.103779+0000","last_unstale":"2023-11-28T13:01:14.089117+0000","last_undegraded":"2023-11-28T13:01:14.089117+0000","last_fullsized":"2023-11-28T13:01:14.089117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:13.669167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.096308+0000","last_change":"2023-11-28T13:01:12.700381+0000","last_active":"2023-11-28T13:01:13.096308+0000","last_peered":"2023-11-28T13:01:13.096308+0000","last_clean":"2023-11-28T13:01:13.096308+0000","last_became_active":"2023-11-28T13:01:12.700088+0000","last_became_peered":"2023-11-28T13:01:12.700088+0000","last_unstale":"2023-11-28T13:01:13.096308+0000","last_undegraded":"2023-11-28T13:01:13.096308+0000","last_fullsized":"2023-11-28T13:01:13.096308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503354+0000","last_change":"2023-11-28T13:01:13.104225+0000","last_active":"2023-11-28T13:01:13.503354+0000","last_peered":"2023-11-28T13:01:13.503354+0000","last_clean":"2023-11-28T13:01:13.503354+0000","last_became_active":"2023-11-28T13:01:13.103835+0000","last_became_peered":"2023-11-28T13:01:13.103835+0000","last_unstale":"2023-11-28T13:01:13.503354+0000","last_undegraded":"2023-11-28T13:01:13.503354+0000","last_fullsized":"2023-11-28T13:01:13.503354+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:00.632035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089210+0000","last_change":"2023-11-28T13:01:12.703691+0000","last_active":"2023-11-28T13:01:14.089210+0000","last_peered":"2023-11-28T13:01:14.089210+0000","last_clean":"2023-11-28T13:01:14.089210+0000","last_became_active":"2023-11-28T13:01:12.703398+0000","last_became_peered":"2023-11-28T13:01:12.703398+0000","last_unstale":"2023-11-28T13:01:14.089210+0000","last_undegraded":"2023-11-28T13:01:14.089210+0000","last_fullsized":"2023-11-28T13:01:14.089210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:24.093954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090818+0000","last_change":"2023-11-28T13:01:13.086251+0000","last_active":"2023-11-28T13:01:14.090818+0000","last_peered":"2023-11-28T13:01:14.090818+0000","last_clean":"2023-11-28T13:01:14.090818+0000","last_became_active":"2023-11-28T13:01:13.085269+0000","last_became_peered":"2023-11-28T13:01:13.085269+0000","last_unstale":"2023-11-28T13:01:14.090818+0000","last_undegraded":"2023-11-28T13:01:14.090818+0000","last_fullsized":"2023-11-28T13:01:14.090818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:04.381796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503344+0000","last_change":"2023-11-28T13:01:12.700070+0000","last_active":"2023-11-28T13:01:13.503344+0000","last_peered":"2023-11-28T13:01:13.503344+0000","last_clean":"2023-11-28T13:01:13.503344+0000","last_became_active":"2023-11-28T13:01:12.699788+0000","last_became_peered":"2023-11-28T13:01:12.699788+0000","last_unstale":"2023-11-28T13:01:13.503344+0000","last_undegraded":"2023-11-28T13:01:13.503344+0000","last_fullsized":"2023-11-28T13:01:13.503344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:37.142758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325299+0000","last_change":"2023-11-28T13:01:13.093841+0000","last_active":"2023-11-28T13:01:14.325299+0000","last_peered":"2023-11-28T13:01:14.325299+0000","last_clean":"2023-11-28T13:01:14.325299+0000","last_became_active":"2023-11-28T13:01:13.093572+0000","last_became_peered":"2023-11-28T13:01:13.093572+0000","last_unstale":"2023-11-28T13:01:14.325299+0000","last_undegraded":"2023-11-28T13:01:14.325299+0000","last_fullsized":"2023-11-28T13:01:14.325299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344851+0000","last_change":"2023-11-28T13:01:12.700665+0000","last_active":"2023-11-28T13:01:13.344851+0000","last_peered":"2023-11-28T13:01:13.344851+0000","last_clean":"2023-11-28T13:01:13.344851+0000","last_became_active":"2023-11-28T13:01:12.700285+0000","last_became_peered":"2023-11-28T13:01:12.700285+0000","last_unstale":"2023-11-28T13:01:13.344851+0000","last_undegraded":"2023-11-28T13:01:13.344851+0000","last_fullsized":"2023-11-28T13:01:13.344851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344764+0000","last_change":"2023-11-28T13:01:13.095159+0000","last_active":"2023-11-28T13:01:13.344764+0000","last_peered":"2023-11-28T13:01:13.344764+0000","last_clean":"2023-11-28T13:01:13.344764+0000","last_became_active":"2023-11-28T13:01:13.094857+0000","last_became_peered":"2023-11-28T13:01:13.094857+0000","last_unstale":"2023-11-28T13:01:13.344764+0000","last_undegraded":"2023-11-28T13:01:13.344764+0000","last_fullsized":"2023-11-28T13:01:13.344764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337170+0000","last_change":"2023-11-28T13:01:12.705296+0000","last_active":"2023-11-28T13:01:13.337170+0000","last_peered":"2023-11-28T13:01:13.337170+0000","last_clean":"2023-11-28T13:01:13.337170+0000","last_became_active":"2023-11-28T13:01:12.705013+0000","last_became_peered":"2023-11-28T13:01:12.705013+0000","last_unstale":"2023-11-28T13:01:13.337170+0000","last_undegraded":"2023-11-28T13:01:13.337170+0000","last_fullsized":"2023-11-28T13:01:13.337170+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:46.378606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503320+0000","last_change":"2023-11-28T13:01:13.094747+0000","last_active":"2023-11-28T13:01:13.503320+0000","last_peered":"2023-11-28T13:01:13.503320+0000","last_clean":"2023-11-28T13:01:13.503320+0000","last_became_active":"2023-11-28T13:01:13.094318+0000","last_became_peered":"2023-11-28T13:01:13.094318+0000","last_unstale":"2023-11-28T13:01:13.503320+0000","last_undegraded":"2023-11-28T13:01:13.503320+0000","last_fullsized":"2023-11-28T13:01:13.503320+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:27.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089212+0000","last_change":"2023-11-28T13:01:12.706058+0000","last_active":"2023-11-28T13:01:14.089212+0000","last_peered":"2023-11-28T13:01:14.089212+0000","last_clean":"2023-11-28T13:01:14.089212+0000","last_became_active":"2023-11-28T13:01:12.705853+0000","last_became_peered":"2023-11-28T13:01:12.705853+0000","last_unstale":"2023-11-28T13:01:14.089212+0000","last_undegraded":"2023-11-28T13:01:14.089212+0000","last_fullsized":"2023-11-28T13:01:14.089212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:10.875697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090633+0000","last_change":"2023-11-28T13:01:13.099221+0000","last_active":"2023-11-28T13:01:14.090633+0000","last_peered":"2023-11-28T13:01:14.090633+0000","last_clean":"2023-11-28T13:01:14.090633+0000","last_became_active":"2023-11-28T13:01:13.098525+0000","last_became_peered":"2023-11-28T13:01:13.098525+0000","last_unstale":"2023-11-28T13:01:14.090633+0000","last_undegraded":"2023-11-28T13:01:14.090633+0000","last_fullsized":"2023-11-28T13:01:14.090633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:01.070045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503253+0000","last_change":"2023-11-28T13:01:12.700642+0000","last_active":"2023-11-28T13:01:13.503253+0000","last_peered":"2023-11-28T13:01:13.503253+0000","last_clean":"2023-11-28T13:01:13.503253+0000","last_became_active":"2023-11-28T13:01:12.700381+0000","last_became_peered":"2023-11-28T13:01:12.700381+0000","last_unstale":"2023-11-28T13:01:13.503253+0000","last_undegraded":"2023-11-28T13:01:13.503253+0000","last_fullsized":"2023-11-28T13:01:13.503253+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:37.268919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536728+0000","last_change":"2023-11-28T13:01:13.107217+0000","last_active":"2023-11-28T13:01:14.536728+0000","last_peered":"2023-11-28T13:01:14.536728+0000","last_clean":"2023-11-28T13:01:14.536728+0000","last_became_active":"2023-11-28T13:01:13.106967+0000","last_became_peered":"2023-11-28T13:01:13.106967+0000","last_unstale":"2023-11-28T13:01:14.536728+0000","last_undegraded":"2023-11-28T13:01:14.536728+0000","last_fullsized":"2023-11-28T13:01:14.536728+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:01.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090884+0000","last_change":"2023-11-28T13:01:12.700536+0000","last_active":"2023-11-28T13:01:14.090884+0000","last_peered":"2023-11-28T13:01:14.090884+0000","last_clean":"2023-11-28T13:01:14.090884+0000","last_became_active":"2023-11-28T13:01:12.699782+0000","last_became_peered":"2023-11-28T13:01:12.699782+0000","last_unstale":"2023-11-28T13:01:14.090884+0000","last_undegraded":"2023-11-28T13:01:14.090884+0000","last_fullsized":"2023-11-28T13:01:14.090884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:17.146583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337238+0000","last_change":"2023-11-28T13:01:13.103462+0000","last_active":"2023-11-28T13:01:13.337238+0000","last_peered":"2023-11-28T13:01:13.337238+0000","last_clean":"2023-11-28T13:01:13.337238+0000","last_became_active":"2023-11-28T13:01:13.102855+0000","last_became_peered":"2023-11-28T13:01:13.102855+0000","last_unstale":"2023-11-28T13:01:13.337238+0000","last_undegraded":"2023-11-28T13:01:13.337238+0000","last_fullsized":"2023-11-28T13:01:13.337238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.987104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344922+0000","last_change":"2023-11-28T13:01:12.703891+0000","last_active":"2023-11-28T13:01:13.344922+0000","last_peered":"2023-11-28T13:01:13.344922+0000","last_clean":"2023-11-28T13:01:13.344922+0000","last_became_active":"2023-11-28T13:01:12.703642+0000","last_became_peered":"2023-11-28T13:01:12.703642+0000","last_unstale":"2023-11-28T13:01:13.344922+0000","last_undegraded":"2023-11-28T13:01:13.344922+0000","last_fullsized":"2023-11-28T13:01:13.344922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:32.360155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325431+0000","last_change":"2023-11-28T13:01:13.096906+0000","last_active":"2023-11-28T13:01:14.325431+0000","last_peered":"2023-11-28T13:01:14.325431+0000","last_clean":"2023-11-28T13:01:14.325431+0000","last_became_active":"2023-11-28T13:01:13.096606+0000","last_became_peered":"2023-11-28T13:01:13.096606+0000","last_unstale":"2023-11-28T13:01:14.325431+0000","last_undegraded":"2023-11-28T13:01:14.325431+0000","last_fullsized":"2023-11-28T13:01:14.325431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:57.375417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344876+0000","last_change":"2023-11-28T13:01:12.704636+0000","last_active":"2023-11-28T13:01:13.344876+0000","last_peered":"2023-11-28T13:01:13.344876+0000","last_clean":"2023-11-28T13:01:13.344876+0000","last_became_active":"2023-11-28T13:01:12.704357+0000","last_became_peered":"2023-11-28T13:01:12.704357+0000","last_unstale":"2023-11-28T13:01:13.344876+0000","last_undegraded":"2023-11-28T13:01:13.344876+0000","last_fullsized":"2023-11-28T13:01:13.344876+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503389+0000","last_change":"2023-11-28T13:01:13.096971+0000","last_active":"2023-11-28T13:01:13.503389+0000","last_peered":"2023-11-28T13:01:13.503389+0000","last_clean":"2023-11-28T13:01:13.503389+0000","last_became_active":"2023-11-28T13:01:13.096698+0000","last_became_peered":"2023-11-28T13:01:13.096698+0000","last_unstale":"2023-11-28T13:01:13.503389+0000","last_undegraded":"2023-11-28T13:01:13.503389+0000","last_fullsized":"2023-11-28T13:01:13.503389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090728+0000","last_change":"2023-11-28T13:01:12.704846+0000","last_active":"2023-11-28T13:01:14.090728+0000","last_peered":"2023-11-28T13:01:14.090728+0000","last_clean":"2023-11-28T13:01:14.090728+0000","last_became_active":"2023-11-28T13:01:12.704605+0000","last_became_peered":"2023-11-28T13:01:12.704605+0000","last_unstale":"2023-11-28T13:01:14.090728+0000","last_undegraded":"2023-11-28T13:01:14.090728+0000","last_fullsized":"2023-11-28T13:01:14.090728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:43.923726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325464+0000","last_change":"2023-11-28T13:01:13.101854+0000","last_active":"2023-11-28T13:01:14.325464+0000","last_peered":"2023-11-28T13:01:14.325464+0000","last_clean":"2023-11-28T13:01:14.325464+0000","last_became_active":"2023-11-28T13:01:13.101608+0000","last_became_peered":"2023-11-28T13:01:13.101608+0000","last_unstale":"2023-11-28T13:01:14.325464+0000","last_undegraded":"2023-11-28T13:01:14.325464+0000","last_fullsized":"2023-11-28T13:01:14.325464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:30.913406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090672+0000","last_change":"2023-11-28T13:01:12.700930+0000","last_active":"2023-11-28T13:01:14.090672+0000","last_peered":"2023-11-28T13:01:14.090672+0000","last_clean":"2023-11-28T13:01:14.090672+0000","last_became_active":"2023-11-28T13:01:12.700670+0000","last_became_peered":"2023-11-28T13:01:12.700670+0000","last_unstale":"2023-11-28T13:01:14.090672+0000","last_undegraded":"2023-11-28T13:01:14.090672+0000","last_fullsized":"2023-11-28T13:01:14.090672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090509+0000","last_change":"2023-11-28T13:01:13.093993+0000","last_active":"2023-11-28T13:01:14.090509+0000","last_peered":"2023-11-28T13:01:14.090509+0000","last_clean":"2023-11-28T13:01:14.090509+0000","last_became_active":"2023-11-28T13:01:13.093750+0000","last_became_peered":"2023-11-28T13:01:13.093750+0000","last_unstale":"2023-11-28T13:01:14.090509+0000","last_undegraded":"2023-11-28T13:01:14.090509+0000","last_fullsized":"2023-11-28T13:01:14.090509+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:47.159742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088952+0000","last_change":"2023-11-28T13:01:12.706087+0000","last_active":"2023-11-28T13:01:14.088952+0000","last_peered":"2023-11-28T13:01:14.088952+0000","last_clean":"2023-11-28T13:01:14.088952+0000","last_became_active":"2023-11-28T13:01:12.705800+0000","last_became_peered":"2023-11-28T13:01:12.705800+0000","last_unstale":"2023-11-28T13:01:14.088952+0000","last_undegraded":"2023-11-28T13:01:14.088952+0000","last_fullsized":"2023-11-28T13:01:14.088952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:04.666677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337151+0000","last_change":"2023-11-28T13:01:13.101116+0000","last_active":"2023-11-28T13:01:13.337151+0000","last_peered":"2023-11-28T13:01:13.337151+0000","last_clean":"2023-11-28T13:01:13.337151+0000","last_became_active":"2023-11-28T13:01:13.100510+0000","last_became_peered":"2023-11-28T13:01:13.100510+0000","last_unstale":"2023-11-28T13:01:13.337151+0000","last_undegraded":"2023-11-28T13:01:13.337151+0000","last_fullsized":"2023-11-28T13:01:13.337151+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:42.989233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336800+0000","last_change":"2023-11-28T13:01:12.702749+0000","last_active":"2023-11-28T13:01:13.336800+0000","last_peered":"2023-11-28T13:01:13.336800+0000","last_clean":"2023-11-28T13:01:13.336800+0000","last_became_active":"2023-11-28T13:01:12.702450+0000","last_became_peered":"2023-11-28T13:01:12.702450+0000","last_unstale":"2023-11-28T13:01:13.336800+0000","last_undegraded":"2023-11-28T13:01:13.336800+0000","last_fullsized":"2023-11-28T13:01:13.336800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:38.139538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090531+0000","last_change":"2023-11-28T13:01:13.095487+0000","last_active":"2023-11-28T13:01:14.090531+0000","last_peered":"2023-11-28T13:01:14.090531+0000","last_clean":"2023-11-28T13:01:14.090531+0000","last_became_active":"2023-11-28T13:01:13.095222+0000","last_became_peered":"2023-11-28T13:01:13.095222+0000","last_unstale":"2023-11-28T13:01:14.090531+0000","last_undegraded":"2023-11-28T13:01:14.090531+0000","last_fullsized":"2023-11-28T13:01:14.090531+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:57.780846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337056+0000","last_change":"2023-11-28T13:01:12.704714+0000","last_active":"2023-11-28T13:01:13.337056+0000","last_peered":"2023-11-28T13:01:13.337056+0000","last_clean":"2023-11-28T13:01:13.337056+0000","last_became_active":"2023-11-28T13:01:12.704422+0000","last_became_peered":"2023-11-28T13:01:12.704422+0000","last_unstale":"2023-11-28T13:01:13.337056+0000","last_undegraded":"2023-11-28T13:01:13.337056+0000","last_fullsized":"2023-11-28T13:01:13.337056+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:42.124531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090726+0000","last_change":"2023-11-28T13:01:13.088050+0000","last_active":"2023-11-28T13:01:14.090726+0000","last_peered":"2023-11-28T13:01:14.090726+0000","last_clean":"2023-11-28T13:01:14.090726+0000","last_became_active":"2023-11-28T13:01:13.085247+0000","last_became_peered":"2023-11-28T13:01:13.085247+0000","last_unstale":"2023-11-28T13:01:14.090726+0000","last_undegraded":"2023-11-28T13:01:14.090726+0000","last_fullsized":"2023-11-28T13:01:14.090726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:39.035787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344953+0000","last_change":"2023-11-28T13:01:12.702173+0000","last_active":"2023-11-28T13:01:13.344953+0000","last_peered":"2023-11-28T13:01:13.344953+0000","last_clean":"2023-11-28T13:01:13.344953+0000","last_became_active":"2023-11-28T13:01:12.701243+0000","last_became_peered":"2023-11-28T13:01:12.701243+0000","last_unstale":"2023-11-28T13:01:13.344953+0000","last_undegraded":"2023-11-28T13:01:13.344953+0000","last_fullsized":"2023-11-28T13:01:13.344953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344898+0000","last_change":"2023-11-28T13:01:13.099301+0000","last_active":"2023-11-28T13:01:13.344898+0000","last_peered":"2023-11-28T13:01:13.344898+0000","last_clean":"2023-11-28T13:01:13.344898+0000","last_became_active":"2023-11-28T13:01:13.099017+0000","last_became_peered":"2023-11-28T13:01:13.099017+0000","last_unstale":"2023-11-28T13:01:13.344898+0000","last_undegraded":"2023-11-28T13:01:13.344898+0000","last_fullsized":"2023-11-28T13:01:13.344898+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:26:17.086086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503377+0000","last_change":"2023-11-28T13:01:12.706078+0000","last_active":"2023-11-28T13:01:13.503377+0000","last_peered":"2023-11-28T13:01:13.503377+0000","last_clean":"2023-11-28T13:01:13.503377+0000","last_became_active":"2023-11-28T13:01:12.705694+0000","last_became_peered":"2023-11-28T13:01:12.705694+0000","last_unstale":"2023-11-28T13:01:13.503377+0000","last_undegraded":"2023-11-28T13:01:13.503377+0000","last_fullsized":"2023-11-28T13:01:13.503377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:33:41.987277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503364+0000","last_change":"2023-11-28T13:01:13.098692+0000","last_active":"2023-11-28T13:01:13.503364+0000","last_peered":"2023-11-28T13:01:13.503364+0000","last_clean":"2023-11-28T13:01:13.503364+0000","last_became_active":"2023-11-28T13:01:13.098426+0000","last_became_peered":"2023-11-28T13:01:13.098426+0000","last_unstale":"2023-11-28T13:01:13.503364+0000","last_undegraded":"2023-11-28T13:01:13.503364+0000","last_fullsized":"2023-11-28T13:01:13.503364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:56.710136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536748+0000","last_change":"2023-11-28T13:01:12.701155+0000","last_active":"2023-11-28T13:01:14.536748+0000","last_peered":"2023-11-28T13:01:14.536748+0000","last_clean":"2023-11-28T13:01:14.536748+0000","last_became_active":"2023-11-28T13:01:12.700897+0000","last_became_peered":"2023-11-28T13:01:12.700897+0000","last_unstale":"2023-11-28T13:01:14.536748+0000","last_undegraded":"2023-11-28T13:01:14.536748+0000","last_fullsized":"2023-11-28T13:01:14.536748+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:57.063655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536807+0000","last_change":"2023-11-28T13:01:13.094885+0000","last_active":"2023-11-28T13:01:14.536807+0000","last_peered":"2023-11-28T13:01:14.536807+0000","last_clean":"2023-11-28T13:01:14.536807+0000","last_became_active":"2023-11-28T13:01:13.094027+0000","last_became_peered":"2023-11-28T13:01:13.094027+0000","last_unstale":"2023-11-28T13:01:14.536807+0000","last_undegraded":"2023-11-28T13:01:14.536807+0000","last_fullsized":"2023-11-28T13:01:14.536807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:43.321762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.089094+0000","last_change":"2023-11-28T13:01:12.703499+0000","last_active":"2023-11-28T13:01:14.089094+0000","last_peered":"2023-11-28T13:01:14.089094+0000","last_clean":"2023-11-28T13:01:14.089094+0000","last_became_active":"2023-11-28T13:01:12.703271+0000","last_became_peered":"2023-11-28T13:01:12.703271+0000","last_unstale":"2023-11-28T13:01:14.089094+0000","last_undegraded":"2023-11-28T13:01:14.089094+0000","last_fullsized":"2023-11-28T13:01:14.089094+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:03:30.119943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337037+0000","last_change":"2023-11-28T13:01:13.101678+0000","last_active":"2023-11-28T13:01:13.337037+0000","last_peered":"2023-11-28T13:01:13.337037+0000","last_clean":"2023-11-28T13:01:13.337037+0000","last_became_active":"2023-11-28T13:01:13.101386+0000","last_became_peered":"2023-11-28T13:01:13.101386+0000","last_unstale":"2023-11-28T13:01:13.337037+0000","last_undegraded":"2023-11-28T13:01:13.337037+0000","last_fullsized":"2023-11-28T13:01:13.337037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:51.226552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536679+0000","last_change":"2023-11-28T13:01:12.697001+0000","last_active":"2023-11-28T13:01:14.536679+0000","last_peered":"2023-11-28T13:01:14.536679+0000","last_clean":"2023-11-28T13:01:14.536679+0000","last_became_active":"2023-11-28T13:01:12.696739+0000","last_became_peered":"2023-11-28T13:01:12.696739+0000","last_unstale":"2023-11-28T13:01:14.536679+0000","last_undegraded":"2023-11-28T13:01:14.536679+0000","last_fullsized":"2023-11-28T13:01:14.536679+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:34:37.438564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325346+0000","last_change":"2023-11-28T13:01:13.096745+0000","last_active":"2023-11-28T13:01:14.325346+0000","last_peered":"2023-11-28T13:01:14.325346+0000","last_clean":"2023-11-28T13:01:14.325346+0000","last_became_active":"2023-11-28T13:01:13.096461+0000","last_became_peered":"2023-11-28T13:01:13.096461+0000","last_unstale":"2023-11-28T13:01:14.325346+0000","last_undegraded":"2023-11-28T13:01:14.325346+0000","last_fullsized":"2023-11-28T13:01:14.325346+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.987104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536639+0000","last_change":"2023-11-28T13:01:12.698523+0000","last_active":"2023-11-28T13:01:14.536639+0000","last_peered":"2023-11-28T13:01:14.536639+0000","last_clean":"2023-11-28T13:01:14.536639+0000","last_became_active":"2023-11-28T13:01:12.698275+0000","last_became_peered":"2023-11-28T13:01:12.698275+0000","last_unstale":"2023-11-28T13:01:14.536639+0000","last_undegraded":"2023-11-28T13:01:14.536639+0000","last_fullsized":"2023-11-28T13:01:14.536639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:02.254485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344792+0000","last_change":"2023-11-28T13:01:13.099007+0000","last_active":"2023-11-28T13:01:13.344792+0000","last_peered":"2023-11-28T13:01:13.344792+0000","last_clean":"2023-11-28T13:01:13.344792+0000","last_became_active":"2023-11-28T13:01:13.098673+0000","last_became_peered":"2023-11-28T13:01:13.098673+0000","last_unstale":"2023-11-28T13:01:13.344792+0000","last_undegraded":"2023-11-28T13:01:13.344792+0000","last_fullsized":"2023-11-28T13:01:13.344792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337132+0000","last_change":"2023-11-28T13:01:12.706237+0000","last_active":"2023-11-28T13:01:13.337132+0000","last_peered":"2023-11-28T13:01:13.337132+0000","last_clean":"2023-11-28T13:01:13.337132+0000","last_became_active":"2023-11-28T13:01:12.705981+0000","last_became_peered":"2023-11-28T13:01:12.705981+0000","last_unstale":"2023-11-28T13:01:13.337132+0000","last_undegraded":"2023-11-28T13:01:13.337132+0000","last_fullsized":"2023-11-28T13:01:13.337132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:09.433771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090578+0000","last_change":"2023-11-28T13:01:13.099382+0000","last_active":"2023-11-28T13:01:14.090578+0000","last_peered":"2023-11-28T13:01:14.090578+0000","last_clean":"2023-11-28T13:01:14.090578+0000","last_became_active":"2023-11-28T13:01:13.099008+0000","last_became_peered":"2023-11-28T13:01:13.099008+0000","last_unstale":"2023-11-28T13:01:14.090578+0000","last_undegraded":"2023-11-28T13:01:14.090578+0000","last_fullsized":"2023-11-28T13:01:14.090578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:35.600930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.536606+0000","last_change":"2023-11-28T13:01:12.696728+0000","last_active":"2023-11-28T13:01:14.536606+0000","last_peered":"2023-11-28T13:01:14.536606+0000","last_clean":"2023-11-28T13:01:14.536606+0000","last_became_active":"2023-11-28T13:01:12.696428+0000","last_became_peered":"2023-11-28T13:01:12.696428+0000","last_unstale":"2023-11-28T13:01:14.536606+0000","last_undegraded":"2023-11-28T13:01:14.536606+0000","last_fullsized":"2023-11-28T13:01:14.536606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:28.781799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.325302+0000","last_change":"2023-11-28T13:01:13.099016+0000","last_active":"2023-11-28T13:01:14.325302+0000","last_peered":"2023-11-28T13:01:14.325302+0000","last_clean":"2023-11-28T13:01:14.325302+0000","last_became_active":"2023-11-28T13:01:13.098565+0000","last_became_peered":"2023-11-28T13:01:13.098565+0000","last_unstale":"2023-11-28T13:01:14.325302+0000","last_undegraded":"2023-11-28T13:01:14.325302+0000","last_fullsized":"2023-11-28T13:01:14.325302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:09:05.863061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.089564+0000","last_change":"2023-11-28T13:01:12.698962+0000","last_active":"2023-11-28T13:01:13.089564+0000","last_peered":"2023-11-28T13:01:13.089564+0000","last_clean":"2023-11-28T13:01:13.089564+0000","last_became_active":"2023-11-28T13:01:12.698648+0000","last_became_peered":"2023-11-28T13:01:12.698648+0000","last_unstale":"2023-11-28T13:01:13.089564+0000","last_undegraded":"2023-11-28T13:01:13.089564+0000","last_fullsized":"2023-11-28T13:01:13.089564+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:20:29.037232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091534+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:01:13.091534+0000","last_peered":"2023-11-28T13:01:13.091534+0000","last_clean":"2023-11-28T13:01:13.091534+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:01:13.091534+0000","last_undegraded":"2023-11-28T13:01:13.091534+0000","last_fullsized":"2023-11-28T13:01:13.091534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090805+0000","last_change":"2023-11-28T13:01:13.098873+0000","last_active":"2023-11-28T13:01:14.090805+0000","last_peered":"2023-11-28T13:01:14.090805+0000","last_clean":"2023-11-28T13:01:14.090805+0000","last_became_active":"2023-11-28T13:01:13.098618+0000","last_became_peered":"2023-11-28T13:01:13.098618+0000","last_unstale":"2023-11-28T13:01:14.090805+0000","last_undegraded":"2023-11-28T13:01:14.090805+0000","last_fullsized":"2023-11-28T13:01:14.090805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:49.374140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.089151+0000","last_change":"2023-11-28T13:01:12.699988+0000","last_active":"2023-11-28T13:01:13.089151+0000","last_peered":"2023-11-28T13:01:13.089151+0000","last_clean":"2023-11-28T13:01:13.089151+0000","last_became_active":"2023-11-28T13:01:12.699721+0000","last_became_peered":"2023-11-28T13:01:12.699721+0000","last_unstale":"2023-11-28T13:01:13.089151+0000","last_undegraded":"2023-11-28T13:01:13.089151+0000","last_fullsized":"2023-11-28T13:01:13.089151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:09.268534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.085701+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:01:13.085701+0000","last_peered":"2023-11-28T13:01:13.085701+0000","last_clean":"2023-11-28T13:01:13.085701+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:01:13.085701+0000","last_undegraded":"2023-11-28T13:01:13.085701+0000","last_fullsized":"2023-11-28T13:01:13.085701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337254+0000","last_change":"2023-11-28T13:01:13.103633+0000","last_active":"2023-11-28T13:01:13.337254+0000","last_peered":"2023-11-28T13:01:13.337254+0000","last_clean":"2023-11-28T13:01:13.337254+0000","last_became_active":"2023-11-28T13:01:13.103198+0000","last_became_peered":"2023-11-28T13:01:13.103198+0000","last_unstale":"2023-11-28T13:01:13.337254+0000","last_undegraded":"2023-11-28T13:01:13.337254+0000","last_fullsized":"2023-11-28T13:01:13.337254+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:12.912302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:15.426201+0000","last_change":"2023-11-28T13:01:12.704388+0000","last_active":"2023-11-28T13:01:15.426201+0000","last_peered":"2023-11-28T13:01:15.426201+0000","last_clean":"2023-11-28T13:01:15.426201+0000","last_became_active":"2023-11-28T13:01:12.704082+0000","last_became_peered":"2023-11-28T13:01:12.704082+0000","last_unstale":"2023-11-28T13:01:15.426201+0000","last_undegraded":"2023-11-28T13:01:15.426201+0000","last_fullsized":"2023-11-28T13:01:15.426201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:08.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.083269+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:01:13.083269+0000","last_peered":"2023-11-28T13:01:13.083269+0000","last_clean":"2023-11-28T13:01:13.083269+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:01:13.083269+0000","last_undegraded":"2023-11-28T13:01:13.083269+0000","last_fullsized":"2023-11-28T13:01:13.083269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344935+0000","last_change":"2023-11-28T13:01:13.105495+0000","last_active":"2023-11-28T13:01:13.344935+0000","last_peered":"2023-11-28T13:01:13.344935+0000","last_clean":"2023-11-28T13:01:13.344935+0000","last_became_active":"2023-11-28T13:01:13.103141+0000","last_became_peered":"2023-11-28T13:01:13.103141+0000","last_unstale":"2023-11-28T13:01:13.344935+0000","last_undegraded":"2023-11-28T13:01:13.344935+0000","last_fullsized":"2023-11-28T13:01:13.344935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:51:08.241448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337222+0000","last_change":"2023-11-28T13:01:12.704702+0000","last_active":"2023-11-28T13:01:13.337222+0000","last_peered":"2023-11-28T13:01:13.337222+0000","last_clean":"2023-11-28T13:01:13.337222+0000","last_became_active":"2023-11-28T13:01:12.704416+0000","last_became_peered":"2023-11-28T13:01:12.704416+0000","last_unstale":"2023-11-28T13:01:13.337222+0000","last_undegraded":"2023-11-28T13:01:13.337222+0000","last_fullsized":"2023-11-28T13:01:13.337222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:02.624167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.087401+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:01:13.087401+0000","last_peered":"2023-11-28T13:01:13.087401+0000","last_clean":"2023-11-28T13:01:13.087401+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:01:13.087401+0000","last_undegraded":"2023-11-28T13:01:13.087401+0000","last_fullsized":"2023-11-28T13:01:13.087401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.090649+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:01:13.090649+0000","last_peered":"2023-11-28T13:01:13.090649+0000","last_clean":"2023-11-28T13:01:13.090649+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:01:13.090649+0000","last_undegraded":"2023-11-28T13:01:13.090649+0000","last_fullsized":"2023-11-28T13:01:13.090649+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088962+0000","last_change":"2023-11-28T13:01:13.104062+0000","last_active":"2023-11-28T13:01:14.088962+0000","last_peered":"2023-11-28T13:01:14.088962+0000","last_clean":"2023-11-28T13:01:14.088962+0000","last_became_active":"2023-11-28T13:01:13.103815+0000","last_became_peered":"2023-11-28T13:01:13.103815+0000","last_unstale":"2023-11-28T13:01:14.088962+0000","last_undegraded":"2023-11-28T13:01:14.088962+0000","last_fullsized":"2023-11-28T13:01:14.088962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:09.350952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.344970+0000","last_change":"2023-11-28T13:01:12.706680+0000","last_active":"2023-11-28T13:01:13.344970+0000","last_peered":"2023-11-28T13:01:13.344970+0000","last_clean":"2023-11-28T13:01:13.344970+0000","last_became_active":"2023-11-28T13:01:12.706387+0000","last_became_peered":"2023-11-28T13:01:12.706387+0000","last_unstale":"2023-11-28T13:01:13.344970+0000","last_undegraded":"2023-11-28T13:01:13.344970+0000","last_fullsized":"2023-11-28T13:01:13.344970+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:44.238089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.090734+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:01:13.090734+0000","last_peered":"2023-11-28T13:01:13.090734+0000","last_clean":"2023-11-28T13:01:13.090734+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:01:13.090734+0000","last_undegraded":"2023-11-28T13:01:13.090734+0000","last_fullsized":"2023-11-28T13:01:13.090734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.088854+0000","last_change":"2023-11-28T13:01:13.100011+0000","last_active":"2023-11-28T13:01:14.088854+0000","last_peered":"2023-11-28T13:01:14.088854+0000","last_clean":"2023-11-28T13:01:14.088854+0000","last_became_active":"2023-11-28T13:01:13.099738+0000","last_became_peered":"2023-11-28T13:01:13.099738+0000","last_unstale":"2023-11-28T13:01:14.088854+0000","last_undegraded":"2023-11-28T13:01:14.088854+0000","last_fullsized":"2023-11-28T13:01:14.088854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:52.246680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.336830+0000","last_change":"2023-11-28T13:01:12.694952+0000","last_active":"2023-11-28T13:01:13.336830+0000","last_peered":"2023-11-28T13:01:13.336830+0000","last_clean":"2023-11-28T13:01:13.336830+0000","last_became_active":"2023-11-28T13:01:12.694694+0000","last_became_peered":"2023-11-28T13:01:12.694694+0000","last_unstale":"2023-11-28T13:01:13.336830+0000","last_undegraded":"2023-11-28T13:01:13.336830+0000","last_fullsized":"2023-11-28T13:01:13.336830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:54.638737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.091695+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:01:13.091695+0000","last_peered":"2023-11-28T13:01:13.091695+0000","last_clean":"2023-11-28T13:01:13.091695+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:01:13.091695+0000","last_undegraded":"2023-11-28T13:01:13.091695+0000","last_fullsized":"2023-11-28T13:01:13.091695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503332+0000","last_change":"2023-11-28T13:01:13.099302+0000","last_active":"2023-11-28T13:01:13.503332+0000","last_peered":"2023-11-28T13:01:13.503332+0000","last_clean":"2023-11-28T13:01:13.503332+0000","last_became_active":"2023-11-28T13:01:13.099047+0000","last_became_peered":"2023-11-28T13:01:13.099047+0000","last_unstale":"2023-11-28T13:01:13.503332+0000","last_undegraded":"2023-11-28T13:01:13.503332+0000","last_fullsized":"2023-11-28T13:01:13.503332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:00.504520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.503282+0000","last_change":"2023-11-28T13:01:12.700645+0000","last_active":"2023-11-28T13:01:13.503282+0000","last_peered":"2023-11-28T13:01:13.503282+0000","last_clean":"2023-11-28T13:01:13.503282+0000","last_became_active":"2023-11-28T13:01:12.700349+0000","last_became_peered":"2023-11-28T13:01:12.700349+0000","last_unstale":"2023-11-28T13:01:13.503282+0000","last_undegraded":"2023-11-28T13:01:13.503282+0000","last_fullsized":"2023-11-28T13:01:13.503282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:50.224596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.086932+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:01:13.086932+0000","last_peered":"2023-11-28T13:01:13.086932+0000","last_clean":"2023-11-28T13:01:13.086932+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:01:13.086932+0000","last_undegraded":"2023-11-28T13:01:13.086932+0000","last_fullsized":"2023-11-28T13:01:13.086932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.337189+0000","last_change":"2023-11-28T13:01:13.101838+0000","last_active":"2023-11-28T13:01:13.337189+0000","last_peered":"2023-11-28T13:01:13.337189+0000","last_clean":"2023-11-28T13:01:13.337189+0000","last_became_active":"2023-11-28T13:01:13.101565+0000","last_became_peered":"2023-11-28T13:01:13.101565+0000","last_unstale":"2023-11-28T13:01:13.337189+0000","last_undegraded":"2023-11-28T13:01:13.337189+0000","last_fullsized":"2023-11-28T13:01:13.337189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:12.679463+0000","last_clean_scrub_stamp":"2023-11-28T13:01:12.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:08.927902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:14.090444+0000","last_change":"2023-11-28T13:01:12.699934+0000","last_active":"2023-11-28T13:01:14.090444+0000","last_peered":"2023-11-28T13:01:14.090444+0000","last_clean":"2023-11-28T13:01:14.090444+0000","last_became_active":"2023-11-28T13:01:12.699617+0000","last_became_peered":"2023-11-28T13:01:12.699617+0000","last_unstale":"2023-11-28T13:01:14.090444+0000","last_undegraded":"2023-11-28T13:01:14.090444+0000","last_fullsized":"2023-11-28T13:01:14.090444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:11.677507+0000","last_clean_scrub_stamp":"2023-11-28T13:01:11.677507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:02.801702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:13.081514+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:01:13.081514+0000","last_peered":"2023-11-28T13:01:13.081514+0000","last_clean":"2023-11-28T13:01:13.081514+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:01:13.081514+0000","last_undegraded":"2023-11-28T13:01:13.081514+0000","last_fullsized":"2023-11-28T13:01:13.081514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":184228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.218,"5min":8.218,"15min":8.218},"last":0.27700000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.29499999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":13.384,"5min":13.384,"15min":13.384},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.226,"5min":5.226,"15min":5.226},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.379},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.48399999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.3209999999999997,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.1029999999999998,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.3849999999999998,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.58099999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8376,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363464,"statfs":{"total":96636764160,"available":96628187136,"internally_reserved":0,"allocated":442368,"data_stored":182716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.379}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.29499999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.58099999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.71099999999999997}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.64500000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8636,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363204,"statfs":{"total":96636764160,"available":96627920896,"internally_reserved":0,"allocated":446464,"data_stored":183234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.37},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.438}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.61699999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9592,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362248,"statfs":{"total":96636764160,"available":96626941952,"internally_reserved":0,"allocated":1032192,"data_stored":772279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":14.997999999999999,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":19.045999999999999,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.377}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":18.995000000000001,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":19.02,"5min":19.02,"15min":19.02},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.1539999999999999,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":18.957999999999998,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.59299999999999997}]}]},{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":1048576,"data_stored":774632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.438},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.39700000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.41499999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.499}]},{"osd":3,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.53600000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.628}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.67800000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.61699999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8508,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363332,"statfs":{"total":96636764160,"available":96628051968,"internally_reserved":0,"allocated":446464,"data_stored":182758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.443}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.59299999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.499}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.625},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61299999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8440,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363400,"statfs":{"total":96636764160,"available":96628121600,"internally_reserved":0,"allocated":442368,"data_stored":183212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46700000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.505},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.60299999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.63}]}]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8504,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363336,"statfs":{"total":96636764160,"available":96628056064,"internally_reserved":0,"allocated":442368,"data_stored":182736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.254},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.31}]},{"osd":3,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.32100000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.36799999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.376}]},{"osd":6,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.45000000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.432}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:01:22.053 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:01:22.053 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:01:22.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.214+0000 7fee321b6700 1 Processor -- start 2023-11-28T13:01:22.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.214+0000 7fee321b6700 1 -- start start 2023-11-28T13:01:22.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee321b6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c071040 0x7fee2c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee321b6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fee2c06dee0 0x7fee2c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee321b6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee2c06cdc0 0x7fee2c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee321b6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c071950 con 0x7fee2c06cdc0 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee321b6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c15d060 con 0x7fee2c06dee0 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee321b6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c15d340 con 0x7fee2c071040 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee2bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee2c06cdc0 0x7fee2c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee2bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fee2c06cdc0 0x7fee2c06d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40336/0 (socket says 172.21.15.153:40336) 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee2bfff700 1 -- 172.21.15.153:0/2495996098 learned_addr learned my addr 172.21.15.153:0/2495996098 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee2affd700 1 --2- 172.21.15.153:0/2495996098 >> v2:172.21.15.177:3300/0 conn(0x7fee2c06dee0 0x7fee2c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.215+0000 7fee2b7fe700 1 --2- 172.21.15.153:0/2495996098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c071040 0x7fee2c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2bfff700 1 -- 172.21.15.153:0/2495996098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c071040 msgr2=0x7fee2c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2bfff700 1 --2- 172.21.15.153:0/2495996098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c071040 0x7fee2c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2bfff700 1 -- 172.21.15.153:0/2495996098 >> v2:172.21.15.177:3300/0 conn(0x7fee2c06dee0 msgr2=0x7fee2c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2bfff700 1 --2- 172.21.15.153:0/2495996098 >> v2:172.21.15.177:3300/0 conn(0x7fee2c06dee0 0x7fee2c06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2bfff700 1 -- 172.21.15.153:0/2495996098 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c15d620 con 0x7fee2c06cdc0 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2bfff700 1 --2- 172.21.15.153:0/2495996098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06cdc0 0x7fee2c06d240 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee2c09e140 tx=0x7fee2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f5ef27ca79501c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2a7fc700 1 -- 172.21.15.153:0/2495996098 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400e070 con 0x7fee2c06cdc0 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2a7fc700 1 -- 172.21.15.153:0/2495996098 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2400b910 con 0x7fee2c06cdc0 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee2a7fc700 1 -- 172.21.15.153:0/2495996098 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400bcc0 con 0x7fee2c06cdc0 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee321b6700 1 -- 172.21.15.153:0/2495996098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06cdc0 msgr2=0x7fee2c06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee321b6700 1 --2- 172.21.15.153:0/2495996098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06cdc0 0x7fee2c06d240 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee2c09e140 tx=0x7fee2400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee321b6700 1 -- 172.21.15.153:0/2495996098 shutdown_connections 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.216+0000 7fee321b6700 1 --2- 172.21.15.153:0/2495996098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c071040 0x7fee2c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.217+0000 7fee321b6700 1 --2- 172.21.15.153:0/2495996098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06cdc0 0x7fee2c06d240 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.217+0000 7fee321b6700 1 --2- 172.21.15.153:0/2495996098 >> v2:172.21.15.177:3300/0 conn(0x7fee2c06dee0 0x7fee2c06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.217+0000 7fee321b6700 1 -- 172.21.15.153:0/2495996098 >> 172.21.15.153:0/2495996098 conn(0x7fee2c0d0000 msgr2=0x7fee2c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.217+0000 7fee321b6700 1 -- 172.21.15.153:0/2495996098 shutdown_connections 2023-11-28T13:01:22.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.217+0000 7fee321b6700 1 -- 172.21.15.153:0/2495996098 wait complete. 2023-11-28T13:01:22.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.218+0000 7fee321b6700 1 Processor -- start 2023-11-28T13:01:22.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.218+0000 7fee321b6700 1 -- start start 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee321b6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c06cdc0 0x7fee2c144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee321b6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06dee0 0x7fee2c0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee321b6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee2c071040 0x7fee2c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee321b6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1454e0 con 0x7fee2c06dee0 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee321b6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0c64b0 con 0x7fee2c071040 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee321b6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0c5ae0 con 0x7fee2c06cdc0 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06dee0 0x7fee2c0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06dee0 0x7fee2c0c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40340/0 (socket says 172.21.15.153:40340) 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 -- 172.21.15.153:0/2680835306 learned_addr learned my addr 172.21.15.153:0/2680835306 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2b7fe700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c06cdc0 0x7fee2c144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2bfff700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee2c071040 0x7fee2c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 -- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c06cdc0 msgr2=0x7fee2c144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c06cdc0 0x7fee2c144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 -- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee2c071040 msgr2=0x7fee2c15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee2c071040 0x7fee2c15bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 -- 172.21.15.153:0/2680835306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24009b30 con 0x7fee2c06dee0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee2affd700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06dee0 0x7fee2c0c5f70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e8d0 tx=0x7fee1c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee13fff700 1 -- 172.21.15.153:0/2680835306 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c013680 con 0x7fee2c06dee0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee13fff700 1 -- 172.21.15.153:0/2680835306 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c0137e0 con 0x7fee2c06dee0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c06e890 con 0x7fee2c06dee0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.219+0000 7fee13fff700 1 -- 172.21.15.153:0/2680835306 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c013b90 con 0x7fee2c06dee0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.220+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c15d900 con 0x7fee2c06dee0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.221+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c15ccd0 con 0x7fee2c06dee0 2023-11-28T13:01:22.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.221+0000 7fee13fff700 1 -- 172.21.15.153:0/2680835306 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fee1c019030 con 0x7fee2c06dee0 2023-11-28T13:01:22.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.221+0000 7fee13fff700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fee180682f0 0x7fee1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.221+0000 7fee13fff700 1 -- 172.21.15.153:0/2680835306 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee1c085940 con 0x7fee2c06dee0 2023-11-28T13:01:22.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.221+0000 7fee2b7fe700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fee180682f0 0x7fee1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.222+0000 7fee2b7fe700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fee180682f0 0x7fee1806a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fee20009b40 tx=0x7fee20009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.226+0000 7fee13fff700 1 -- 172.21.15.153:0/2680835306 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fee1c0521a0 con 0x7fee2c06dee0 2023-11-28T13:01:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.486+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fee2c0c5c90 con 0x7fee2c06dee0 2023-11-28T13:01:22.491 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.489+0000 7fee13fff700 1 -- 172.21.15.153:0/2680835306 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7fee2c0c5c90 con 0x7fee2c06dee0 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fee180682f0 msgr2=0x7fee1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fee180682f0 0x7fee1806a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fee20009b40 tx=0x7fee20009210 comp rx=0 tx=0).stop 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06dee0 msgr2=0x7fee2c0c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06dee0 0x7fee2c0c5f70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e8d0 tx=0x7fee1c00ebe0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 shutdown_connections 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee2c071040 0x7fee2c15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee2c06cdc0 0x7fee2c144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fee180682f0 0x7fee1806a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 --2- 172.21.15.153:0/2680835306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c06dee0 0x7fee2c0c5f70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 >> 172.21.15.153:0/2680835306 conn(0x7fee2c0d0000 msgr2=0x7fee2c15e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 shutdown_connections 2023-11-28T13:01:22.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.492+0000 7fee321b6700 1 -- 172.21.15.153:0/2680835306 wait complete. 2023-11-28T13:01:22.495 INFO:teuthology.orchestra.run.smithi153.stderr:updated caps for client.0 2023-11-28T13:01:22.506 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:22.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.625+0000 7fc105de6700 1 Processor -- start 2023-11-28T13:01:22.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.626+0000 7fc105de6700 1 -- start start 2023-11-28T13:01:22.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.626+0000 7fc105de6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1000ce040 0x7fc10015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.626+0000 7fc105de6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc1000d0a00 0x7fc10006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.626+0000 7fc105de6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10006b420 con 0x7fc1000b2930 2023-11-28T13:01:22.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.626+0000 7fc105de6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10006b700 con 0x7fc1000ce040 2023-11-28T13:01:22.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.626+0000 7fc105de6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10006b9e0 con 0x7fc1000d0a00 2023-11-28T13:01:22.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.626+0000 7fc0ff7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1000ce040 0x7fc10015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0ff7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1000ce040 0x7fc10015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40346/0 (socket says 172.21.15.153:40346) 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0ff7fe700 1 -- 172.21.15.153:0/1085713249 learned_addr learned my addr 172.21.15.153:0/1085713249 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0feffd700 1 --2- 172.21.15.153:0/1085713249 >> v2:172.21.15.177:3300/0 conn(0x7fc1000d0a00 0x7fc10006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0ff7fe700 1 -- 172.21.15.153:0/1085713249 >> v2:172.21.15.177:3300/0 conn(0x7fc1000d0a00 msgr2=0x7fc10006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0ff7fe700 1 --2- 172.21.15.153:0/1085713249 >> v2:172.21.15.177:3300/0 conn(0x7fc1000d0a00 0x7fc10006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0ff7fe700 1 -- 172.21.15.153:0/1085713249 >> v1:172.21.15.177:6789/0 conn(0x7fc1000b2930 legacy=0x7fc10006c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0ff7fe700 1 -- 172.21.15.153:0/1085713249 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10006bcc0 con 0x7fc1000ce040 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0feffd700 1 --2- 172.21.15.153:0/1085713249 >> v2:172.21.15.177:3300/0 conn(0x7fc1000d0a00 0x7fc10006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0ff7fe700 1 --2- 172.21.15.153:0/1085713249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc10015df10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc10006ed50 tx=0x7fc0f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace47a8764c18034 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0fe7fc700 1 -- 172.21.15.153:0/1085713249 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0005e30 con 0x7fc1000ce040 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0fe7fc700 1 -- 172.21.15.153:0/1085713249 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f0004030 con 0x7fc1000ce040 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.627+0000 7fc0fe7fc700 1 -- 172.21.15.153:0/1085713249 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f00043e0 con 0x7fc1000ce040 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 -- 172.21.15.153:0/1085713249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 msgr2=0x7fc10015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 --2- 172.21.15.153:0/1085713249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc10015df10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc10006ed50 tx=0x7fc0f0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 -- 172.21.15.153:0/1085713249 shutdown_connections 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 --2- 172.21.15.153:0/1085713249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc10015df10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 --2- 172.21.15.153:0/1085713249 >> v2:172.21.15.177:3300/0 conn(0x7fc1000d0a00 0x7fc10006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 -- 172.21.15.153:0/1085713249 >> 172.21.15.153:0/1085713249 conn(0x7fc10006aed0 msgr2=0x7fc10006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 -- 172.21.15.153:0/1085713249 shutdown_connections 2023-11-28T13:01:22.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.628+0000 7fc105de6700 1 -- 172.21.15.153:0/1085713249 wait complete. 2023-11-28T13:01:22.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.629+0000 7fc105de6700 1 Processor -- start 2023-11-28T13:01:22.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.629+0000 7fc105de6700 1 -- start start 2023-11-28T13:01:22.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc105de6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1000b2930 0x7fc1000c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc105de6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc1000ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc105de6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1000d0a00 0x7fc1000c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc105de6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000cae10 con 0x7fc1000ce040 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc105de6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000c5970 con 0x7fc1000b2930 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc105de6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000c5c50 con 0x7fc1000d0a00 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc1000ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc1000ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40354/0 (socket says 172.21.15.153:40354) 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 -- 172.21.15.153:0/70489183 learned_addr learned my addr 172.21.15.153:0/70489183 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0fffff700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1000d0a00 0x7fc1000c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0ff7fe700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1000b2930 0x7fc1000c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 -- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1000d0a00 msgr2=0x7fc1000c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1000d0a00 0x7fc1000c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 -- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1000b2930 msgr2=0x7fc1000c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1000b2930 0x7fc1000c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.630+0000 7fc0feffd700 1 -- 172.21.15.153:0/70489183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0009500 con 0x7fc1000ce040 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.631+0000 7fc0feffd700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc1000ca8d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800e810 tx=0x7fc0e800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.631+0000 7fc0f7fff700 1 -- 172.21.15.153:0/70489183 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8013610 con 0x7fc1000ce040 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.631+0000 7fc0f7fff700 1 -- 172.21.15.153:0/70489183 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8013770 con 0x7fc1000ce040 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.631+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10006fd90 con 0x7fc1000ce040 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.631+0000 7fc0f7fff700 1 -- 172.21.15.153:0/70489183 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8013b20 con 0x7fc1000ce040 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.631+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10006bfa0 con 0x7fc1000ce040 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.632+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10006ff20 con 0x7fc1000ce040 2023-11-28T13:01:22.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.632+0000 7fc0f7fff700 1 -- 172.21.15.153:0/70489183 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc0e8023070 con 0x7fc1000ce040 2023-11-28T13:01:22.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.632+0000 7fc0f7fff700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc0e00682f0 0x7fc0e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.633+0000 7fc0f7fff700 1 -- 172.21.15.153:0/70489183 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc0e8085780 con 0x7fc1000ce040 2023-11-28T13:01:22.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.633+0000 7fc0ff7fe700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc0e00682f0 0x7fc0e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.633+0000 7fc0ff7fe700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc0e00682f0 0x7fc0e006a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc10006ed50 tx=0x7fc0f0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.637+0000 7fc0f7fff700 1 -- 172.21.15.153:0/70489183 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc0e8051fe0 con 0x7fc1000ce040 2023-11-28T13:01:22.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.860+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc100068690 con 0x7fc1000ce040 2023-11-28T13:01:22.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.861+0000 7fc0f7fff700 1 -- 172.21.15.153:0/70489183 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fc0e8055600 con 0x7fc1000ce040 2023-11-28T13:01:22.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:22.863 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc0e00682f0 msgr2=0x7fc0e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc0e00682f0 0x7fc0e006a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc10006ed50 tx=0x7fc0f0008070 comp rx=0 tx=0).stop 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 msgr2=0x7fc1000ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc1000ca8d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800e810 tx=0x7fc0e800eb20 comp rx=0 tx=0).stop 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 shutdown_connections 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1000d0a00 0x7fc1000c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1000b2930 0x7fc1000c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc0e00682f0 0x7fc0e006a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 --2- 172.21.15.153:0/70489183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1000ce040 0x7fc1000ca8d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 >> 172.21.15.153:0/70489183 conn(0x7fc10006aed0 msgr2=0x7fc10015cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 shutdown_connections 2023-11-28T13:01:22.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:22.863+0000 7fc105de6700 1 -- 172.21.15.153:0/70489183 wait complete. 2023-11-28T13:01:22.866 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:23.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:01:13.076520+0000', 'modified': '2023-11-28T13:01:16.090678+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5073}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5073': {'gid': 5073, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6837/122658331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6835', 'nonce': 122658331}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 122658331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:01:23.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:01:23.279 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T13:01:23.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.402+0000 7f66157a2700 1 Processor -- start 2023-11-28T13:01:23.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.402+0000 7f66157a2700 1 -- start start 2023-11-28T13:01:23.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.402+0000 7f66157a2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160b10 0x7f66100cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.402+0000 7f66157a2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f661006b060 0x7f661006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.402+0000 7f66157a2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66100c9e10 con 0x7f6610160740 2023-11-28T13:01:23.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.402+0000 7f66157a2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66100ca720 con 0x7f661006b060 2023-11-28T13:01:23.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.402+0000 7f66157a2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661006efc0 con 0x7f6610160b10 2023-11-28T13:01:23.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f661006b060 0x7f661006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f661006b060 0x7f661006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40362/0 (socket says 172.21.15.153:40362) 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 -- 172.21.15.153:0/2118011299 learned_addr learned my addr 172.21.15.153:0/2118011299 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660e7fc700 1 --2- 172.21.15.153:0/2118011299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160b10 0x7f66100cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 -- 172.21.15.153:0/2118011299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160b10 msgr2=0x7f66100cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 --2- 172.21.15.153:0/2118011299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160b10 0x7f66100cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 -- 172.21.15.153:0/2118011299 >> v1:172.21.15.177:6789/0 conn(0x7f6610160740 legacy=0x7f66100cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 -- 172.21.15.153:0/2118011299 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100ca0d0 con 0x7f661006b060 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660e7fc700 1 --2- 172.21.15.153:0/2118011299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160b10 0x7f66100cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660f7fe700 1 --2- 172.21.15.153:0/2118011299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661006b4e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6608009e50 tx=0x7f660800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5569df4fe348a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660dffb700 1 -- 172.21.15.153:0/2118011299 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800e070 con 0x7f661006b060 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660dffb700 1 -- 172.21.15.153:0/2118011299 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660800b9c0 con 0x7f661006b060 2023-11-28T13:01:23.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.403+0000 7f660dffb700 1 -- 172.21.15.153:0/2118011299 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800bd70 con 0x7f661006b060 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 -- 172.21.15.153:0/2118011299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 msgr2=0x7f661006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 --2- 172.21.15.153:0/2118011299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661006b4e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6608009e50 tx=0x7f660800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 -- 172.21.15.153:0/2118011299 shutdown_connections 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 --2- 172.21.15.153:0/2118011299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661006b4e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 --2- 172.21.15.153:0/2118011299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160b10 0x7f66100cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 -- 172.21.15.153:0/2118011299 >> 172.21.15.153:0/2118011299 conn(0x7f66100b12b0 msgr2=0x7f66100b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 -- 172.21.15.153:0/2118011299 shutdown_connections 2023-11-28T13:01:23.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.404+0000 7f66157a2700 1 -- 172.21.15.153:0/2118011299 wait complete. 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.405+0000 7f66157a2700 1 Processor -- start 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.405+0000 7f66157a2700 1 -- start start 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f66157a2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f66157a2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160740 0x7f661006d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f66157a2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6610160b10 0x7f661006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f66157a2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6610158d80 con 0x7f661006b060 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f66157a2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6610159060 con 0x7f6610160b10 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f66157a2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6610159340 con 0x7f6610160740 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661015b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40364/0 (socket says 172.21.15.153:40364) 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 -- 172.21.15.153:0/3965584686 learned_addr learned my addr 172.21.15.153:0/3965584686 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160740 0x7f661006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660f7fe700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6610160b10 0x7f661006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 -- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160740 msgr2=0x7f661006d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160740 0x7f661006d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 -- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6610160b10 msgr2=0x7f661006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6610160b10 0x7f661006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 -- 172.21.15.153:0/3965584686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608009b30 con 0x7f661006b060 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f660effd700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661015b8c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f660400c4e0 tx=0x7f660400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.406+0000 7f65f77fe700 1 -- 172.21.15.153:0/3965584686 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040052d0 con 0x7f661006b060 2023-11-28T13:01:23.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.407+0000 7f65f77fe700 1 -- 172.21.15.153:0/3965584686 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6604005430 con 0x7f661006b060 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.407+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66100cd8d0 con 0x7f661006b060 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.407+0000 7f65f77fe700 1 -- 172.21.15.153:0/3965584686 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040038c0 con 0x7f661006b060 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.407+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66100ca3b0 con 0x7f661006b060 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.408+0000 7f65f77fe700 1 -- 172.21.15.153:0/3965584686 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6604019030 con 0x7f661006b060 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.409+0000 7f65f77fe700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65fc068320 0x7f65fc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.409+0000 7f65f77fe700 1 -- 172.21.15.153:0/3965584686 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66040850d0 con 0x7f661006b060 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.409+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65fc068320 0x7f65fc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.409+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65fc068320 0x7f65fc06a7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6600006fd0 tx=0x7f6600008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.410+0000 7f662192bb80 1 Processor -- start 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.411+0000 7f662192bb80 1 -- start start 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.411+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a16810 0x55d0769eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.411+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a14130 0x55d076a16be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.411+0000 7f662192bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a17120 0x55d076a32770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.411+0000 7f662192bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0768d6bc0 con 0x55d076a17120 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.411+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d076945350 con 0x55d076a14130 2023-11-28T13:01:23.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.411+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d07693e880 con 0x55d076a16810 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a17120 0x55d076a32770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a17120 0x55d076a32770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40376/0 (socket says 172.21.15.153:40376) 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 -- 172.21.15.153:0/271918462 learned_addr learned my addr 172.21.15.153:0/271918462 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660e7fc700 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a14130 0x55d076a16be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660effd700 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a16810 0x55d0769eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 -- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a16810 msgr2=0x55d0769eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a16810 0x55d0769eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 -- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a14130 msgr2=0x55d076a16be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a14130 0x55d076a16be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 -- 172.21.15.153:0/271918462 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0767c05e0 con 0x55d076a17120 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660effd700 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a16810 0x55d0769eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f660f7fe700 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a17120 0x55d076a32770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f660800b730 tx=0x7f6608010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c91e4c92a40002d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f65f4ff9700 1 -- 172.21.15.153:0/271918462 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608003a20 con 0x55d076a17120 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f65f4ff9700 1 -- 172.21.15.153:0/271918462 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608003b80 con 0x55d076a17120 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.412+0000 7f65f4ff9700 1 -- 172.21.15.153:0/271918462 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608004040 con 0x55d076a17120 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 -- 172.21.15.153:0/271918462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a17120 msgr2=0x55d076a32770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a17120 0x55d076a32770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f660800b730 tx=0x7f6608010000 comp rx=0 tx=0).stop 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 -- 172.21.15.153:0/271918462 shutdown_connections 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a17120 0x55d076a32770 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a16810 0x55d0769eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 --2- 172.21.15.153:0/271918462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a14130 0x55d076a16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 -- 172.21.15.153:0/271918462 >> 172.21.15.153:0/271918462 conn(0x55d0769e9d60 msgr2=0x55d0769ea6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.422 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 -- 172.21.15.153:0/271918462 shutdown_connections 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.413+0000 7f662192bb80 1 -- 172.21.15.153:0/271918462 wait complete. 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 -- start start 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 Processor -- start 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a0d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a0dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0767c28d0 con 0x55d076a16810 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d076a29870 con 0x55d076a17120 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d076a0e020 con 0x55d076a14130 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f662192bb80 10 client.0 _ll_register_callbacks cb 0x55d0767e4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.414+0000 7f660e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a0d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a0d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40380/0 (socket says 172.21.15.153:40380) 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 -- 172.21.15.153:0/3391042586 learned_addr learned my addr 172.21.15.153:0/3391042586 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660effd700 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660f7fe700 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a0dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 -- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 msgr2=0x55d076a397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 -- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 msgr2=0x55d076a0dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a0dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 -- 172.21.15.153:0/3391042586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66100ca0d0 con 0x55d076a16810 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a0d450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6610150cd0 tx=0x7f66000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50567b267a05a5a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f65edffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f65edffb700 1 -- 172.21.15.153:0/3391042586 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000fe60 con 0x55d076a16810 2023-11-28T13:01:23.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f65edffb700 1 -- 172.21.15.153:0/3391042586 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6600021420 con 0x55d076a16810 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d07690c140 con 0x55d076a16810 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f65edffb700 1 -- 172.21.15.153:0/3391042586 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66000215e0 con 0x55d076a16810 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f662192bb80 10 client.5274 fetch_fsmap learned FSMap version 16 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d07676cd90 con 0x55d076a16810 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.415+0000 7f65edffb700 1 -- 172.21.15.153:0/3391042586 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d07690c140 con 0x55d076a16810 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f65edffb700 1 -- 172.21.15.153:0/3391042586 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6600021740 con 0x55d076a16810 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f662192bb80 10 client.5274 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f662192bb80 10 client.5274 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f662192bb80 20 client.5274 populate_metadata read hostname 'smithi153' 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f662192bb80 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a372f0 0x55d076a15de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f662192bb80 4 client.5274 mds_command: new command op to 5073 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 --> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0767c2f10 con 0x55d076a372f0 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f660effd700 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a372f0 0x55d076a15de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f65edffb700 10 client.5274 ms_handle_connect on v2:172.21.15.153:6835/122658331 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.416+0000 7f660effd700 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a372f0 0x55d076a15de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a042e1758cb6fe0b server_cookie=6d6918fde82cebe5 in_seq=0 out_seq=0 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f65edffb700 1 -- 172.21.15.153:0/3391042586 <== mds.0 v2:172.21.15.153:6835/122658331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55d0767c2f10 con 0x55d076a372f0 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f65edffb700 10 client.5274 handle_command_reply: tid=0 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 client.5274 shutdown 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 msgr2=0x55d076a0d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a0d450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6610150cd0 tx=0x7f66000047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 shutdown_connections 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a372f0 0x55d076a15de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a0dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f65edffb700 0 client.5274 ms_handle_reset on v2:172.21.15.153:6835/122658331 2023-11-28T13:01:23.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 --2- 172.21.15.153:0/3391042586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a0d450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 >> 172.21.15.153:0/3391042586 conn(0x55d0769e9d60 msgr2=0x55d0769ec990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 shutdown_connections 2023-11-28T13:01:23.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 1 -- 172.21.15.153:0/3391042586 wait complete. 2023-11-28T13:01:23.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.417+0000 7f662192bb80 20 client.5274 trim_cache size 0 max 16384 2023-11-28T13:01:23.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.434+0000 7f662192bb80 1 Processor -- start 2023-11-28T13:01:23.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.434+0000 7f662192bb80 1 -- start start 2023-11-28T13:01:23.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f662192bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a14130 0x55d076a36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a16810 0x55d076a0f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a17120 0x55d076a34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f662192bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d076a385a0 con 0x55d076a14130 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d076a35800 con 0x55d076a16810 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d076a0ee80 con 0x55d076a17120 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a14130 0x55d076a36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a14130 0x55d076a36d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40386/0 (socket says 172.21.15.153:40386) 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 -- 172.21.15.153:0/3756351310 learned_addr learned my addr 172.21.15.153:0/3756351310 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660f7fe700 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a17120 0x55d076a34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a16810 0x55d076a0f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 -- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a17120 msgr2=0x55d076a34c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a17120 0x55d076a34c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 -- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a16810 msgr2=0x55d076a0f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a16810 0x55d076a0f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 -- 172.21.15.153:0/3756351310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d076905140 con 0x55d076a14130 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f660effd700 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a14130 0x55d076a36d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6604055490 tx=0x7f6604059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc42fb34b1fb70e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.435+0000 7f65f57fa700 1 -- 172.21.15.153:0/3756351310 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604083470 con 0x55d076a14130 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f65f57fa700 1 -- 172.21.15.153:0/3756351310 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66040539c0 con 0x55d076a14130 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f65f57fa700 1 -- 172.21.15.153:0/3756351310 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604054c50 con 0x55d076a14130 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 -- 172.21.15.153:0/3756351310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a14130 msgr2=0x55d076a36d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a14130 0x55d076a36d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6604055490 tx=0x7f6604059040 comp rx=0 tx=0).stop 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 -- 172.21.15.153:0/3756351310 shutdown_connections 2023-11-28T13:01:23.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a14130 0x55d076a36d90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a17120 0x55d076a34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 --2- 172.21.15.153:0/3756351310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a16810 0x55d076a0f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 -- 172.21.15.153:0/3756351310 >> 172.21.15.153:0/3756351310 conn(0x55d0769e9d60 msgr2=0x55d0769e8100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 -- 172.21.15.153:0/3756351310 shutdown_connections 2023-11-28T13:01:23.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.436+0000 7f662192bb80 1 -- 172.21.15.153:0/3756351310 wait complete. 2023-11-28T13:01:23.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 -- start start 2023-11-28T13:01:23.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 Processor -- start 2023-11-28T13:01:23.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a15ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d076a0dec0 con 0x55d076a16810 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d076a36930 con 0x55d076a17120 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.437+0000 7f662192bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0769e77a0 con 0x55d076a14130 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f662192bb80 10 client.0 _ll_register_callbacks cb 0x55d0767e4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a35c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40394/0 (socket says 172.21.15.153:40394) 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 -- 172.21.15.153:0/3347125134 learned_addr learned my addr 172.21.15.153:0/3347125134 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660effd700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660f7fe700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a15ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 -- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 msgr2=0x55d076a1a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a1a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 -- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 msgr2=0x55d076a15ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a15ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 -- 172.21.15.153:0/3347125134 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66100ca3b0 con 0x55d076a16810 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660f7fe700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a15ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f660e7fc700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a35c10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6610150cd0 tx=0x7f6600016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce6fa66cce9e5bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f65edffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f65edffb700 1 -- 172.21.15.153:0/3347125134 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600038070 con 0x55d076a16810 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f65edffb700 1 -- 172.21.15.153:0/3347125134 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f660000fe60 con 0x55d076a16810 2023-11-28T13:01:23.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0768fc160 con 0x55d076a16810 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f65edffb700 1 -- 172.21.15.153:0/3347125134 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6600014a20 con 0x55d076a16810 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f662192bb80 10 client.5280 fetch_fsmap learned FSMap version 16 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d076793200 con 0x55d076a16810 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.438+0000 7f65edffb700 1 -- 172.21.15.153:0/3347125134 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0768fc160 con 0x55d076a16810 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f65edffb700 1 -- 172.21.15.153:0/3347125134 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6600021420 con 0x55d076a16810 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 10 client.5280 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 10 client.5280 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 20 client.5280 populate_metadata read hostname 'smithi153' 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a16010 0x55d076a28300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 4 client.5280 mds_command: new command op to 5073 tid=0[{"prefix": "status"}] 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 --> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d0768e6170 con 0x55d076a16010 2023-11-28T13:01:23.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f660effd700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a16010 0x55d076a28300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f65edffb700 10 client.5280 ms_handle_connect on v2:172.21.15.153:6835/122658331 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f660effd700 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a16010 0x55d076a28300 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cba6086ed46c435 server_cookie=3ce9da2c463e9825 in_seq=0 out_seq=0 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f65edffb700 1 -- 172.21.15.153:0/3347125134 <== mds.0 v2:172.21.15.153:6835/122658331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55d0768e6170 con 0x55d076a16010 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f65edffb700 10 client.5280 handle_command_reply: tid=0 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 client.5280 shutdown 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 msgr2=0x55d076a35c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a35c10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6610150cd0 tx=0x7f6600016000 comp rx=0 tx=0).stop 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 shutdown_connections 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:6835/122658331,v1:172.21.15.153:6837/122658331] conn(0x55d076a16010 0x55d076a28300 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d076a17120 0x55d076a15ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d076a14130 0x55d076a1a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 --2- 172.21.15.153:0/3347125134 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d076a16810 0x55d076a35c10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 >> 172.21.15.153:0/3347125134 conn(0x55d0769e9d60 msgr2=0x55d0769eb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f65edffb700 0 client.5280 ms_handle_reset on v2:172.21.15.153:6835/122658331 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 shutdown_connections 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.439+0000 7f662192bb80 1 -- 172.21.15.153:0/3347125134 wait complete. 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f662192bb80 20 client.5280 trim_cache size 0 max 16384 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stdout: "cluster_fsid": "47bfaa00-a151-4537-aa4c-32b0866eb248", 2023-11-28T13:01:23.449 INFO:teuthology.orchestra.run.smithi153.stdout: "whoami": 0, 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 5073, 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "want_state": "up:active", 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "state": "up:active", 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs", 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "rank_uptime": 10.362995521, 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "mdsmap_epoch": 16, 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch": 39, 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch_barrier": 39, 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout: "uptime": 17.676198734 2023-11-28T13:01:23.450 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:01:23.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65fc068320 msgr2=0x7f65fc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65fc068320 0x7f65fc06a7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6600006fd0 tx=0x7f6600008040 comp rx=0 tx=0).stop 2023-11-28T13:01:23.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 msgr2=0x7f661015b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661015b8c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f660400c4e0 tx=0x7f660400c8a0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 shutdown_connections 2023-11-28T13:01:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f65fc068320 0x7f65fc06a7d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f661006b060 0x7f661015b8c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6610160b10 0x7f661006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 --2- 172.21.15.153:0/3965584686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6610160740 0x7f661006d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 >> 172.21.15.153:0/3965584686 conn(0x7f66100b12b0 msgr2=0x7f66100b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 shutdown_connections 2023-11-28T13:01:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.440+0000 7f66157a2700 1 -- 172.21.15.153:0/3965584686 wait complete. 2023-11-28T13:01:23.454 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:01:23.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.572+0000 7f1c93f04700 1 Processor -- start 2023-11-28T13:01:23.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.572+0000 7f1c93f04700 1 -- start start 2023-11-28T13:01:23.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c93f04700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1c8c071040 0x7f1c8c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c93f04700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c160540 0x7f1c8c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c93f04700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8c071950 con 0x7f1c8c06dee0 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c93f04700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c8c15d060 con 0x7f1c8c071040 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c93f04700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c15d340 con 0x7f1c8c160540 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c91ca0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1c8c071040 0x7f1c8c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c91ca0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1c8c071040 0x7f1c8c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40400/0 (socket says 172.21.15.153:40400) 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1860350689 learned_addr learned my addr 172.21.15.153:0/1860350689 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.573+0000 7f1c9149f700 1 --1- 172.21.15.153:0/1860350689 >> v1:172.21.15.153:6789/0 conn(0x7f1c8c06dee0 0x7f1c8c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58564/0 (socket says 172.21.15.153:58564) 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c924a1700 1 --2- 172.21.15.153:0/1860350689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c160540 0x7f1c8c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1860350689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c160540 msgr2=0x7f1c8c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c91ca0700 1 --2- 172.21.15.153:0/1860350689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c160540 0x7f1c8c1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1860350689 >> v1:172.21.15.153:6789/0 conn(0x7f1c8c06dee0 legacy=0x7f1c8c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1860350689 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c15d620 con 0x7f1c8c071040 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c90c9e700 1 -- 172.21.15.153:0/1860350689 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868482755 0 0) 0x7f1c8c071950 con 0x7f1c8c06dee0 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c924a1700 1 --2- 172.21.15.153:0/1860350689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c160540 0x7f1c8c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c91ca0700 1 --2- 172.21.15.153:0/1860350689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c071040 0x7f1c8c071410 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009e50 tx=0x7f1c8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dd09bc95e2af5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c90c9e700 1 -- 172.21.15.153:0/1860350689 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8400e070 con 0x7f1c8c071040 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c90c9e700 1 -- 172.21.15.153:0/1860350689 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8400b9c0 con 0x7f1c8c071040 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c90c9e700 1 -- 172.21.15.153:0/1860350689 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8400bd70 con 0x7f1c8c071040 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c93f04700 1 -- 172.21.15.153:0/1860350689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c071040 msgr2=0x7f1c8c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.574+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1860350689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c071040 0x7f1c8c071410 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009e50 tx=0x7f1c8400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.575+0000 7f1c93f04700 1 -- 172.21.15.153:0/1860350689 shutdown_connections 2023-11-28T13:01:23.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.575+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1860350689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c160540 0x7f1c8c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.575+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1860350689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c071040 0x7f1c8c071410 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.575+0000 7f1c93f04700 1 -- 172.21.15.153:0/1860350689 >> 172.21.15.153:0/1860350689 conn(0x7f1c8c0d0000 msgr2=0x7f1c8c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.575+0000 7f1c93f04700 1 -- 172.21.15.153:0/1860350689 shutdown_connections 2023-11-28T13:01:23.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.575+0000 7f1c93f04700 1 -- 172.21.15.153:0/1860350689 wait complete. 2023-11-28T13:01:23.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.576+0000 7f1c93f04700 1 Processor -- start 2023-11-28T13:01:23.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.576+0000 7f1c93f04700 1 -- start start 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c06dee0 0x7f1c8c1f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c071040 0x7f1c8c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c8c160540 0x7f1c8c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0b3e20 con 0x7f1c8c06dee0 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c1f31d0 con 0x7f1c8c160540 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c0b31a0 con 0x7f1c8c071040 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c06dee0 0x7f1c8c1f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c06dee0 0x7f1c8c1f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40404/0 (socket says 172.21.15.153:40404) 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1005795023 learned_addr learned my addr 172.21.15.153:0/1005795023 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c924a1700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c8c160540 0x7f1c8c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c071040 msgr2=0x7f1c8c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c9149f700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c071040 0x7f1c8c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c071040 0x7f1c8c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c8c160540 msgr2=0x7f1c8c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c8c160540 0x7f1c8c0c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 -- 172.21.15.153:0/1005795023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84009b30 con 0x7f1c8c06dee0 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c924a1700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c8c160540 0x7f1c8c0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c9149f700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c071040 0x7f1c8c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c91ca0700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c06dee0 0x7f1c8c1f2c90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400b370 tx=0x7f1c84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c827fc700 1 -- 172.21.15.153:0/1005795023 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c840099b0 con 0x7f1c8c06dee0 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c827fc700 1 -- 172.21.15.153:0/1005795023 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c840092e0 con 0x7f1c8c06dee0 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c827fc700 1 -- 172.21.15.153:0/1005795023 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8401b630 con 0x7f1c8c06dee0 2023-11-28T13:01:23.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c15ccd0 con 0x7f1c8c06dee0 2023-11-28T13:01:23.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.577+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c15d900 con 0x7f1c8c06dee0 2023-11-28T13:01:23.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.578+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8c06a270 con 0x7f1c8c06dee0 2023-11-28T13:01:23.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.579+0000 7f1c827fc700 1 -- 172.21.15.153:0/1005795023 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c8402b070 con 0x7f1c8c06dee0 2023-11-28T13:01:23.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.579+0000 7f1c827fc700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c700682f0 0x7f1c7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.579+0000 7f1c9149f700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c700682f0 0x7f1c7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.579+0000 7f1c827fc700 1 -- 172.21.15.153:0/1005795023 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c84087500 con 0x7f1c8c06dee0 2023-11-28T13:01:23.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.580+0000 7f1c9149f700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c700682f0 0x7f1c7006a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c007fd0 tx=0x7f1c7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.583+0000 7f1c827fc700 1 -- 172.21.15.153:0/1005795023 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c84053e10 con 0x7f1c8c06dee0 2023-11-28T13:01:23.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.809+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1c8c0720a0 con 0x7f1c8c06dee0 2023-11-28T13:01:23.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.810+0000 7f1c827fc700 1 -- 172.21.15.153:0/1005795023 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1c84057430 con 0x7f1c8c06dee0 2023-11-28T13:01:23.812 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:01:23.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c700682f0 msgr2=0x7f1c7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c700682f0 0x7f1c7006a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c007fd0 tx=0x7f1c7c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:23.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c06dee0 msgr2=0x7f1c8c1f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c06dee0 0x7f1c8c1f2c90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400b370 tx=0x7f1c84004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 shutdown_connections 2023-11-28T13:01:23.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c700682f0 0x7f1c7006a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c8c06dee0 0x7f1c8c1f2c90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c8c160540 0x7f1c8c0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 --2- 172.21.15.153:0/1005795023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c8c071040 0x7f1c8c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 >> 172.21.15.153:0/1005795023 conn(0x7f1c8c0d0000 msgr2=0x7f1c8c15e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 shutdown_connections 2023-11-28T13:01:23.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:23.812+0000 7f1c93f04700 1 -- 172.21.15.153:0/1005795023 wait complete. 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:01:23.826 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:01:23.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:23.827 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-11-28T13:01:23.844 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:01:23.844 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:01:23.844 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:01:23.844 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:23.844 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-11-28T13:01:23.844 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42296sec preferred_lft 42296sec 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ad7d:b13e:f07:e265/64 scope link noprefixroute 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:23.845 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 4e:45:7e:fd:cf:4e brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::6cbb:9eff:fe64:da81/64 scope link 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether a2:10:9d:7d:8b:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::a010:9dff:fe7d:8b89/64 scope link 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 4e:45:7e:fd:cf:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:01:23.846 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::4c45:7eff:fefd:cf4e/64 scope link 2023-11-28T13:01:23.847 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:23.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:23.847 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:01:23.909 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-11-28T13:01:23.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:23.911 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-11-28T13:01:23.970 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-11-28T13:01:23.970 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:01:23.970 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:01:23.971 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:01:23.971 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:01:23.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:23.971 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:01:23.971 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:01:23.971 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:01:23.971 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:01:23.972 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:01:23.972 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:01:24.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.076 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:01:24.092 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:01:24.093 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:01:24.093 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:01:24.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.093 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-11-28T13:01:24.159 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:01:24.160 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:24.216 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:01:24.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:01:24.217 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:24.273 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:01:24.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.273 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:01:24.358 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:01:24.359 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:01:24.359 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:01:24.367 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:01:24.367 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:01:24.367 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:01:24.368 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:01:24.368 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:01:24.394 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:01:24.394 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:01:24.394 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=47bfaa00-a151-4537-aa4c-32b0866eb248" will pass to kernel 2023-11-28T13:01:24.400 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:01:24.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.401 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:24.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.434 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:01:24.517 INFO:teuthology.orchestra.run.smithi153.stdout:client5292 2023-11-28T13:01:24.522 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:01:24.523 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:24.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.580 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T13:01:24.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.643 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T13:01:24.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.701 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T13:01:24.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.758 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T13:01:24.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.819 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T13:01:24.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:24.877 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T13:01:24.939 INFO:teuthology.orchestra.run.smithi153.stdout:./ 2023-11-28T13:01:24.939 INFO:teuthology.orchestra.run.smithi153.stdout:./rfile 2023-11-28T13:01:24.939 INFO:teuthology.orchestra.run.smithi153.stdout:./parentdir 2023-11-28T13:01:24.939 INFO:teuthology.orchestra.run.smithi153.stdout:./parentdir/pfile 2023-11-28T13:01:24.939 INFO:teuthology.orchestra.run.smithi153.stdout:./parentdir/childdir 2023-11-28T13:01:24.939 INFO:teuthology.orchestra.run.smithi153.stdout:./parentdir/childdir/cfile 2023-11-28T13:01:24.940 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:24.940 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:01:24.940 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:01:24.940 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:01:24.940 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T13:01:24.940 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:01:25.044 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-11-28T13:01:25.234 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:25.234 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:01:25.234 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:01:25.235 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:01:25.235 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T13:01:25.235 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:01:25.342 INFO:teuthology.orchestra.run.smithi153.stdout:1099511627778 2023-11-28T13:01:25.531 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:25.531 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:01:25.531 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:01:25.531 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:01:25.531 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T13:01:25.531 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:01:25.637 INFO:teuthology.orchestra.run.smithi153.stdout:1099511627776 2023-11-28T13:01:25.827 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:25.827 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:01:25.827 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:01:25.827 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:01:25.827 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T13:01:25.827 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:01:25.932 INFO:teuthology.orchestra.run.smithi153.stdout:1099511628279 2023-11-28T13:01:26.122 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:26.122 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:01:26.122 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:01:26.122 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:01:26.122 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T13:01:26.122 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:01:26.228 INFO:teuthology.orchestra.run.smithi153.stdout:1099511627777 2023-11-28T13:01:26.417 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:26.418 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:01:26.418 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:01:26.418 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:01:26.418 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T13:01:26.418 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:01:26.524 INFO:teuthology.orchestra.run.smithi153.stdout:1099511628280 2023-11-28T13:01:26.714 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:01:26.714 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:26.773 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:01:26.773 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:26.830 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:26.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.989+0000 7f5528f41700 1 Processor -- start 2023-11-28T13:01:26.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.989+0000 7f5528f41700 1 -- start start 2023-11-28T13:01:26.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f5528f41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 0x7f552406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f5528f41700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552413ff00 con 0x7f5524140950 2023-11-28T13:01:26.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f5528f41700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55241404c0 con 0x7f55240cd8c0 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f5528f41700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5524140dc0 con 0x7f552406f850 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f5521d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5524140950 0x7f55240b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58580/0 (socket says 172.21.15.153:58580) 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f5521d9b700 1 -- 172.21.15.153:0/883903276 learned_addr learned my addr 172.21.15.153:0/883903276 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f5522d9d700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 0x7f552406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882113923 0 0) 0x7f552413ff00 con 0x7f5524140950 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c0035b0 con 0x7f5524140950 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957160006 0 0) 0x7f55241404c0 con 0x7f55240cd8c0 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f552413ff00 con 0x7f55240cd8c0 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679087947 0 0) 0x7f550c0035b0 con 0x7f5524140950 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 msgr2=0x7f552406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:26.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 0x7f552406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 >> v1:172.21.15.177:6789/0 conn(0x7f55240cd8c0 legacy=0x7f55240cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552415aa80 con 0x7f5524140950 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f5518001670 con 0x7f5524140950 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 >> v1:172.21.15.153:6789/0 conn(0x7f5524140950 legacy=0x7f55240b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.990+0000 7f552159a700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f550c003bc0 0x7f550c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f550c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 0x7f550c005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524140dc0 con 0x7f550c004850 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c00bbd0 con 0x7f550c003bc0 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c00bd90 con 0x7f552406f850 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5521d9b700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f550c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552259c700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f550c003bc0 0x7f550c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5522d9d700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 0x7f550c005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5521d9b700 1 -- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 msgr2=0x7f550c005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5521d9b700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 0x7f550c005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5521d9b700 1 -- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f550c003bc0 msgr2=0x7f550c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5521d9b700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f550c003bc0 0x7f550c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5521d9b700 1 -- 172.21.15.153:0/883903276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0035b0 con 0x7f550c004850 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5522d9d700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f552406f850 0x7f550c005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f5521d9b700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f550c004cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f551800c0e0 tx=0x7f550c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d347733a22a165f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800cc80 con 0x7f550c004850 2023-11-28T13:01:26.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551800cde0 con 0x7f550c004850 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.991+0000 7f552159a700 1 -- 172.21.15.153:0/883903276 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518005b60 con 0x7f550c004850 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.992+0000 7f5528f41700 1 -- 172.21.15.153:0/883903276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 msgr2=0x7f550c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.992+0000 7f5528f41700 1 --2- 172.21.15.153:0/883903276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f550c004cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f551800c0e0 tx=0x7f550c005880 comp rx=0 tx=0).stop 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.992+0000 7f5522d9d700 1 -- 172.21.15.153:0/883903276 reap_dead start 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.992+0000 7f5528f41700 1 -- 172.21.15.153:0/883903276 shutdown_connections 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.992+0000 7f5528f41700 1 -- 172.21.15.153:0/883903276 >> 172.21.15.153:0/883903276 conn(0x7f552406aed0 msgr2=0x7f552406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.992+0000 7f5528f41700 1 -- 172.21.15.153:0/883903276 shutdown_connections 2023-11-28T13:01:26.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.992+0000 7f5528f41700 1 -- 172.21.15.153:0/883903276 wait complete. 2023-11-28T13:01:27.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 Processor -- start 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 -- start start 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f5524158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55240c65d0 0x7f55240c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55240c7f80 0x7f55240c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c8540 con 0x7f550c004850 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c8820 con 0x7f55240c7f80 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.994+0000 7f5528f41700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c8b00 con 0x7f55240c65d0 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f5524158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f5524158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40426/0 (socket says 172.21.15.153:40426) 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 -- 172.21.15.153:0/1827030737 learned_addr learned my addr 172.21.15.153:0/1827030737 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:27.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f5522d9d700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55240c7f80 0x7f55240c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 -- 172.21.15.153:0/1827030737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55240c65d0 msgr2=0x7f55240c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55240c65d0 0x7f55240c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 -- 172.21.15.153:0/1827030737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55240c7f80 msgr2=0x7f55240c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55240c7f80 0x7f55240c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 -- 172.21.15.153:0/1827030737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240b3310 con 0x7f550c004850 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f552259c700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f5524158e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f551400f7a0 tx=0x7f551400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f5512ffd700 1 -- 172.21.15.153:0/1827030737 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514005320 con 0x7f550c004850 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f5512ffd700 1 -- 172.21.15.153:0/1827030737 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140054a0 con 0x7f550c004850 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55240b3040 con 0x7f550c004850 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f5512ffd700 1 -- 172.21.15.153:0/1827030737 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514006af0 con 0x7f550c004850 2023-11-28T13:01:27.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.995+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55240b35f0 con 0x7f550c004850 2023-11-28T13:01:27.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.996+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55240c8960 con 0x7f550c004850 2023-11-28T13:01:27.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.997+0000 7f5512ffd700 1 -- 172.21.15.153:0/1827030737 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f551401c030 con 0x7f550c004850 2023-11-28T13:01:27.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.997+0000 7f5512ffd700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f55040682f0 0x7f550406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.997+0000 7f5512ffd700 1 -- 172.21.15.153:0/1827030737 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5514085550 con 0x7f550c004850 2023-11-28T13:01:27.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.997+0000 7f5521d9b700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f55040682f0 0x7f550406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:26.997+0000 7f5521d9b700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f55040682f0 0x7f550406a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f551800c0e0 tx=0x7f551800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.001+0000 7f5512ffd700 1 -- 172.21.15.153:0/1827030737 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55240c8960 con 0x7f550c004850 2023-11-28T13:01:27.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.183+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55240c8960 con 0x7f550c004850 2023-11-28T13:01:27.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.184+0000 7f5512ffd700 1 -- 172.21.15.153:0/1827030737 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f55240c8960 con 0x7f550c004850 2023-11-28T13:01:27.186 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:27.187 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":39,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:14.078173+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:11.364224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:12.036978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f55040682f0 msgr2=0x7f550406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f55040682f0 0x7f550406a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f551800c0e0 tx=0x7f551800a000 comp rx=0 tx=0).stop 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 msgr2=0x7f5524158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f5524158e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f551400f7a0 tx=0x7f551400fb60 comp rx=0 tx=0).stop 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 shutdown_connections 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f55040682f0 0x7f550406a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f550c004850 0x7f5524158e70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55240c7f80 0x7f55240c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 --2- 172.21.15.153:0/1827030737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55240c65d0 0x7f55240c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 >> 172.21.15.153:0/1827030737 conn(0x7f552406aed0 msgr2=0x7f552413eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 shutdown_connections 2023-11-28T13:01:27.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.187+0000 7f5528f41700 1 -- 172.21.15.153:0/1827030737 wait complete. 2023-11-28T13:01:27.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:01:11.364224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:01:12.036978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:01:27.202 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.325+0000 7f1d44d5d700 1 Processor -- start 2023-11-28T13:01:27.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.325+0000 7f1d44d5d700 1 -- start start 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d44d5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400ce040 0x7f1d4015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d44d5d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1d400d0a00 0x7f1d4006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d44d5d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4006b420 con 0x7f1d400b2930 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d44d5d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4006b700 con 0x7f1d400d0a00 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d44d5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4006b9e0 con 0x7f1d400ce040 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1d400d0a00 0x7f1d4006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3ed9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1d400b2930 0x7f1d4006c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58600/0 (socket says 172.21.15.153:58600) 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3ed9d700 1 -- 172.21.15.153:0/1282875884 learned_addr learned my addr 172.21.15.153:0/1282875884 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3e59c700 1 --2- 172.21.15.153:0/1282875884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400ce040 0x7f1d4015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3dd9b700 1 -- 172.21.15.153:0/1282875884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400ce040 msgr2=0x7f1d4015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3dd9b700 1 --2- 172.21.15.153:0/1282875884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400ce040 0x7f1d4015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3dd9b700 1 -- 172.21.15.153:0/1282875884 >> v1:172.21.15.153:6789/0 conn(0x7f1d400b2930 legacy=0x7f1d4006c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3dd9b700 1 -- 172.21.15.153:0/1282875884 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4006bcc0 con 0x7f1d400d0a00 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3d59a700 1 -- 172.21.15.153:0/1282875884 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250468730 0 0) 0x7f1d4006b420 con 0x7f1d400b2930 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3dd9b700 1 --2- 172.21.15.153:0/1282875884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400d0a00 0x7f1d4006f850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009840 tx=0x7f1d34009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0780b35d0a7e85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3d59a700 1 -- 172.21.15.153:0/1282875884 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34011830 con 0x7f1d400d0a00 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3d59a700 1 -- 172.21.15.153:0/1282875884 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34011990 con 0x7f1d400d0a00 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.326+0000 7f1d3d59a700 1 -- 172.21.15.153:0/1282875884 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34011d20 con 0x7f1d400d0a00 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 -- 172.21.15.153:0/1282875884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400d0a00 msgr2=0x7f1d4006f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/1282875884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400d0a00 0x7f1d4006f850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009840 tx=0x7f1d34009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 -- 172.21.15.153:0/1282875884 shutdown_connections 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/1282875884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400ce040 0x7f1d4015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/1282875884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400d0a00 0x7f1d4006f850 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 -- 172.21.15.153:0/1282875884 >> 172.21.15.153:0/1282875884 conn(0x7f1d4006aed0 msgr2=0x7f1d4006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 -- 172.21.15.153:0/1282875884 shutdown_connections 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.327+0000 7f1d44d5d700 1 -- 172.21.15.153:0/1282875884 wait complete. 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.328+0000 7f1d44d5d700 1 Processor -- start 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.328+0000 7f1d44d5d700 1 -- start start 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d44d5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400b2930 0x7f1d401f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d44d5d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400ce040 0x7f1d40150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d44d5d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d400d0a00 0x7f1d400cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d44d5d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4014ee90 con 0x7f1d400ce040 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d44d5d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4014f170 con 0x7f1d400d0a00 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d44d5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4014f450 con 0x7f1d400b2930 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d3dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400ce040 0x7f1d40150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d3dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400ce040 0x7f1d40150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40442/0 (socket says 172.21.15.153:40442) 2023-11-28T13:01:27.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.329+0000 7f1d3dd9b700 1 -- 172.21.15.153:0/2431866624 learned_addr learned my addr 172.21.15.153:0/2431866624 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3ed9d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d400d0a00 0x7f1d400cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3e59c700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400b2930 0x7f1d401f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3dd9b700 1 -- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400b2930 msgr2=0x7f1d401f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3dd9b700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400b2930 0x7f1d401f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3dd9b700 1 -- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d400d0a00 msgr2=0x7f1d400cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3dd9b700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d400d0a00 0x7f1d400cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3dd9b700 1 -- 172.21.15.153:0/2431866624 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d340094a0 con 0x7f1d400ce040 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3ed9d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d400d0a00 0x7f1d400cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d3dd9b700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400ce040 0x7f1d40150140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1d340090b0 tx=0x7f1d34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d2effd700 1 -- 172.21.15.153:0/2431866624 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d340038a0 con 0x7f1d400ce040 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d2effd700 1 -- 172.21.15.153:0/2431866624 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3401f070 con 0x7f1d400ce040 2023-11-28T13:01:27.335 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4006fd90 con 0x7f1d400ce040 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d2effd700 1 -- 172.21.15.153:0/2431866624 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34003c20 con 0x7f1d400ce040 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.330+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4006bfa0 con 0x7f1d400ce040 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.331+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4006ff20 con 0x7f1d400ce040 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.331+0000 7f1d2effd700 1 -- 172.21.15.153:0/2431866624 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1d3400f050 con 0x7f1d400ce040 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.332+0000 7f1d2effd700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1d200682f0 0x7f1d2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.332+0000 7f1d2effd700 1 -- 172.21.15.153:0/2431866624 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d34021070 con 0x7f1d400ce040 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.332+0000 7f1d3e59c700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1d200682f0 0x7f1d2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.336 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.332+0000 7f1d3e59c700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1d200682f0 0x7f1d2006a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006fd0 tx=0x7f1d30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.337 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.336+0000 7f1d2effd700 1 -- 172.21.15.153:0/2431866624 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1d34053f80 con 0x7f1d400ce040 2023-11-28T13:01:27.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.519+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d40068690 con 0x7f1d400ce040 2023-11-28T13:01:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.519+0000 7f1d2effd700 1 -- 172.21.15.153:0/2431866624 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d340575a0 con 0x7f1d400ce040 2023-11-28T13:01:27.521 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:01:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1d200682f0 msgr2=0x7f1d2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1d200682f0 0x7f1d2006a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d30006fd0 tx=0x7f1d30008040 comp rx=0 tx=0).stop 2023-11-28T13:01:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400ce040 msgr2=0x7f1d40150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400ce040 0x7f1d40150140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1d340090b0 tx=0x7f1d34004750 comp rx=0 tx=0).stop 2023-11-28T13:01:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 shutdown_connections 2023-11-28T13:01:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d400d0a00 0x7f1d400cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d400b2930 0x7f1d401f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1d200682f0 0x7f1d2006a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 --2- 172.21.15.153:0/2431866624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d400ce040 0x7f1d40150140 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 >> 172.21.15.153:0/2431866624 conn(0x7f1d4006aed0 msgr2=0x7f1d400d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 shutdown_connections 2023-11-28T13:01:27.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.521+0000 7f1d44d5d700 1 -- 172.21.15.153:0/2431866624 wait complete. 2023-11-28T13:01:27.535 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:27.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.656+0000 7ff1b95fa700 1 Processor -- start 2023-11-28T13:01:27.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.656+0000 7ff1b95fa700 1 -- start start 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b95fa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4160b10 0x7ff1b40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b95fa700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff1b406b060 0x7ff1b406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b95fa700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b40c9e10 con 0x7ff1b4160740 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b95fa700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b40ca720 con 0x7ff1b406b060 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b95fa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b406efc0 con 0x7ff1b4160b10 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b37fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff1b406b060 0x7ff1b406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b2ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff1b4160740 0x7ff1b40cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58612/0 (socket says 172.21.15.153:58612) 2023-11-28T13:01:27.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b2ffd700 1 -- 172.21.15.153:0/54810362 learned_addr learned my addr 172.21.15.153:0/54810362 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b27fc700 1 --2- 172.21.15.153:0/54810362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4160b10 0x7ff1b40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b37fe700 1 -- 172.21.15.153:0/54810362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4160b10 msgr2=0x7ff1b40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b37fe700 1 --2- 172.21.15.153:0/54810362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4160b10 0x7ff1b40cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b37fe700 1 -- 172.21.15.153:0/54810362 >> v1:172.21.15.153:6789/0 conn(0x7ff1b4160740 legacy=0x7ff1b40cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b1ffb700 1 -- 172.21.15.153:0/54810362 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271213940 0 0) 0x7ff1b40c9e10 con 0x7ff1b4160740 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b37fe700 1 -- 172.21.15.153:0/54810362 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40ca0d0 con 0x7ff1b406b060 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b37fe700 1 --2- 172.21.15.153:0/54810362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b406b060 0x7ff1b406b4e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009d90 tx=0x7ff1ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f22544a110edea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b1ffb700 1 -- 172.21.15.153:0/54810362 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac00e070 con 0x7ff1b406b060 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b1ffb700 1 -- 172.21.15.153:0/54810362 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac00bbd0 con 0x7ff1b406b060 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.657+0000 7ff1b1ffb700 1 -- 172.21.15.153:0/54810362 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac012430 con 0x7ff1b406b060 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 -- 172.21.15.153:0/54810362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b406b060 msgr2=0x7ff1b406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/54810362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b406b060 0x7ff1b406b4e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009d90 tx=0x7ff1ac00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:27.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 -- 172.21.15.153:0/54810362 shutdown_connections 2023-11-28T13:01:27.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/54810362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b406b060 0x7ff1b406b4e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/54810362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4160b10 0x7ff1b40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 -- 172.21.15.153:0/54810362 >> 172.21.15.153:0/54810362 conn(0x7ff1b40b12b0 msgr2=0x7ff1b40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 -- 172.21.15.153:0/54810362 shutdown_connections 2023-11-28T13:01:27.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.658+0000 7ff1b95fa700 1 -- 172.21.15.153:0/54810362 wait complete. 2023-11-28T13:01:27.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.659+0000 7ff1b95fa700 1 Processor -- start 2023-11-28T13:01:27.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.659+0000 7ff1b95fa700 1 -- start start 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b95fa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b406b060 0x7ff1b4147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b95fa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b4160740 0x7ff1b4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b95fa700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4160b10 0x7ff1b4146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b95fa700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40ad2d0 con 0x7ff1b4160740 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b95fa700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40ad5b0 con 0x7ff1b4160b10 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b95fa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40ad890 con 0x7ff1b406b060 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b27fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b4160740 0x7ff1b4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b27fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b4160740 0x7ff1b4143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40456/0 (socket says 172.21.15.153:40456) 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b27fc700 1 -- 172.21.15.153:0/989152004 learned_addr learned my addr 172.21.15.153:0/989152004 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b37fe700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4160b10 0x7ff1b4146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.660+0000 7ff1b2ffd700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b406b060 0x7ff1b4147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b27fc700 1 -- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b406b060 msgr2=0x7ff1b4147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b27fc700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b406b060 0x7ff1b4147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b27fc700 1 -- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4160b10 msgr2=0x7ff1b4146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b27fc700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4160b10 0x7ff1b4146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b27fc700 1 -- 172.21.15.153:0/989152004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac009a70 con 0x7ff1b4160740 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b37fe700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4160b10 0x7ff1b4146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b27fc700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b4160740 0x7ff1b4143d30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400d800 tx=0x7ff1a400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff19b7fe700 1 -- 172.21.15.153:0/989152004 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40125f0 con 0x7ff1b4160740 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff19b7fe700 1 -- 172.21.15.153:0/989152004 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4012750 con 0x7ff1b4160740 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff19b7fe700 1 -- 172.21.15.153:0/989152004 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4012b00 con 0x7ff1b4160740 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b40cd8d0 con 0x7ff1b4160740 2023-11-28T13:01:27.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.661+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b40ca3b0 con 0x7ff1b4160740 2023-11-28T13:01:27.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.662+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b4160f60 con 0x7ff1b4160740 2023-11-28T13:01:27.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.662+0000 7ff19b7fe700 1 -- 172.21.15.153:0/989152004 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff1a4012c60 con 0x7ff1b4160740 2023-11-28T13:01:27.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.663+0000 7ff19b7fe700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff1a00682f0 0x7ff1a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.663+0000 7ff19b7fe700 1 -- 172.21.15.153:0/989152004 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1a4086300 con 0x7ff1b4160740 2023-11-28T13:01:27.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.663+0000 7ff1b2ffd700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff1a00682f0 0x7ff1a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.663+0000 7ff1b2ffd700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff1a00682f0 0x7ff1a006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000f60 tx=0x7ff1a8007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.667+0000 7ff19b7fe700 1 -- 172.21.15.153:0/989152004 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff1a4052ad0 con 0x7ff1b4160740 2023-11-28T13:01:27.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.848+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1b40745c0 con 0x7ff1b4160740 2023-11-28T13:01:27.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.848+0000 7ff19b7fe700 1 -- 172.21.15.153:0/989152004 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff1a40560f0 con 0x7ff1b4160740 2023-11-28T13:01:27.850 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:01:27.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff1a00682f0 msgr2=0x7ff1a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff1a00682f0 0x7ff1a006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000f60 tx=0x7ff1a8007be0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b4160740 msgr2=0x7ff1b4143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b4160740 0x7ff1b4143d30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400d800 tx=0x7ff1a400dbc0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 shutdown_connections 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4160b10 0x7ff1b4146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b406b060 0x7ff1b4147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff1a00682f0 0x7ff1a006a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 --2- 172.21.15.153:0/989152004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff1b4160740 0x7ff1b4143d30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 >> 172.21.15.153:0/989152004 conn(0x7ff1b40b12b0 msgr2=0x7ff1b406b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 shutdown_connections 2023-11-28T13:01:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.850+0000 7ff1b95fa700 1 -- 172.21.15.153:0/989152004 wait complete. 2023-11-28T13:01:27.864 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:27.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.985+0000 7fa12817a700 1 Processor -- start 2023-11-28T13:01:27.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.985+0000 7fa12817a700 1 -- start start 2023-11-28T13:01:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa12817a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa12015cef0 0x7fa1200af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa12817a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa1200cbdf0 0x7fa12006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa12817a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa12006b8c0 0x7fa120071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa12817a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12006bd40 con 0x7fa1200cbdf0 2023-11-28T13:01:27.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa12817a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1200cc5f0 con 0x7fa12015cef0 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa12817a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12015d470 con 0x7fa12006b8c0 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125715700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa1200cbdf0 0x7fa12006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125715700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa1200cbdf0 0x7fa12006b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40462/0 (socket says 172.21.15.153:40462) 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125715700 1 -- 172.21.15.153:0/1907185545 learned_addr learned my addr 172.21.15.153:0/1907185545 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125f16700 1 --2- 172.21.15.153:0/1907185545 >> v2:172.21.15.177:3300/0 conn(0x7fa12015cef0 0x7fa1200af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125715700 1 -- 172.21.15.153:0/1907185545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa12006b8c0 msgr2=0x7fa120071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125715700 1 --2- 172.21.15.153:0/1907185545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa12006b8c0 0x7fa120071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125715700 1 -- 172.21.15.153:0/1907185545 >> v2:172.21.15.177:3300/0 conn(0x7fa12015cef0 msgr2=0x7fa1200af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.986+0000 7fa125715700 1 --2- 172.21.15.153:0/1907185545 >> v2:172.21.15.177:3300/0 conn(0x7fa12015cef0 0x7fa1200af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa125715700 1 -- 172.21.15.153:0/1907185545 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12015d750 con 0x7fa1200cbdf0 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa125715700 1 --2- 172.21.15.153:0/1907185545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1200cbdf0 0x7fa12006b380 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098f0 tx=0x7fa11c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e8b785c9300088 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa124f14700 1 -- 172.21.15.153:0/1907185545 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00b860 con 0x7fa1200cbdf0 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa124f14700 1 -- 172.21.15.153:0/1907185545 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c00b9c0 con 0x7fa1200cbdf0 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa124f14700 1 -- 172.21.15.153:0/1907185545 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00bd50 con 0x7fa1200cbdf0 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 -- 172.21.15.153:0/1907185545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1200cbdf0 msgr2=0x7fa12006b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 --2- 172.21.15.153:0/1907185545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1200cbdf0 0x7fa12006b380 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098f0 tx=0x7fa11c009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 -- 172.21.15.153:0/1907185545 shutdown_connections 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 --2- 172.21.15.153:0/1907185545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa12006b8c0 0x7fa120071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 --2- 172.21.15.153:0/1907185545 >> v2:172.21.15.177:3300/0 conn(0x7fa12015cef0 0x7fa1200af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 --2- 172.21.15.153:0/1907185545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1200cbdf0 0x7fa12006b380 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 -- 172.21.15.153:0/1907185545 >> 172.21.15.153:0/1907185545 conn(0x7fa1200d0000 msgr2=0x7fa1200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 -- 172.21.15.153:0/1907185545 shutdown_connections 2023-11-28T13:01:27.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.987+0000 7fa12817a700 1 -- 172.21.15.153:0/1907185545 wait complete. 2023-11-28T13:01:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 Processor -- start 2023-11-28T13:01:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 -- start start 2023-11-28T13:01:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa12006b8c0 0x7fa1200b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1200cbdf0 0x7fa1200b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa12015cef0 0x7fa120154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200cae60 con 0x7fa12006b8c0 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200b5c50 con 0x7fa12015cef0 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa12817a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200b3510 con 0x7fa1200cbdf0 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa125f16700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa12006b8c0 0x7fa1200b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa125f16700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa12006b8c0 0x7fa1200b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40468/0 (socket says 172.21.15.153:40468) 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.989+0000 7fa125f16700 1 -- 172.21.15.153:0/3681645917 learned_addr learned my addr 172.21.15.153:0/3681645917 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa125715700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1200cbdf0 0x7fa1200b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa126717700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa12015cef0 0x7fa120154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa125f16700 1 -- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1200cbdf0 msgr2=0x7fa1200b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa125f16700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1200cbdf0 0x7fa1200b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa125f16700 1 -- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa12015cef0 msgr2=0x7fa120154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa125f16700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa12015cef0 0x7fa120154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa125f16700 1 -- 172.21.15.153:0/3681645917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c009550 con 0x7fa12006b8c0 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa126717700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa12015cef0 0x7fa120154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa125f16700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa12006b8c0 0x7fa1200b2e30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c7f0 tx=0x7fa11000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa1167fc700 1 -- 172.21.15.153:0/3681645917 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110004220 con 0x7fa12006b8c0 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa1167fc700 1 -- 172.21.15.153:0/3681645917 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa110004380 con 0x7fa12006b8c0 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12015dd10 con 0x7fa12006b8c0 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa1167fc700 1 -- 172.21.15.153:0/3681645917 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100036a0 con 0x7fa12006b8c0 2023-11-28T13:01:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.990+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12015da30 con 0x7fa12006b8c0 2023-11-28T13:01:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.991+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12015dff0 con 0x7fa12006b8c0 2023-11-28T13:01:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.991+0000 7fa1167fc700 1 -- 172.21.15.153:0/3681645917 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa110020070 con 0x7fa12006b8c0 2023-11-28T13:01:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.992+0000 7fa1167fc700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa1040682f0 0x7fa10406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.992+0000 7fa1167fc700 1 -- 172.21.15.153:0/3681645917 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa1100857b0 con 0x7fa12006b8c0 2023-11-28T13:01:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.992+0000 7fa125715700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa1040682f0 0x7fa10406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.992+0000 7fa125715700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa1040682f0 0x7fa10406a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009840 tx=0x7fa11c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:27.996+0000 7fa1167fc700 1 -- 172.21.15.153:0/3681645917 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa110051f90 con 0x7fa12006b8c0 2023-11-28T13:01:28.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.180+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1200d0e60 con 0x7fa12006b8c0 2023-11-28T13:01:28.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.180+0000 7fa1167fc700 1 -- 172.21.15.153:0/3681645917 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1100555b0 con 0x7fa12006b8c0 2023-11-28T13:01:28.182 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:01:28.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.182+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa1040682f0 msgr2=0x7fa10406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.182+0000 7fa12817a700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa1040682f0 0x7fa10406a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009840 tx=0x7fa11c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.182+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa12006b8c0 msgr2=0x7fa1200b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.182+0000 7fa12817a700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa12006b8c0 0x7fa1200b2e30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c7f0 tx=0x7fa11000cb00 comp rx=0 tx=0).stop 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 shutdown_connections 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa1040682f0 0x7fa10406a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa12006b8c0 0x7fa1200b2e30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa12015cef0 0x7fa120154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 --2- 172.21.15.153:0/3681645917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1200cbdf0 0x7fa1200b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 >> 172.21.15.153:0/3681645917 conn(0x7fa1200d0000 msgr2=0x7fa1200af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 shutdown_connections 2023-11-28T13:01:28.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.183+0000 7fa12817a700 1 -- 172.21.15.153:0/3681645917 wait complete. 2023-11-28T13:01:28.197 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:28.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.317+0000 7f7d857ee700 1 Processor -- start 2023-11-28T13:01:28.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.317+0000 7f7d857ee700 1 -- start start 2023-11-28T13:01:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d857ee700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7d8015ca50 0x7f7d8006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d857ee700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d800cce10 0x7f7d800cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d857ee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d800cf5e0 con 0x7f7d8006b8c0 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d857ee700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8006bd60 con 0x7f7d800cce10 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d857ee700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d800cd470 con 0x7f7d8015ca50 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d800cce10 0x7f7d800cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7d800cce10 0x7f7d800cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40478/0 (socket says 172.21.15.153:40478) 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/855223214 learned_addr learned my addr 172.21.15.153:0/855223214 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7effd700 1 --2- 172.21.15.153:0/855223214 >> v2:172.21.15.177:3300/0 conn(0x7f7d8015ca50 0x7f7d8006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/855223214 >> v2:172.21.15.177:3300/0 conn(0x7f7d8015ca50 msgr2=0x7f7d8006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 --2- 172.21.15.153:0/855223214 >> v2:172.21.15.177:3300/0 conn(0x7f7d8015ca50 0x7f7d8006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/855223214 >> v1:172.21.15.177:6789/0 conn(0x7f7d8006b8c0 legacy=0x7f7d800cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/855223214 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d800cecb0 con 0x7f7d800cce10 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7f7fe700 1 --2- 172.21.15.153:0/855223214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d800cce10 0x7f7d800cfa70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d80071240 tx=0x7f7d7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393f2fdfe1c434c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7dffb700 1 -- 172.21.15.153:0/855223214 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800e070 con 0x7f7d800cce10 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7dffb700 1 -- 172.21.15.153:0/855223214 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7800b8d0 con 0x7f7d800cce10 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.318+0000 7f7d7dffb700 1 -- 172.21.15.153:0/855223214 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800bc80 con 0x7f7d800cce10 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 -- 172.21.15.153:0/855223214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d800cce10 msgr2=0x7f7d800cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 --2- 172.21.15.153:0/855223214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d800cce10 0x7f7d800cfa70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d80071240 tx=0x7f7d7800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 -- 172.21.15.153:0/855223214 shutdown_connections 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 --2- 172.21.15.153:0/855223214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d800cce10 0x7f7d800cfa70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 --2- 172.21.15.153:0/855223214 >> v2:172.21.15.177:3300/0 conn(0x7f7d8015ca50 0x7f7d8006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 -- 172.21.15.153:0/855223214 >> 172.21.15.153:0/855223214 conn(0x7f7d8009a230 msgr2=0x7f7d8008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 -- 172.21.15.153:0/855223214 shutdown_connections 2023-11-28T13:01:28.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.319+0000 7f7d857ee700 1 -- 172.21.15.153:0/855223214 wait complete. 2023-11-28T13:01:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.321+0000 7f7d857ee700 1 Processor -- start 2023-11-28T13:01:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.321+0000 7f7d857ee700 1 -- start start 2023-11-28T13:01:28.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d8006b8c0 0x7f7d800adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d800cce10 0x7f7d801555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d8015ca50 0x7f7d80156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800ae170 con 0x7f7d8015ca50 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800ae450 con 0x7f7d8006b8c0 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800ae730 con 0x7f7d800cce10 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d8015ca50 0x7f7d80156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d8015ca50 0x7f7d80156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40484/0 (socket says 172.21.15.153:40484) 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/1351862269 learned_addr learned my addr 172.21.15.153:0/1351862269 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7effd700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d8006b8c0 0x7f7d800adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7e7fc700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d800cce10 0x7f7d801555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d800cce10 msgr2=0x7f7d801555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d800cce10 0x7f7d801555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d8006b8c0 msgr2=0x7f7d800adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d8006b8c0 0x7f7d800adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 -- 172.21.15.153:0/1351862269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009b30 con 0x7f7d8015ca50 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d7f7fe700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d8015ca50 0x7f7d80156dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d8006d870 tx=0x7f7d78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d677fe700 1 -- 172.21.15.153:0/1351862269 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780092e0 con 0x7f7d8015ca50 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d677fe700 1 -- 172.21.15.153:0/1351862269 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d78004960 con 0x7f7d8015ca50 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d800cf270 con 0x7f7d8015ca50 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d677fe700 1 -- 172.21.15.153:0/1351862269 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7801c680 con 0x7f7d8015ca50 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.322+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d800cef90 con 0x7f7d8015ca50 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.323+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8015ce20 con 0x7f7d8015ca50 2023-11-28T13:01:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.324+0000 7f7d677fe700 1 -- 172.21.15.153:0/1351862269 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d78022030 con 0x7f7d8015ca50 2023-11-28T13:01:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.324+0000 7f7d677fe700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7d6c0682f0 0x7f7d6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.324+0000 7f7d677fe700 1 -- 172.21.15.153:0/1351862269 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d780874c0 con 0x7f7d8015ca50 2023-11-28T13:01:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.324+0000 7f7d7effd700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7d6c0682f0 0x7f7d6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.325+0000 7f7d7effd700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7d6c0682f0 0x7f7d6c06a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d800cc1b0 tx=0x7f7d70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.328+0000 7f7d677fe700 1 -- 172.21.15.153:0/1351862269 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d78053d20 con 0x7f7d8015ca50 2023-11-28T13:01:28.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.512+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d800745c0 con 0x7f7d8015ca50 2023-11-28T13:01:28.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.512+0000 7f7d677fe700 1 -- 172.21.15.153:0/1351862269 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d78057340 con 0x7f7d8015ca50 2023-11-28T13:01:28.514 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:01:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7d6c0682f0 msgr2=0x7f7d6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7d6c0682f0 0x7f7d6c06a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d800cc1b0 tx=0x7f7d70008040 comp rx=0 tx=0).stop 2023-11-28T13:01:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d8015ca50 msgr2=0x7f7d80156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d8015ca50 0x7f7d80156dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d8006d870 tx=0x7f7d78004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 shutdown_connections 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7d6c0682f0 0x7f7d6c06a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d8015ca50 0x7f7d80156dd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d8006b8c0 0x7f7d800adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 --2- 172.21.15.153:0/1351862269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d800cce10 0x7f7d801555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 >> 172.21.15.153:0/1351862269 conn(0x7f7d8009a230 msgr2=0x7f7d800cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 shutdown_connections 2023-11-28T13:01:28.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.514+0000 7f7d857ee700 1 -- 172.21.15.153:0/1351862269 wait complete. 2023-11-28T13:01:28.528 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.648+0000 7f5fb57a6700 1 Processor -- start 2023-11-28T13:01:28.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.648+0000 7f5fb57a6700 1 -- start start 2023-11-28T13:01:28.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fb57a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb015ca50 0x7f5fb006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fb57a6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5fb006b8c0 0x7f5fb00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fb57a6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5fb00ccc40 0x7f5fb00cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fb57a6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb00cd470 con 0x7f5fb00ccc40 2023-11-28T13:01:28.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fb57a6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb006bd30 con 0x7f5fb006b8c0 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fb57a6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00cf5e0 con 0x7f5fb015ca50 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5fb00ccc40 0x7f5fb00cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5fb00ccc40 0x7f5fb00cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40492/0 (socket says 172.21.15.153:40492) 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 -- 172.21.15.153:0/2941337058 learned_addr learned my addr 172.21.15.153:0/2941337058 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fae7fc700 1 --2- 172.21.15.153:0/2941337058 >> v2:172.21.15.177:3300/0 conn(0x7f5fb006b8c0 0x7f5fb00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faeffd700 1 --2- 172.21.15.153:0/2941337058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb015ca50 0x7f5fb006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 -- 172.21.15.153:0/2941337058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb015ca50 msgr2=0x7f5fb006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 --2- 172.21.15.153:0/2941337058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb015ca50 0x7f5fb006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 -- 172.21.15.153:0/2941337058 >> v2:172.21.15.177:3300/0 conn(0x7f5fb006b8c0 msgr2=0x7f5fb00cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 --2- 172.21.15.153:0/2941337058 >> v2:172.21.15.177:3300/0 conn(0x7f5fb006b8c0 0x7f5fb00cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 -- 172.21.15.153:0/2941337058 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00cecb0 con 0x7f5fb00ccc40 2023-11-28T13:01:28.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5faf7fe700 1 --2- 172.21.15.153:0/2941337058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ccc40 0x7f5fb00cd7e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00ceae0 tx=0x7f5fa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cf0f5e58b9def1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.649+0000 7f5fadffb700 1 -- 172.21.15.153:0/2941337058 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800e070 con 0x7f5fb00ccc40 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fadffb700 1 -- 172.21.15.153:0/2941337058 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa800b980 con 0x7f5fb00ccc40 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fadffb700 1 -- 172.21.15.153:0/2941337058 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800bd30 con 0x7f5fb00ccc40 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2941337058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ccc40 msgr2=0x7f5fb00cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2941337058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ccc40 0x7f5fb00cd7e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00ceae0 tx=0x7f5fa800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2941337058 shutdown_connections 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2941337058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb015ca50 0x7f5fb006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2941337058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ccc40 0x7f5fb00cd7e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2941337058 >> v2:172.21.15.177:3300/0 conn(0x7f5fb006b8c0 0x7f5fb00cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2941337058 >> 172.21.15.153:0/2941337058 conn(0x7f5fb009a230 msgr2=0x7f5fb008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2941337058 shutdown_connections 2023-11-28T13:01:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.650+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2941337058 wait complete. 2023-11-28T13:01:28.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 Processor -- start 2023-11-28T13:01:28.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 -- start start 2023-11-28T13:01:28.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb006b8c0 0x7f5fb015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb00ccc40 0x7f5fb015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb015ca50 0x7f5fb00b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00c5010 con 0x7f5fb006b8c0 2023-11-28T13:01:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00b4860 con 0x7f5fb015ca50 2023-11-28T13:01:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5fb57a6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00b3f30 con 0x7f5fb00ccc40 2023-11-28T13:01:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb006b8c0 0x7f5fb015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb006b8c0 0x7f5fb015ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40494/0 (socket says 172.21.15.153:40494) 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.652+0000 7f5faeffd700 1 -- 172.21.15.153:0/2324914324 learned_addr learned my addr 172.21.15.153:0/2324914324 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faf7fe700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb015ca50 0x7f5fb00b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5fae7fc700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb00ccc40 0x7f5fb015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faeffd700 1 -- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb00ccc40 msgr2=0x7f5fb015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faeffd700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb00ccc40 0x7f5fb015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faeffd700 1 -- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb015ca50 msgr2=0x7f5fb00b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faeffd700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb015ca50 0x7f5fb00b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faeffd700 1 -- 172.21.15.153:0/2324914324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa8009b30 con 0x7f5fb006b8c0 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faf7fe700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb015ca50 0x7f5fb00b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:28.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5fae7fc700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb00ccc40 0x7f5fb015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5faeffd700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb006b8c0 0x7f5fb015ad80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400d800 tx=0x7f5fa400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5f977fe700 1 -- 172.21.15.153:0/2324914324 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40125f0 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5f977fe700 1 -- 172.21.15.153:0/2324914324 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa4012750 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb00cf270 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5f977fe700 1 -- 172.21.15.153:0/2324914324 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4012b00 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.653+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb00cef90 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.654+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb00cd0c0 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.655+0000 7f5f977fe700 1 -- 172.21.15.153:0/2324914324 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fa4012c60 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.655+0000 7f5f977fe700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f9c0682f0 0x7f5f9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.655+0000 7f5f977fe700 1 -- 172.21.15.153:0/2324914324 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5fa4086220 con 0x7f5fb006b8c0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.655+0000 7f5fae7fc700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f9c0682f0 0x7f5f9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.655+0000 7f5fae7fc700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f9c0682f0 0x7f5f9c06a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0156ee0 tx=0x7f5fa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.659+0000 7f5f977fe700 1 -- 172.21.15.153:0/2324914324 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5fa4052a00 con 0x7f5fb006b8c0 2023-11-28T13:01:28.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.858+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5fb00745c0 con 0x7f5fb006b8c0 2023-11-28T13:01:28.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.859+0000 7f5f977fe700 1 -- 172.21.15.153:0/2324914324 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5fa4056020 con 0x7f5fb006b8c0 2023-11-28T13:01:28.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.860+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f9c0682f0 msgr2=0x7f5f9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.860+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f9c0682f0 0x7f5f9c06a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0156ee0 tx=0x7f5fa0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:28.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.860+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb006b8c0 msgr2=0x7f5fb015ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.860+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb006b8c0 0x7f5fb015ad80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400d800 tx=0x7f5fa400dbc0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 shutdown_connections 2023-11-28T13:01:28.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5f9c0682f0 0x7f5f9c06a7a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb006b8c0 0x7f5fb015ad80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb015ca50 0x7f5fb00b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 --2- 172.21.15.153:0/2324914324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb00ccc40 0x7f5fb015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 >> 172.21.15.153:0/2324914324 conn(0x7f5fb009a230 msgr2=0x7f5fb00d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 shutdown_connections 2023-11-28T13:01:28.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:28.861+0000 7f5fb57a6700 1 -- 172.21.15.153:0/2324914324 wait complete. 2023-11-28T13:01:28.864 INFO:teuthology.orchestra.run.smithi153.stderr:listed 0 entries 2023-11-28T13:01:28.874 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:01:28.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:28.875 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:28.932 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:01:28.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:28.933 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:01:28.963 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:29.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.126+0000 7fd3743fc700 1 Processor -- start 2023-11-28T13:01:29.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.126+0000 7fd3743fc700 1 -- start start 2023-11-28T13:01:29.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd3743fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd36c140950 0x7fd36c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd3743fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c0b2e70 0x7fd36c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd3743fc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36c13ff00 con 0x7fd36c0cd8c0 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd3743fc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36c1404c0 con 0x7fd36c140950 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd3743fc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c140dc0 con 0x7fd36c0b2e70 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd372999700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c0b2e70 0x7fd36c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371997700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd36c140950 0x7fd36c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371997700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd36c140950 0x7fd36c0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:49970/0 (socket says 172.21.15.153:49970) 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd372999700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c0b2e70 0x7fd36c06f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:58116/0 (socket says 172.21.15.153:58116) 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371997700 1 -- 172.21.15.153:0/3137597353 learned_addr learned my addr 172.21.15.153:0/3137597353 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd372198700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd36c0cd8c0 0x7fd36c0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:35280/0 (socket says 172.21.15.153:35280) 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371997700 1 -- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c0b2e70 msgr2=0x7fd36c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:29.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371997700 1 --2- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c0b2e70 0x7fd36c06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371997700 1 -- 172.21.15.153:0/3137597353 >> v1:172.21.15.177:6789/0 conn(0x7fd36c0cd8c0 legacy=0x7fd36c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371997700 1 -- 172.21.15.153:0/3137597353 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c15aa80 con 0x7fd36c140950 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd371196700 1 -- 172.21.15.153:0/3137597353 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213441659 0 0) 0x7fd36c13ff00 con 0x7fd36c0cd8c0 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.127+0000 7fd372999700 1 --2- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c0b2e70 0x7fd36c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd371997700 1 --2- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c140950 0x7fd36c0b2930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0098f0 tx=0x7fd35c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ffb0617b2f263 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd371196700 1 -- 172.21.15.153:0/3137597353 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c011830 con 0x7fd36c140950 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd371196700 1 -- 172.21.15.153:0/3137597353 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c011990 con 0x7fd36c140950 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd371196700 1 -- 172.21.15.153:0/3137597353 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c011d20 con 0x7fd36c140950 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 -- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c140950 msgr2=0x7fd36c0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 --2- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c140950 0x7fd36c0b2930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0098f0 tx=0x7fd35c009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 -- 172.21.15.153:0/3137597353 shutdown_connections 2023-11-28T13:01:29.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 --2- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c0b2e70 0x7fd36c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 --2- 172.21.15.153:0/3137597353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c140950 0x7fd36c0b2930 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 -- 172.21.15.153:0/3137597353 >> 172.21.15.153:0/3137597353 conn(0x7fd36c06aed0 msgr2=0x7fd36c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 -- 172.21.15.153:0/3137597353 shutdown_connections 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.128+0000 7fd3743fc700 1 -- 172.21.15.153:0/3137597353 wait complete. 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 Processor -- start 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 -- start start 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c0b2e70 0x7fd36c0b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0cd8c0 0x7fd36c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c140950 0x7fd36c13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0b40b0 con 0x7fd36c0b2e70 2023-11-28T13:01:29.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0b6b70 con 0x7fd36c0cd8c0 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd3743fc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0b4350 con 0x7fd36c140950 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd372198700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c0b2e70 0x7fd36c0b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd372198700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c0b2e70 0x7fd36c0b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40508/0 (socket says 172.21.15.153:40508) 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.130+0000 7fd372198700 1 -- 172.21.15.153:0/2739749305 learned_addr learned my addr 172.21.15.153:0/2739749305 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd371997700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0cd8c0 0x7fd36c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372999700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c140950 0x7fd36c13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372198700 1 -- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c140950 msgr2=0x7fd36c13ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372198700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c140950 0x7fd36c13ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372198700 1 -- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0cd8c0 msgr2=0x7fd36c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372198700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0cd8c0 0x7fd36c0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372198700 1 -- 172.21.15.153:0/2739749305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364009a70 con 0x7fd36c0b2e70 2023-11-28T13:01:29.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372999700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c140950 0x7fd36c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd372198700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c0b2e70 0x7fd36c0b6630 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c4e0 tx=0x7fd36800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd362ffd700 1 -- 172.21.15.153:0/2739749305 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368004cd0 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd362ffd700 1 -- 172.21.15.153:0/2739749305 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368004e30 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c009550 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd362ffd700 1 -- 172.21.15.153:0/2739749305 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680072f0 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.131+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36c0b35f0 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.132+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36c0b3310 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.133+0000 7fd362ffd700 1 -- 172.21.15.153:0/2739749305 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd36801a020 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.133+0000 7fd362ffd700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3500682f0 0x7fd35006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.133+0000 7fd362ffd700 1 -- 172.21.15.153:0/2739749305 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd3680858f0 con 0x7fd36c0b2e70 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.133+0000 7fd371997700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3500682f0 0x7fd35006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.133+0000 7fd371997700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3500682f0 0x7fd35006a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0098f0 tx=0x7fd35c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.137+0000 7fd362ffd700 1 -- 172.21.15.153:0/2739749305 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd368052190 con 0x7fd36c0b2e70 2023-11-28T13:01:29.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.361+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd36c140150 con 0x7fd36c0b2e70 2023-11-28T13:01:29.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.362+0000 7fd362ffd700 1 -- 172.21.15.153:0/2739749305 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fd3680557b0 con 0x7fd36c0b2e70 2023-11-28T13:01:29.364 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:29.364 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:29.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3500682f0 msgr2=0x7fd35006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3500682f0 0x7fd35006a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0098f0 tx=0x7fd35c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:29.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c0b2e70 msgr2=0x7fd36c0b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c0b2e70 0x7fd36c0b6630 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c4e0 tx=0x7fd36800c7f0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 shutdown_connections 2023-11-28T13:01:29.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3500682f0 0x7fd35006a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd36c0b2e70 0x7fd36c0b6630 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c140950 0x7fd36c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 --2- 172.21.15.153:0/2739749305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0cd8c0 0x7fd36c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 >> 172.21.15.153:0/2739749305 conn(0x7fd36c06aed0 msgr2=0x7fd36c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 shutdown_connections 2023-11-28T13:01:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.364+0000 7fd3743fc700 1 -- 172.21.15.153:0/2739749305 wait complete. 2023-11-28T13:01:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:29.378 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T13:01:29.527 INFO:teuthology.orchestra.run.smithi153.stdout:{"message":"","return_code":0} 2023-11-28T13:01:29.527 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:01:29.528 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.645+0000 7fcbfaa6a700 1 Processor -- start 2023-11-28T13:01:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.645+0000 7fcbfaa6a700 1 -- start start 2023-11-28T13:01:29.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.646+0000 7fcbfaa6a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 0x7fcbf406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.646+0000 7fcbfaa6a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf413ff00 con 0x7fcbf40cd8c0 2023-11-28T13:01:29.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.646+0000 7fcbfaa6a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf41404c0 con 0x7fcbf4140950 2023-11-28T13:01:29.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.646+0000 7fcbfaa6a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4140dc0 con 0x7fcbf406f850 2023-11-28T13:01:29.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.646+0000 7fcbf8806700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcbf40cd8c0 0x7fcbf40cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58680/0 (socket says 172.21.15.153:58680) 2023-11-28T13:01:29.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.646+0000 7fcbf8806700 1 -- 172.21.15.153:0/3936533636 learned_addr learned my addr 172.21.15.153:0/3936533636 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:29.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.646+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 0x7fcbf406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662385566 0 0) 0x7fcbf413ff00 con 0x7fcbf40cd8c0 2023-11-28T13:01:29.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe40035b0 con 0x7fcbf40cd8c0 2023-11-28T13:01:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804154610 0 0) 0x7fcbf41404c0 con 0x7fcbf4140950 2023-11-28T13:01:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf413ff00 con 0x7fcbf4140950 2023-11-28T13:01:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3634198303 0 0) 0x7fcbe40035b0 con 0x7fcbf40cd8c0 2023-11-28T13:01:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 msgr2=0x7fcbf406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 0x7fcbf406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 >> v1:172.21.15.177:6789/0 conn(0x7fcbf4140950 legacy=0x7fcbf40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf415aa80 con 0x7fcbf40cd8c0 2023-11-28T13:01:29.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fcbe80017f0 con 0x7fcbf40cd8c0 2023-11-28T13:01:29.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 >> v1:172.21.15.153:6789/0 conn(0x7fcbf40cd8c0 legacy=0x7fcbf40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4003bc0 0x7fcbe4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 0x7fcbe4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe4004d90 0x7fcbe4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4140dc0 con 0x7fcbe4004d90 2023-11-28T13:01:29.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe400bb40 con 0x7fcbe4003bc0 2023-11-28T13:01:29.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe400bd00 con 0x7fcbf406f850 2023-11-28T13:01:29.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe4004d90 0x7fcbe4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf8806700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4003bc0 0x7fcbe4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf3fff700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 0x7fcbe4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf9007700 1 -- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 msgr2=0x7fcbe4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf406f850 0x7fcbe4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf9007700 1 -- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4003bc0 msgr2=0x7fcbe4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4003bc0 0x7fcbe4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf9007700 1 -- 172.21.15.153:0/3936533636 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe40035b0 con 0x7fcbe4004d90 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.647+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe4004d90 0x7fcbe4005200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009fc0 tx=0x7fcbe4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ad95b5e3ca72fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00ec30 con 0x7fcbe4004d90 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec00f040 con 0x7fcbe4004d90 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbf37fe700 1 -- 172.21.15.153:0/3936533636 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec014480 con 0x7fcbe4004d90 2023-11-28T13:01:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3936533636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe4004d90 msgr2=0x7fcbe4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbfaa6a700 1 --2- 172.21.15.153:0/3936533636 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe4004d90 0x7fcbe4005200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009fc0 tx=0x7fcbe4005880 comp rx=0 tx=0).stop 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbf9007700 1 -- 172.21.15.153:0/3936533636 reap_dead start 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3936533636 shutdown_connections 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3936533636 >> 172.21.15.153:0/3936533636 conn(0x7fcbf406aed0 msgr2=0x7fcbf406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3936533636 shutdown_connections 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.648+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3936533636 wait complete. 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.649+0000 7fcbfaa6a700 1 Processor -- start 2023-11-28T13:01:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.649+0000 7fcbfaa6a700 1 -- start start 2023-11-28T13:01:29.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbfaa6a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4004d90 0x7fcbf41efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbfaa6a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf40c7520 0x7fcbf40b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbfaa6a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf4150b40 0x7fcbf40b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbfaa6a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c7f80 con 0x7fcbf4150b40 2023-11-28T13:01:29.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbfaa6a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c4ef0 con 0x7fcbe4004d90 2023-11-28T13:01:29.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbfaa6a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c8790 con 0x7fcbf40c7520 2023-11-28T13:01:29.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbf9007700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf4150b40 0x7fcbf40b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbf9007700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf4150b40 0x7fcbf40b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40532/0 (socket says 172.21.15.153:40532) 2023-11-28T13:01:29.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.650+0000 7fcbf9007700 1 -- 172.21.15.153:0/3828729584 learned_addr learned my addr 172.21.15.153:0/3828729584 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:29.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf8806700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4004d90 0x7fcbf41efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf3fff700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf40c7520 0x7fcbf40b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf9007700 1 -- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf40c7520 msgr2=0x7fcbf40b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf40c7520 0x7fcbf40b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf9007700 1 -- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4004d90 msgr2=0x7fcbf41efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4004d90 0x7fcbf41efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf9007700 1 -- 172.21.15.153:0/3828729584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf40b3310 con 0x7fcbf4150b40 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf3fff700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf40c7520 0x7fcbf40b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf8806700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4004d90 0x7fcbf41efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf9007700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf4150b40 0x7fcbf40b29b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009f60 tx=0x7fcbec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf17fa700 1 -- 172.21.15.153:0/3828729584 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec007000 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf17fa700 1 -- 172.21.15.153:0/3828729584 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec004470 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf40b3040 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbf17fa700 1 -- 172.21.15.153:0/3828729584 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec004820 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.651+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf40b35f0 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.652+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf40c5060 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.652+0000 7fcbf17fa700 1 -- 172.21.15.153:0/3828729584 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbec014480 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.653+0000 7fcbf17fa700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbd8068320 0x7fcbd806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.653+0000 7fcbf17fa700 1 -- 172.21.15.153:0/3828729584 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbec087850 con 0x7fcbf4150b40 2023-11-28T13:01:29.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.653+0000 7fcbf8806700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbd8068320 0x7fcbd806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.653+0000 7fcbf8806700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbd8068320 0x7fcbd806a7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000ef0 tx=0x7fcbe8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.657+0000 7fcbf17fa700 1 -- 172.21.15.153:0/3828729584 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbec054050 con 0x7fcbf4150b40 2023-11-28T13:01:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.882+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbf4068690 con 0x7fcbf4150b40 2023-11-28T13:01:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.883+0000 7fcbf17fa700 1 -- 172.21.15.153:0/3828729584 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fcbec057670 con 0x7fcbf4150b40 2023-11-28T13:01:29.885 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:29.885 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbd8068320 msgr2=0x7fcbd806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbd8068320 0x7fcbd806a7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000ef0 tx=0x7fcbe8009040 comp rx=0 tx=0).stop 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf4150b40 msgr2=0x7fcbf40b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf4150b40 0x7fcbf40b29b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009f60 tx=0x7fcbec009750 comp rx=0 tx=0).stop 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 shutdown_connections 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbd8068320 0x7fcbd806a7d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbf4150b40 0x7fcbf40b29b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbe4004d90 0x7fcbf41efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 --2- 172.21.15.153:0/3828729584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf40c7520 0x7fcbf40b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 >> 172.21.15.153:0/3828729584 conn(0x7fcbf406aed0 msgr2=0x7fcbf406fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 shutdown_connections 2023-11-28T13:01:29.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:29.885+0000 7fcbfaa6a700 1 -- 172.21.15.153:0/3828729584 wait complete. 2023-11-28T13:01:29.888 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:29.900 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2023-11-28T13:01:30.041 INFO:teuthology.orchestra.run.smithi153.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T13:01:30.041 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T13:01:40.042 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T13:01:40.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.092+0000 7f22ff89a080 1 Processor -- start 2023-11-28T13:01:40.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.092+0000 7f22ff89a080 1 -- start start 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22ff89a080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 0x5594f1c346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22ff89a080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c3e450 0x5594f1c3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22ff89a080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c42c10 0x5594f1b94930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22ff89a080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f1aa0fe0 con 0x5594f1c42c10 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22ff89a080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f1ba7240 con 0x5594f1c3e450 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22ff89a080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594f1b944e0 con 0x5594f1bb09d0 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c42c10 0x5594f1b94930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c42c10 0x5594f1b94930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40546/0 (socket says 172.21.15.153:40546) 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 -- 172.21.15.153:0/3917127371 learned_addr learned my addr 172.21.15.153:0/3917127371 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f519c700 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c3e450 0x5594f1c3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f599d700 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 0x5594f1c346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 -- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 msgr2=0x5594f1c346c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 0x5594f1c346c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 -- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c3e450 msgr2=0x5594f1c3e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c3e450 0x5594f1c3e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 -- 172.21.15.153:0/3917127371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec005d20 con 0x5594f1c42c10 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f519c700 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c3e450 0x5594f1c3e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.093+0000 7f22f619e700 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c42c10 0x5594f1b94930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001b20 tx=0x7f22f0006760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5339230e4b75e9a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22e7fff700 1 -- 172.21.15.153:0/3917127371 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f00062e0 con 0x5594f1c42c10 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22e7fff700 1 -- 172.21.15.153:0/3917127371 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f0003450 con 0x5594f1c42c10 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22e7fff700 1 -- 172.21.15.153:0/3917127371 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f00037e0 con 0x5594f1c42c10 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 -- 172.21.15.153:0/3917127371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c42c10 msgr2=0x5594f1b94930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c42c10 0x5594f1b94930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f22f0001b20 tx=0x7f22f0006760 comp rx=0 tx=0).stop 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 -- 172.21.15.153:0/3917127371 shutdown_connections 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c42c10 0x5594f1b94930 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 0x5594f1c346c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 --2- 172.21.15.153:0/3917127371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c3e450 0x5594f1c3e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 -- 172.21.15.153:0/3917127371 >> 172.21.15.153:0/3917127371 conn(0x5594f1b92b60 msgr2=0x5594f1c3d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 -- 172.21.15.153:0/3917127371 shutdown_connections 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.094+0000 7f22ff89a080 1 -- 172.21.15.153:0/3917127371 wait complete. 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.095+0000 7f22ff89a080 1 Processor -- start 2023-11-28T13:01:40.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.095+0000 7f22ff89a080 1 -- start start 2023-11-28T13:01:40.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22ff89a080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 0x5594f1c30c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22ff89a080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c3e450 0x5594f1b92f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22ff89a080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c42c10 0x5594f1b93470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22ff89a080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f1b65860 con 0x5594f1c3e450 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22ff89a080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594f1c312f0 con 0x5594f1c42c10 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22ff89a080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594f1ba8240 con 0x5594f1bb09d0 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c3e450 0x5594f1b92f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c3e450 0x5594f1b92f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40550/0 (socket says 172.21.15.153:40550) 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 -- 172.21.15.153:0/4100316427 learned_addr learned my addr 172.21.15.153:0/4100316427 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f619e700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c42c10 0x5594f1b93470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f599d700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 0x5594f1c30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 -- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 msgr2=0x5594f1c30c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5594f1bb09d0 0x5594f1c30c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 -- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c42c10 msgr2=0x5594f1b93470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594f1c42c10 0x5594f1b93470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 -- 172.21.15.153:0/4100316427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0001750 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22f519c700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c3e450 0x5594f1b92f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f22ec008500 tx=0x7f22ec008810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22e6ffd700 1 -- 172.21.15.153:0/4100316427 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00f050 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22e6ffd700 1 -- 172.21.15.153:0/4100316427 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec010040 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.096+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5594f1aac830 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.097+0000 7f22e6ffd700 1 -- 172.21.15.153:0/4100316427 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec006990 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.097+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e8005d20 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.098+0000 7f22e6ffd700 1 -- 172.21.15.153:0/4100316427 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f22ec006af0 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.098+0000 7f22e6ffd700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22d8066c10 0x7f22d80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.098+0000 7f22e6ffd700 1 -- 172.21.15.153:0/4100316427 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22ec084b20 con 0x5594f1c3e450 2023-11-28T13:01:40.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.098+0000 7f22f599d700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22d8066c10 0x7f22d80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.098+0000 7f22ff89a080 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x5594f1b963e0 0x5594f1cd8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.099+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 --> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x5594f1baf010 con 0x5594f1b963e0 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.099+0000 7f22f619e700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x5594f1b963e0 0x5594f1cd8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.099+0000 7f22f599d700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22d8066c10 0x7f22d80690c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f22e8000c00 tx=0x7f22e800e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.099+0000 7f22f619e700 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x5594f1b963e0 0x5594f1cd8340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.099+0000 7f22f619e700 1 -- 172.21.15.153:0/4100316427 <== osd.7 v2:172.21.15.177:6808/103771 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f22f00035c0 con 0x5594f1b963e0 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x5594f1b963e0 msgr2=0x5594f1cd8340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x5594f1b963e0 0x5594f1cd8340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22d8066c10 msgr2=0x7f22d80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22d8066c10 0x7f22d80690c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f22e8000c00 tx=0x7f22e800e000 comp rx=0 tx=0).stop 2023-11-28T13:01:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c3e450 msgr2=0x5594f1b92f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 --2- 172.21.15.153:0/4100316427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5594f1c3e450 0x5594f1b92f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f22ec008500 tx=0x7f22ec008810 comp rx=0 tx=0).stop 2023-11-28T13:01:40.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22f619e700 1 -- 172.21.15.153:0/4100316427 reap_dead start 2023-11-28T13:01:40.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 shutdown_connections 2023-11-28T13:01:40.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 >> 172.21.15.153:0/4100316427 conn(0x5594f1b92b60 msgr2=0x5594f1b94910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 shutdown_connections 2023-11-28T13:01:40.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.100+0000 7f22ff89a080 1 -- 172.21.15.153:0/4100316427 wait complete. 2023-11-28T13:01:40.107 INFO:teuthology.orchestra.run.smithi153.stdout:mytag 2023-11-28T13:01:40.107 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T13:01:40.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.152+0000 7f5fc65ed080 1 Processor -- start 2023-11-28T13:01:40.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fc65ed080 1 -- start start 2023-11-28T13:01:40.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fc65ed080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f690839d0 0x560f69079b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fc65ed080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f690675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fc65ed080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f69115c10 0x560f69113c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fc65ed080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f68f73fe0 con 0x560f69111450 2023-11-28T13:01:40.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fc65ed080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f69069a60 con 0x560f69115c10 2023-11-28T13:01:40.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fc65ed080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f69069bd0 con 0x560f690839d0 2023-11-28T13:01:40.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fbbeef700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f690675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fbbeef700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f690675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40564/0 (socket says 172.21.15.153:40564) 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fbbeef700 1 -- 172.21.15.153:0/2830758207 learned_addr learned my addr 172.21.15.153:0/2830758207 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fbcef1700 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f69115c10 0x560f69113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.153+0000 7f5fbc6f0700 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f690839d0 0x560f69079b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbbeef700 1 -- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f690839d0 msgr2=0x560f69079b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbbeef700 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f690839d0 0x560f69079b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbbeef700 1 -- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f69115c10 msgr2=0x560f69113c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbbeef700 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f69115c10 0x560f69113c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbbeef700 1 -- 172.21.15.153:0/2830758207 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb0005d20 con 0x560f69111450 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbc6f0700 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f690839d0 0x560f69079b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbbeef700 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f690675b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009790 tx=0x7f5fac009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c37108c559bfab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbaeed700 1 -- 172.21.15.153:0/2830758207 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00e070 con 0x560f69111450 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbaeed700 1 -- 172.21.15.153:0/2830758207 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac012410 con 0x560f69111450 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fbaeed700 1 -- 172.21.15.153:0/2830758207 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac012780 con 0x560f69111450 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 -- 172.21.15.153:0/2830758207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 msgr2=0x560f690675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f690675b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009790 tx=0x7f5fac009aa0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 -- 172.21.15.153:0/2830758207 shutdown_connections 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f69115c10 0x560f69113c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f690839d0 0x560f69079b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/2830758207 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f690675b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 -- 172.21.15.153:0/2830758207 >> 172.21.15.153:0/2830758207 conn(0x560f69065b60 msgr2=0x560f6907a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 -- 172.21.15.153:0/2830758207 shutdown_connections 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.154+0000 7f5fc65ed080 1 -- 172.21.15.153:0/2830758207 wait complete. 2023-11-28T13:01:40.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.155+0000 7f5fc65ed080 1 Processor -- start 2023-11-28T13:01:40.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.155+0000 7f5fc65ed080 1 -- start start 2023-11-28T13:01:40.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fc65ed080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f690839d0 0x560f6910e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fc65ed080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f6905f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fc65ed080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f69115c10 0x560f690675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fc65ed080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f69110030 con 0x560f69111450 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fc65ed080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6910f420 con 0x560f690839d0 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fc65ed080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f6910f700 con 0x560f69115c10 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fbbeef700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f6905f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fbbeef700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f6905f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40570/0 (socket says 172.21.15.153:40570) 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.156+0000 7f5fbbeef700 1 -- 172.21.15.153:0/1170934619 learned_addr learned my addr 172.21.15.153:0/1170934619 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbc6f0700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f690839d0 0x560f6910e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbcef1700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f69115c10 0x560f690675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbbeef700 1 -- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f69115c10 msgr2=0x560f690675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbbeef700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f69115c10 0x560f690675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbbeef700 1 -- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f690839d0 msgr2=0x560f6910e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbbeef700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f690839d0 0x560f6910e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbbeef700 1 -- 172.21.15.153:0/1170934619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac009410 con 0x560f69111450 2023-11-28T13:01:40.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbcef1700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560f69115c10 0x560f690675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fbbeef700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f6905f290 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5fac015040 tx=0x7f5fac003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fb9eeb700 1 -- 172.21.15.153:0/1170934619 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac01c850 con 0x560f69111450 2023-11-28T13:01:40.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fb9eeb700 1 -- 172.21.15.153:0/1170934619 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac01c9b0 con 0x560f69111450 2023-11-28T13:01:40.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f68f8c120 con 0x560f69111450 2023-11-28T13:01:40.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fb9eeb700 1 -- 172.21.15.153:0/1170934619 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac01cd60 con 0x560f69111450 2023-11-28T13:01:40.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.157+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f68f7f830 con 0x560f69111450 2023-11-28T13:01:40.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.158+0000 7f5fb9eeb700 1 -- 172.21.15.153:0/1170934619 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fac0073f0 con 0x560f69111450 2023-11-28T13:01:40.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.159+0000 7f5fb9eeb700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5fa8066c10 0x7f5fa80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.159+0000 7f5fb9eeb700 1 -- 172.21.15.153:0/1170934619 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5fac085a30 con 0x560f69111450 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.159+0000 7f5fbc6f0700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5fa8066c10 0x7f5fa80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.159+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x560f6905f7d0 0x560f6907c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.159+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 --> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x560f6907cbc0 con 0x560f6905f7d0 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.159+0000 7f5fbc6f0700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5fa8066c10 0x7f5fa80690c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000a440 tx=0x7f5fb000f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.159+0000 7f5fbcef1700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x560f6905f7d0 0x560f6907c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fbcef1700 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x560f6905f7d0 0x560f6907c680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fbcef1700 1 -- 172.21.15.153:0/1170934619 <== osd.5 v2:172.21.15.177:6800/103770 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f5fb4003450 con 0x560f6905f7d0 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x560f6905f7d0 msgr2=0x560f6907c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x560f6905f7d0 0x560f6907c680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5fa8066c10 msgr2=0x7f5fa80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5fa8066c10 0x7f5fa80690c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000a440 tx=0x7f5fb000f000 comp rx=0 tx=0).stop 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 msgr2=0x560f6905f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 --2- 172.21.15.153:0/1170934619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x560f69111450 0x560f6905f290 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5fac015040 tx=0x7f5fac003020 comp rx=0 tx=0).stop 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fbcef1700 1 -- 172.21.15.153:0/1170934619 reap_dead start 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 shutdown_connections 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 >> 172.21.15.153:0/1170934619 conn(0x560f69065b60 msgr2=0x560f6907a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 shutdown_connections 2023-11-28T13:01:40.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.160+0000 7f5fc65ed080 1 -- 172.21.15.153:0/1170934619 wait complete. 2023-11-28T13:01:40.170 INFO:teuthology.orchestra.run.smithi153.stdout:mytag 2023-11-28T13:01:40.171 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T13:01:40.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.214+0000 7f5dfbb06080 1 Processor -- start 2023-11-28T13:01:40.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.214+0000 7f5dfbb06080 1 -- start start 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5dfbb06080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a35c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5dfbb06080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b0e450 0x565234b9ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5dfbb06080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b12c10 0x565234ba0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5dfbb06080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565234b0bbd0 con 0x565234a809d0 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5dfbb06080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x565234b0d290 con 0x565234b12c10 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5dfbb06080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x565234a64930 con 0x565234b0e450 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a35c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a35c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40586/0 (socket says 172.21.15.153:40586) 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 -- 172.21.15.153:0/2583424360 learned_addr learned my addr 172.21.15.153:0/2583424360 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df240a700 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b12c10 0x565234ba0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1408700 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b0e450 0x565234b9ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 -- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b0e450 msgr2=0x565234b9ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b0e450 0x565234b9ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 -- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b12c10 msgr2=0x565234ba0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b12c10 0x565234ba0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 -- 172.21.15.153:0/2583424360 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de4005de0 con 0x565234a809d0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1408700 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b0e450 0x565234b9ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.215+0000 7f5df1c09700 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a35c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5de80095a0 tx=0x7f5de80098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603b14798c8fc293 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5de3fff700 1 -- 172.21.15.153:0/2583424360 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800e070 con 0x565234a809d0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5de3fff700 1 -- 172.21.15.153:0/2583424360 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8012410 con 0x565234a809d0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5de3fff700 1 -- 172.21.15.153:0/2583424360 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80127a0 con 0x565234a809d0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2583424360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 msgr2=0x565234a35c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a35c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5de80095a0 tx=0x7f5de80098b0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2583424360 shutdown_connections 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a35c80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b12c10 0x565234ba0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2583424360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b0e450 0x565234b9ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2583424360 >> 172.21.15.153:0/2583424360 conn(0x565234a62b60 msgr2=0x565234b0d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2583424360 shutdown_connections 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.216+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2583424360 wait complete. 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.217+0000 7f5dfbb06080 1 Processor -- start 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.217+0000 7f5dfbb06080 1 -- start start 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a63fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b0e450 0x565234b05210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b12c10 0x565234b047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565234b10950 con 0x565234a809d0 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x565234b0fd40 con 0x565234b0e450 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x565234b10020 con 0x565234b12c10 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a63fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a63fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40590/0 (socket says 172.21.15.153:40590) 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 -- 172.21.15.153:0/2943481919 learned_addr learned my addr 172.21.15.153:0/2943481919 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1408700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b0e450 0x565234b05210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df240a700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b12c10 0x565234b047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 -- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b12c10 msgr2=0x565234b047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565234b12c10 0x565234b047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 -- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b0e450 msgr2=0x565234b05210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565234b0e450 0x565234b05210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 -- 172.21.15.153:0/2943481919 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8005df0 con 0x565234a809d0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5df1c09700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a63fa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x565234a74680 tx=0x7f5de8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5de2ffd700 1 -- 172.21.15.153:0/2943481919 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8012c10 con 0x565234a809d0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5de2ffd700 1 -- 172.21.15.153:0/2943481919 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8012d70 con 0x565234a809d0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5652349a34f0 con 0x565234a809d0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5de2ffd700 1 -- 172.21.15.153:0/2943481919 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de801d6b0 con 0x565234a809d0 2023-11-28T13:01:40.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.218+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565234989120 con 0x565234a809d0 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.220+0000 7f5de2ffd700 1 -- 172.21.15.153:0/2943481919 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5de801d810 con 0x565234a809d0 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.220+0000 7f5de2ffd700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5dd4066c40 0x7f5dd40690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.220+0000 7f5de2ffd700 1 -- 172.21.15.153:0/2943481919 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5de8085b90 con 0x565234a809d0 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.220+0000 7f5df1408700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5dd4066c40 0x7f5dd40690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.220+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x565234a62650 0x565234b10de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.220+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 --> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x565234b11320 con 0x565234a62650 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.220+0000 7f5df240a700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x565234a62650 0x565234b10de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5df1408700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5dd4066c40 0x7f5dd40690f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5de4015040 tx=0x7f5de400f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5df240a700 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x565234a62650 0x565234b10de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5df240a700 1 -- 172.21.15.153:0/2943481919 <== osd.5 v2:172.21.15.177:6800/103770 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f5dec00ea40 con 0x565234a62650 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x565234a62650 msgr2=0x565234b10de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x565234a62650 0x565234b10de0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5dd4066c40 msgr2=0x7f5dd40690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5dd4066c40 0x7f5dd40690f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5de4015040 tx=0x7f5de400f000 comp rx=0 tx=0).stop 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 msgr2=0x565234a63fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 --2- 172.21.15.153:0/2943481919 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565234a809d0 0x565234a63fa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x565234a74680 tx=0x7f5de8003020 comp rx=0 tx=0).stop 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5df240a700 1 -- 172.21.15.153:0/2943481919 reap_dead start 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 shutdown_connections 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 >> 172.21.15.153:0/2943481919 conn(0x565234a62b60 msgr2=0x565234b0d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 shutdown_connections 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.221+0000 7f5dfbb06080 1 -- 172.21.15.153:0/2943481919 wait complete. 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi153.stdout:mytag 2023-11-28T13:01:40.228 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T13:01:40.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.313+0000 7fb7a012a080 1 Processor -- start 2023-11-28T13:01:40.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.313+0000 7fb7a012a080 1 -- start start 2023-11-28T13:01:40.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.314+0000 7fb7a012a080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7800030b0 0x55e331fbb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.314+0000 7fb7a012a080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 0x55e331eec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.314+0000 7fb7a012a080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e331f1c280 0x55e331eecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.314+0000 7fb7a012a080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e331e95000 con 0x7fb7800030b0 2023-11-28T13:01:40.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.314+0000 7fb7a012a080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e331e27fe0 con 0x55e331f2b700 2023-11-28T13:01:40.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.314+0000 7fb7a012a080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e331fbbf00 con 0x55e331f1c280 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.314+0000 7fb79622d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7800030b0 0x55e331fbb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7800030b0 0x55e331fbb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40612/0 (socket says 172.21.15.153:40612) 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 -- 172.21.15.153:0/4287920527 learned_addr learned my addr 172.21.15.153:0/4287920527 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb795a2c700 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 0x55e331eec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb796a2e700 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e331f1c280 0x55e331eecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 -- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e331f1c280 msgr2=0x55e331eecf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e331f1c280 0x55e331eecf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 -- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 msgr2=0x55e331eec9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 0x55e331eec9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 -- 172.21.15.153:0/4287920527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e331e40360 con 0x7fb7800030b0 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb796a2e700 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e331f1c280 0x55e331eecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:40.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb79622d700 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7800030b0 0x55e331fbb870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb78800d420 tx=0x7fb788001760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc0d61bdd4f8d6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb794a2a700 1 -- 172.21.15.153:0/4287920527 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78800ddf0 con 0x7fb7800030b0 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb794a2a700 1 -- 172.21.15.153:0/4287920527 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78800f450 con 0x7fb7800030b0 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb794a2a700 1 -- 172.21.15.153:0/4287920527 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78800f800 con 0x7fb7800030b0 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb7a012a080 1 -- 172.21.15.153:0/4287920527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7800030b0 msgr2=0x55e331fbb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb7a012a080 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7800030b0 0x55e331fbb870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb78800d420 tx=0x7fb788001760 comp rx=0 tx=0).stop 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb7a012a080 1 -- 172.21.15.153:0/4287920527 shutdown_connections 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.315+0000 7fb7a012a080 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7800030b0 0x55e331fbb870 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.316+0000 7fb7a012a080 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e331f1c280 0x55e331eecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.316+0000 7fb7a012a080 1 --2- 172.21.15.153:0/4287920527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 0x55e331eec9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.316+0000 7fb7a012a080 1 -- 172.21.15.153:0/4287920527 >> 172.21.15.153:0/4287920527 conn(0x55e331f19b60 msgr2=0x55e331fc4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.316+0000 7fb7a012a080 1 -- 172.21.15.153:0/4287920527 shutdown_connections 2023-11-28T13:01:40.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.316+0000 7fb7a012a080 1 -- 172.21.15.153:0/4287920527 wait complete. 2023-11-28T13:01:40.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.316+0000 7fb7a012a080 1 Processor -- start 2023-11-28T13:01:40.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.316+0000 7fb7a012a080 1 -- start start 2023-11-28T13:01:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb7a012a080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e331f1c280 0x55e331fba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb7a012a080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 0x55e33205b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb7a012a080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7800030b0 0x55e331fa78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb7a012a080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e331fa6820 con 0x55e331f1c280 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb7a012a080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e331fa6b00 con 0x55e331f2b700 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb7a012a080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e331fa6de0 con 0x7fb7800030b0 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e331f1c280 0x55e331fba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e331f1c280 0x55e331fba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40618/0 (socket says 172.21.15.153:40618) 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 -- 172.21.15.153:0/656462342 learned_addr learned my addr 172.21.15.153:0/656462342 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb795a2c700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 0x55e33205b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb796a2e700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7800030b0 0x55e331fa78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 -- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7800030b0 msgr2=0x55e331fa78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7800030b0 0x55e331fa78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 -- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 msgr2=0x55e33205b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e331f2b700 0x55e33205b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 -- 172.21.15.153:0/656462342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e331e40120 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb796a2e700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7800030b0 0x55e331fa78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.317+0000 7fb79622d700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e331f1c280 0x55e331fba3c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb78800dac0 tx=0x55e331f1aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.318+0000 7fb7877fe700 1 -- 172.21.15.153:0/656462342 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788019870 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.318+0000 7fb7877fe700 1 -- 172.21.15.153:0/656462342 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7880199d0 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.318+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e331f36ea0 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.318+0000 7fb7877fe700 1 -- 172.21.15.153:0/656462342 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788019d80 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.318+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e331e33830 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.319+0000 7fb7877fe700 1 -- 172.21.15.153:0/656462342 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb78801b450 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.319+0000 7fb7877fe700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb778066c10 0x7fb7780690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.319+0000 7fb7877fe700 1 -- 172.21.15.153:0/656462342 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb788086af0 con 0x55e331f1c280 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.319+0000 7fb795a2c700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb778066c10 0x7fb7780690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.319+0000 7fb7a012a080 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x55e331fa7e00 0x55e331f2dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.320+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 --> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x55e331fa62d0 con 0x55e331fa7e00 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.320+0000 7fb796a2e700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x55e331fa7e00 0x55e331f2dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.320+0000 7fb795a2c700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb778066c10 0x7fb7780690c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0099a0 tx=0x7fb78c011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.320+0000 7fb796a2e700 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x55e331fa7e00 0x55e331f2dd10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.320+0000 7fb796a2e700 1 -- 172.21.15.153:0/656462342 <== osd.5 v2:172.21.15.177:6800/103770 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fb790006020 con 0x55e331fa7e00 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x55e331fa7e00 msgr2=0x55e331f2dd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x55e331fa7e00 0x55e331f2dd10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb778066c10 msgr2=0x7fb7780690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb778066c10 0x7fb7780690c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0099a0 tx=0x7fb78c011000 comp rx=0 tx=0).stop 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e331f1c280 msgr2=0x55e331fba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 --2- 172.21.15.153:0/656462342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55e331f1c280 0x55e331fba3c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb78800dac0 tx=0x55e331f1aab0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb796a2e700 1 -- 172.21.15.153:0/656462342 reap_dead start 2023-11-28T13:01:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 shutdown_connections 2023-11-28T13:01:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 >> 172.21.15.153:0/656462342 conn(0x55e331f19b60 msgr2=0x55e331fc4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 shutdown_connections 2023-11-28T13:01:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.321+0000 7fb7a012a080 1 -- 172.21.15.153:0/656462342 wait complete. 2023-11-28T13:01:40.329 INFO:teuthology.orchestra.run.smithi153.stdout:mytag 2023-11-28T13:01:40.329 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T13:01:40.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 Processor -- start 2023-11-28T13:01:40.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 -- start start 2023-11-28T13:01:40.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 0x55ed0642f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed0642efe0 0x55ed063fc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed063a6000 con 0x55ed06444380 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed06338fe0 con 0x55ed0642efe0 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.372+0000 7fed29392080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed064459c0 con 0x55ed063fe520 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40638/0 (socket says 172.21.15.153:40638) 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 -- 172.21.15.153:0/2918887784 learned_addr learned my addr 172.21.15.153:0/2918887784 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1ec94700 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed0642efe0 0x55ed063fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1f495700 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 0x55ed0642f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 -- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 msgr2=0x55ed0642f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 0x55ed0642f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 -- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed0642efe0 msgr2=0x55ed063fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed0642efe0 0x55ed063fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 -- 172.21.15.153:0/2918887784 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed10005d20 con 0x55ed06444380 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1ec94700 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed0642efe0 0x55ed063fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1fc96700 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed06444380 0x55ed06429350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed18006f50 tx=0x7fed18001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff02d9142c05ba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1dc92700 1 -- 172.21.15.153:0/2918887784 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18012600 con 0x55ed06444380 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1dc92700 1 -- 172.21.15.153:0/2918887784 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed18012760 con 0x55ed06444380 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.373+0000 7fed1dc92700 1 -- 172.21.15.153:0/2918887784 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18012b10 con 0x55ed06444380 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 -- 172.21.15.153:0/2918887784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed06444380 msgr2=0x55ed06429350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed06444380 0x55ed06429350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed18006f50 tx=0x7fed18001820 comp rx=0 tx=0).stop 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 -- 172.21.15.153:0/2918887784 shutdown_connections 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 0x55ed0642f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 --2- 172.21.15.153:0/2918887784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed0642efe0 0x55ed063fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 -- 172.21.15.153:0/2918887784 >> 172.21.15.153:0/2918887784 conn(0x55ed0642b3e0 msgr2=0x55ed064d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 -- 172.21.15.153:0/2918887784 shutdown_connections 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 -- 172.21.15.153:0/2918887784 wait complete. 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 Processor -- start 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.374+0000 7fed29392080 1 -- start start 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed29392080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 0x55ed064ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed29392080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0642efe0 0x55ed06441a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed29392080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed29392080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed064d1b70 con 0x55ed0642efe0 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed29392080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed064cd310 con 0x55ed06444380 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed29392080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed064cc940 con 0x55ed063fe520 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0642efe0 0x55ed06441a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0642efe0 0x55ed06441a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40640/0 (socket says 172.21.15.153:40640) 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 -- 172.21.15.153:0/857934330 learned_addr learned my addr 172.21.15.153:0/857934330 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1f495700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 0x55ed064ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1fc96700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 -- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 msgr2=0x55ed064ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ed063fe520 0x55ed064ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 -- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed06444380 msgr2=0x55ed06429350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 -- 172.21.15.153:0/857934330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0636b4f0 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1fc96700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed06444380 0x55ed06429350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.375+0000 7fed1ec94700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0642efe0 0x55ed06441a30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed10013000 tx=0x7fed10003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.376+0000 7fed1cc90700 1 -- 172.21.15.153:0/857934330 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1001c630 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.376+0000 7fed1cc90700 1 -- 172.21.15.153:0/857934330 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1001c790 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.376+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ed06344830 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.376+0000 7fed1cc90700 1 -- 172.21.15.153:0/857934330 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1001cb20 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.376+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed06351360 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.377+0000 7fed1cc90700 1 -- 172.21.15.153:0/857934330 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed1001cc80 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.377+0000 7fed1cc90700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fed0c066c10 0x7fed0c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.377+0000 7fed1cc90700 1 -- 172.21.15.153:0/857934330 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed10085760 con 0x55ed0642efe0 2023-11-28T13:01:40.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.377+0000 7fed1f495700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fed0c066c10 0x7fed0c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.377+0000 7fed29392080 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x55ed06441f70 0x55ed064cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.377+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 --> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) v8 -- 0x55ed064c5c50 con 0x55ed06441f70 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed1f495700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fed0c066c10 0x7fed0c0690c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55ed064cfd30 tx=0x7fed140016e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed1fc96700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x55ed06441f70 0x55ed064cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed1fc96700 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x55ed06441f70 0x55ed064cb320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed1fc96700 1 -- 172.21.15.153:0/857934330 <== osd.7 v2:172.21.15.177:6808/103771 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fed180128d0 con 0x55ed06441f70 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x55ed06441f70 msgr2=0x55ed064cb320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed29392080 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x55ed06441f70 0x55ed064cb320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fed0c066c10 msgr2=0x7fed0c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed29392080 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fed0c066c10 0x7fed0c0690c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55ed064cfd30 tx=0x7fed140016e0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0642efe0 msgr2=0x55ed06441a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.378+0000 7fed29392080 1 --2- 172.21.15.153:0/857934330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55ed0642efe0 0x55ed06441a30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed10013000 tx=0x7fed10003020 comp rx=0 tx=0).stop 2023-11-28T13:01:40.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.379+0000 7fed1fc96700 1 -- 172.21.15.153:0/857934330 reap_dead start 2023-11-28T13:01:40.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.379+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 shutdown_connections 2023-11-28T13:01:40.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.379+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 >> 172.21.15.153:0/857934330 conn(0x55ed0642b3e0 msgr2=0x55ed063fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.379+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 shutdown_connections 2023-11-28T13:01:40.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.379+0000 7fed29392080 1 -- 172.21.15.153:0/857934330 wait complete. 2023-11-28T13:01:40.388 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:01:40.388 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:01:40.467 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:01:40.511 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:01:40.511 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:01:40.511 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:40.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:01:40.529 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:01:40.529 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:40.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:01:40.572 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:01:40.572 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:40.589 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:01:40.589 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:01:40.589 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:40.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:01:40.630 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:40.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.749+0000 7f8bf7849700 1 Processor -- start 2023-11-28T13:01:40.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.750+0000 7f8bf7849700 1 -- start start 2023-11-28T13:01:40.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.750+0000 7f8bf7849700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.750+0000 7f8bf7849700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8bf00b2930 0x7f8bf00b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf7849700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf006b420 con 0x7f8bf00d0a00 2023-11-28T13:01:40.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf7849700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf00b32f0 con 0x7f8bf00b2930 2023-11-28T13:01:40.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf7849700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf006ff60 con 0x7f8bf00ce040 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8bf00b2930 0x7f8bf00b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8bf00b2930 0x7f8bf00b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40654/0 (socket says 172.21.15.153:40654) 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 -- 172.21.15.153:0/2738177364 learned_addr learned my addr 172.21.15.153:0/2738177364 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf55e5700 1 --2- 172.21.15.153:0/2738177364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 -- 172.21.15.153:0/2738177364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 msgr2=0x7f8bf015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 --2- 172.21.15.153:0/2738177364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 -- 172.21.15.153:0/2738177364 >> v1:172.21.15.177:6789/0 conn(0x7f8bf00d0a00 legacy=0x7f8bf006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 -- 172.21.15.153:0/2738177364 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf006b700 con 0x7f8bf00b2930 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf55e5700 1 --2- 172.21.15.153:0/2738177364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8bf5de6700 1 --2- 172.21.15.153:0/2738177364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00b2db0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009e50 tx=0x7f8be800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04e360e3c483034 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8be7fff700 1 -- 172.21.15.153:0/2738177364 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800e070 con 0x7f8bf00b2930 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8be7fff700 1 -- 172.21.15.153:0/2738177364 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800b9c0 con 0x7f8bf00b2930 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.751+0000 7f8be7fff700 1 -- 172.21.15.153:0/2738177364 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800bd70 con 0x7f8bf00b2930 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 -- 172.21.15.153:0/2738177364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 msgr2=0x7f8bf00b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2738177364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00b2db0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009e50 tx=0x7f8be800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 -- 172.21.15.153:0/2738177364 shutdown_connections 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2738177364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00b2db0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2738177364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 -- 172.21.15.153:0/2738177364 >> 172.21.15.153:0/2738177364 conn(0x7f8bf006aed0 msgr2=0x7f8bf006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 -- 172.21.15.153:0/2738177364 shutdown_connections 2023-11-28T13:01:40.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.752+0000 7f8bf7849700 1 -- 172.21.15.153:0/2738177364 wait complete. 2023-11-28T13:01:40.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.753+0000 7f8bf7849700 1 Processor -- start 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.753+0000 7f8bf7849700 1 -- start start 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf7849700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf7849700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf7849700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bf00d0a00 0x7f8bf01f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf7849700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00c8fd0 con 0x7f8bf00b2930 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf7849700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00b63a0 con 0x7f8bf00d0a00 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf7849700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00b6640 con 0x7f8bf00ce040 2023-11-28T13:01:40.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40656/0 (socket says 172.21.15.153:40656) 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 -- 172.21.15.153:0/2283053262 learned_addr learned my addr 172.21.15.153:0/2283053262 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf5de6700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bf00d0a00 0x7f8bf01f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf4de4700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 -- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 msgr2=0x7f8bf00b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf00b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 -- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bf00d0a00 msgr2=0x7f8bf01f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bf00d0a00 0x7f8bf01f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf55e5700 1 -- 172.21.15.153:0/2283053262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8009b30 con 0x7f8bf00b2930 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.754+0000 7f8bf4de4700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf00b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:40.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.755+0000 7f8bf55e5700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00c7290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8be000e810 tx=0x7f8be000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.755+0000 7f8be5ffb700 1 -- 172.21.15.153:0/2283053262 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0005c20 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.755+0000 7f8be5ffb700 1 -- 172.21.15.153:0/2283053262 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0005d80 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.755+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf006bcc0 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.755+0000 7f8be5ffb700 1 -- 172.21.15.153:0/2283053262 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0006270 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.755+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf006b9e0 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.756+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf006bfa0 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.756+0000 7f8be5ffb700 1 -- 172.21.15.153:0/2283053262 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8be0022070 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.756+0000 7f8be5ffb700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8bd40682f0 0x7f8bd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.756+0000 7f8be5ffb700 1 -- 172.21.15.153:0/2283053262 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8be00856e0 con 0x7f8bf00b2930 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.756+0000 7f8bf4de4700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8bd40682f0 0x7f8bd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.757+0000 7f8bf4de4700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8bd40682f0 0x7f8bd406a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00b38b0 tx=0x7f8bec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.763 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.761+0000 7f8be5ffb700 1 -- 172.21.15.153:0/2283053262 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8be0051f40 con 0x7f8bf00b2930 2023-11-28T13:01:40.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.983+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bf00c92b0 con 0x7f8bf00b2930 2023-11-28T13:01:40.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.984+0000 7f8be5ffb700 1 -- 172.21.15.153:0/2283053262 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f8be00094d0 con 0x7f8bf00b2930 2023-11-28T13:01:40.985 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:40.985 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.985+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8bd40682f0 msgr2=0x7f8bd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.985+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8bd40682f0 0x7f8bd406a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00b38b0 tx=0x7f8bec009040 comp rx=0 tx=0).stop 2023-11-28T13:01:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.985+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 msgr2=0x7f8bf00c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.985+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00c7290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8be000e810 tx=0x7f8be000ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 shutdown_connections 2023-11-28T13:01:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8bd40682f0 0x7f8bd406a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bf00b2930 0x7f8bf00c7290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bf00d0a00 0x7f8bf01f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 --2- 172.21.15.153:0/2283053262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bf00ce040 0x7f8bf00b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 >> 172.21.15.153:0/2283053262 conn(0x7f8bf006aed0 msgr2=0x7f8bf00d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 shutdown_connections 2023-11-28T13:01:40.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:40.986+0000 7f8bf7849700 1 -- 172.21.15.153:0/2283053262 wait complete. 2023-11-28T13:01:40.989 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:40.999 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:41.120 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.119+0000 7f32b5547700 1 Processor -- start 2023-11-28T13:01:41.120 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.119+0000 7f32b5547700 1 -- start start 2023-11-28T13:01:41.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.119+0000 7f32b5547700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b015ca50 0x7f32b006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.119+0000 7f32b5547700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f32b006b8c0 0x7f32b00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.119+0000 7f32b5547700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b00cf5e0 con 0x7f32b00ccc40 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.119+0000 7f32b5547700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b00cd470 con 0x7f32b006b8c0 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32b5547700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b006bd30 con 0x7f32b015ca50 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b015ca50 0x7f32b006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f32b006b8c0 0x7f32b00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f32b006b8c0 0x7f32b00cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:50132/0 (socket says 172.21.15.153:50132) 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 -- 172.21.15.153:0/4115333452 learned_addr learned my addr 172.21.15.153:0/4115333452 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:41.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32af7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f32b00ccc40 0x7f32b00cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:35446/0 (socket says 172.21.15.153:35446) 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 -- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b015ca50 msgr2=0x7f32b006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 --2- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b015ca50 0x7f32b006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 -- 172.21.15.153:0/4115333452 >> v1:172.21.15.177:6789/0 conn(0x7f32b00ccc40 legacy=0x7f32b00cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 -- 172.21.15.153:0/4115333452 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00cecb0 con 0x7f32b006b8c0 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32aeffd700 1 --2- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b015ca50 0x7f32b006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.120+0000 7f32ae7fc700 1 --2- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b006b8c0 0x7f32b00cc700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009920 tx=0x7f32a4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfceff6e1e8ed781 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32adffb700 1 -- 172.21.15.153:0/4115333452 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4011830 con 0x7f32b006b8c0 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32adffb700 1 -- 172.21.15.153:0/4115333452 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a4011990 con 0x7f32b006b8c0 2023-11-28T13:01:41.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32adffb700 1 -- 172.21.15.153:0/4115333452 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4011d20 con 0x7f32b006b8c0 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 -- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b006b8c0 msgr2=0x7f32b00cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 --2- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b006b8c0 0x7f32b00cc700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009920 tx=0x7f32a4009c30 comp rx=0 tx=0).stop 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 -- 172.21.15.153:0/4115333452 shutdown_connections 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 --2- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b015ca50 0x7f32b006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 --2- 172.21.15.153:0/4115333452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b006b8c0 0x7f32b00cc700 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 -- 172.21.15.153:0/4115333452 >> 172.21.15.153:0/4115333452 conn(0x7f32b009a230 msgr2=0x7f32b008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 -- 172.21.15.153:0/4115333452 shutdown_connections 2023-11-28T13:01:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.121+0000 7f32b5547700 1 -- 172.21.15.153:0/4115333452 wait complete. 2023-11-28T13:01:41.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.122+0000 7f32b5547700 1 Processor -- start 2023-11-28T13:01:41.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32b5547700 1 -- start start 2023-11-28T13:01:41.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32b5547700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b006b8c0 0x7f32b00b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32b5547700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b00ccc40 0x7f32b00b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32b5547700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b015ca50 0x7f32b01f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32b5547700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b015e080 con 0x7f32b006b8c0 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32b5547700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01f5b50 con 0x7f32b015ca50 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32b5547700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00b4c90 con 0x7f32b00ccc40 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b006b8c0 0x7f32b00b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b006b8c0 0x7f32b00b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40670/0 (socket says 172.21.15.153:40670) 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.123+0000 7f32aeffd700 1 -- 172.21.15.153:0/1870875832 learned_addr learned my addr 172.21.15.153:0/1870875832 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32af7fe700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b015ca50 0x7f32b01f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32ae7fc700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b00ccc40 0x7f32b00b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32aeffd700 1 -- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b00ccc40 msgr2=0x7f32b00b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32aeffd700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b00ccc40 0x7f32b00b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32aeffd700 1 -- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b015ca50 msgr2=0x7f32b01f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32aeffd700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b015ca50 0x7f32b01f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32aeffd700 1 -- 172.21.15.153:0/1870875832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00094a0 con 0x7f32b006b8c0 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32af7fe700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b015ca50 0x7f32b01f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32aeffd700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b006b8c0 0x7f32b00b4750 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f32a000ec10 tx=0x7f32a0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32977fe700 1 -- 172.21.15.153:0/1870875832 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000c8e0 con 0x7f32b006b8c0 2023-11-28T13:01:41.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32977fe700 1 -- 172.21.15.153:0/1870875832 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a000ca40 con 0x7f32b006b8c0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a4009550 con 0x7f32b006b8c0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32977fe700 1 -- 172.21.15.153:0/1870875832 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000cdd0 con 0x7f32b006b8c0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.124+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b00cf270 con 0x7f32b006b8c0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.125+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b00cef90 con 0x7f32b006b8c0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.125+0000 7f32977fe700 1 -- 172.21.15.153:0/1870875832 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32a0019030 con 0x7f32b006b8c0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.126+0000 7f32977fe700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f329c0682f0 0x7f329c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.126+0000 7f32977fe700 1 -- 172.21.15.153:0/1870875832 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32a0085c90 con 0x7f32b006b8c0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.126+0000 7f32ae7fc700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f329c0682f0 0x7f329c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.126+0000 7f32ae7fc700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f329c0682f0 0x7f329c06a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f32a4000c00 tx=0x7f32a4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.130+0000 7f32977fe700 1 -- 172.21.15.153:0/1870875832 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32a00524f0 con 0x7f32b006b8c0 2023-11-28T13:01:41.316 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.316+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32b00cd0c0 con 0x7f32b006b8c0 2023-11-28T13:01:41.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.317+0000 7f32977fe700 1 -- 172.21.15.153:0/1870875832 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f32a0055b10 con 0x7f32b006b8c0 2023-11-28T13:01:41.318 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:41.318 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":39,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:14.078173+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:11.364224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:12.036978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:41.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f329c0682f0 msgr2=0x7f329c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f329c0682f0 0x7f329c06a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f32a4000c00 tx=0x7f32a4008030 comp rx=0 tx=0).stop 2023-11-28T13:01:41.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b006b8c0 msgr2=0x7f32b00b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b006b8c0 0x7f32b00b4750 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f32a000ec10 tx=0x7f32a0009e20 comp rx=0 tx=0).stop 2023-11-28T13:01:41.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 shutdown_connections 2023-11-28T13:01:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f329c0682f0 0x7f329c06a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32b006b8c0 0x7f32b00b4750 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b015ca50 0x7f32b01f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 --2- 172.21.15.153:0/1870875832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b00ccc40 0x7f32b00b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 >> 172.21.15.153:0/1870875832 conn(0x7f32b009a230 msgr2=0x7f32b00cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 shutdown_connections 2023-11-28T13:01:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.319+0000 7f32b5547700 1 -- 172.21.15.153:0/1870875832 wait complete. 2023-11-28T13:01:41.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:01:11.364224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:01:12.036978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:01:41.333 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:41.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.456+0000 7fe6665c6700 1 Processor -- start 2023-11-28T13:01:41.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.456+0000 7fe6665c6700 1 -- start start 2023-11-28T13:01:41.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe6665c6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe66015cef0 0x7fe6600af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe6665c6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe66006ba90 0x7fe6600ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe6665c6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6600cc5f0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe6665c6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66015d470 con 0x7fe66015cef0 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe6665c6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66015d750 con 0x7fe66006ba90 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65ffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe66015cef0 0x7fe6600af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe6600cbdf0 0x7fe66006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58842/0 (socket says 172.21.15.153:58842) 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65f7fe700 1 -- 172.21.15.153:0/1659563493 learned_addr learned my addr 172.21.15.153:0/1659563493 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe664b63700 1 --2- 172.21.15.153:0/1659563493 >> v2:172.21.15.177:3300/0 conn(0x7fe66006ba90 0x7fe6600ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65ffff700 1 -- 172.21.15.153:0/1659563493 >> v2:172.21.15.177:3300/0 conn(0x7fe66006ba90 msgr2=0x7fe6600ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:41.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65ffff700 1 --2- 172.21.15.153:0/1659563493 >> v2:172.21.15.177:3300/0 conn(0x7fe66006ba90 0x7fe6600ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65ffff700 1 -- 172.21.15.153:0/1659563493 >> v1:172.21.15.153:6789/0 conn(0x7fe6600cbdf0 legacy=0x7fe66006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65effd700 1 -- 172.21.15.153:0/1659563493 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221334031 0 0) 0x7fe6600cc5f0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.457+0000 7fe65ffff700 1 -- 172.21.15.153:0/1659563493 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66015da30 con 0x7fe66015cef0 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe65ffff700 1 --2- 172.21.15.153:0/1659563493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe66015cef0 0x7fe6600af400 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe654009840 tx=0x7fe654009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152859c04397ca80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe65effd700 1 -- 172.21.15.153:0/1659563493 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654011830 con 0x7fe66015cef0 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe65effd700 1 -- 172.21.15.153:0/1659563493 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe654011990 con 0x7fe66015cef0 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe65effd700 1 -- 172.21.15.153:0/1659563493 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654011d20 con 0x7fe66015cef0 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 -- 172.21.15.153:0/1659563493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe66015cef0 msgr2=0x7fe6600af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 --2- 172.21.15.153:0/1659563493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe66015cef0 0x7fe6600af400 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe654009840 tx=0x7fe654009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 -- 172.21.15.153:0/1659563493 shutdown_connections 2023-11-28T13:01:41.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 --2- 172.21.15.153:0/1659563493 >> v2:172.21.15.177:3300/0 conn(0x7fe66006ba90 0x7fe6600ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 --2- 172.21.15.153:0/1659563493 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe66015cef0 0x7fe6600af400 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 -- 172.21.15.153:0/1659563493 >> 172.21.15.153:0/1659563493 conn(0x7fe6600d0000 msgr2=0x7fe6600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 -- 172.21.15.153:0/1659563493 shutdown_connections 2023-11-28T13:01:41.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.458+0000 7fe6665c6700 1 -- 172.21.15.153:0/1659563493 wait complete. 2023-11-28T13:01:41.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.459+0000 7fe6665c6700 1 Processor -- start 2023-11-28T13:01:41.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.460+0000 7fe6665c6700 1 -- start start 2023-11-28T13:01:41.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.460+0000 7fe6665c6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe66006ba90 0x7fe6600b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.460+0000 7fe6665c6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6600cbdf0 0x7fe6601593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.460+0000 7fe6665c6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe66015cef0 0x7fe660157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.460+0000 7fe6665c6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660159930 con 0x7fe6600cbdf0 2023-11-28T13:01:41.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.460+0000 7fe6665c6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660157dc0 con 0x7fe66015cef0 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.460+0000 7fe6665c6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600b34f0 con 0x7fe66006ba90 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6600cbdf0 0x7fe6601593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6600cbdf0 0x7fe6601593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40684/0 (socket says 172.21.15.153:40684) 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 -- 172.21.15.153:0/3359875193 learned_addr learned my addr 172.21.15.153:0/3359875193 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe664b63700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe66015cef0 0x7fe660157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65ffff700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe66006ba90 0x7fe6600b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 -- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe66006ba90 msgr2=0x7fe6600b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe66006ba90 0x7fe6600b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 -- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe66015cef0 msgr2=0x7fe660157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe66015cef0 0x7fe660157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 -- 172.21.15.153:0/3359875193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540094a0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65f7fe700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6600cbdf0 0x7fe6601593f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe6500011e0 tx=0x7fe65000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65cff9700 1 -- 172.21.15.153:0/3359875193 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650004c80 con 0x7fe6600cbdf0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65cff9700 1 -- 172.21.15.153:0/3359875193 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650004de0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe65cff9700 1 -- 172.21.15.153:0/3359875193 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650007310 con 0x7fe6600cbdf0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66015dff0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.461+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66015dd10 con 0x7fe6600cbdf0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.462+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6600cab90 con 0x7fe6600cbdf0 2023-11-28T13:01:41.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.463+0000 7fe65cff9700 1 -- 172.21.15.153:0/3359875193 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe65001a020 con 0x7fe6600cbdf0 2023-11-28T13:01:41.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.463+0000 7fe65cff9700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe64c0682f0 0x7fe64c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.463+0000 7fe65cff9700 1 -- 172.21.15.153:0/3359875193 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe650085840 con 0x7fe6600cbdf0 2023-11-28T13:01:41.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.463+0000 7fe65ffff700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe64c0682f0 0x7fe64c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.464+0000 7fe65ffff700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe64c0682f0 0x7fe64c06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe654000c00 tx=0x7fe65401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.468+0000 7fe65cff9700 1 -- 172.21.15.153:0/3359875193 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe6500520a0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.652+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe66015aad0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.652+0000 7fe65cff9700 1 -- 172.21.15.153:0/3359875193 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6500556c0 con 0x7fe6600cbdf0 2023-11-28T13:01:41.653 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:01:41.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.654+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe64c0682f0 msgr2=0x7fe64c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.654+0000 7fe6665c6700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe64c0682f0 0x7fe64c06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe654000c00 tx=0x7fe65401c040 comp rx=0 tx=0).stop 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.654+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6600cbdf0 msgr2=0x7fe6601593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.654+0000 7fe6665c6700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6600cbdf0 0x7fe6601593f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe6500011e0 tx=0x7fe65000ca30 comp rx=0 tx=0).stop 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.654+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 shutdown_connections 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.654+0000 7fe6665c6700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe66015cef0 0x7fe660157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.655+0000 7fe6665c6700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe66006ba90 0x7fe6600b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.655+0000 7fe6665c6700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe64c0682f0 0x7fe64c06a7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.655+0000 7fe6665c6700 1 --2- 172.21.15.153:0/3359875193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6600cbdf0 0x7fe6601593f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.655+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 >> 172.21.15.153:0/3359875193 conn(0x7fe6600d0000 msgr2=0x7fe660073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.655+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 shutdown_connections 2023-11-28T13:01:41.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.655+0000 7fe6665c6700 1 -- 172.21.15.153:0/3359875193 wait complete. 2023-11-28T13:01:41.667 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:41.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.790+0000 7f6af8017700 1 Processor -- start 2023-11-28T13:01:41.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.790+0000 7f6af8017700 1 -- start start 2023-11-28T13:01:41.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af8017700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6af015cef0 0x7f6af00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af8017700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af00cbdf0 0x7f6af006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af8017700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6af006b8c0 0x7f6af0071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af8017700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af006bd40 con 0x7f6af006b8c0 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af8017700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af00cc5f0 con 0x7f6af015cef0 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af8017700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af015d470 con 0x7f6af00cbdf0 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6af006b8c0 0x7f6af0071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6af006b8c0 0x7f6af0071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40692/0 (socket says 172.21.15.153:40692) 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 -- 172.21.15.153:0/819280890 learned_addr learned my addr 172.21.15.153:0/819280890 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af5db3700 1 --2- 172.21.15.153:0/819280890 >> v2:172.21.15.177:3300/0 conn(0x7f6af015cef0 0x7f6af00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af55b2700 1 --2- 172.21.15.153:0/819280890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af00cbdf0 0x7f6af006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 -- 172.21.15.153:0/819280890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af00cbdf0 msgr2=0x7f6af006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 --2- 172.21.15.153:0/819280890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af00cbdf0 0x7f6af006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 -- 172.21.15.153:0/819280890 >> v2:172.21.15.177:3300/0 conn(0x7f6af015cef0 msgr2=0x7f6af00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 --2- 172.21.15.153:0/819280890 >> v2:172.21.15.177:3300/0 conn(0x7f6af015cef0 0x7f6af00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 -- 172.21.15.153:0/819280890 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af015d750 con 0x7f6af006b8c0 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.791+0000 7f6af65b4700 1 --2- 172.21.15.153:0/819280890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0071040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6af00723f0 tx=0x7f6ae800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f106870003033f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af4db1700 1 -- 172.21.15.153:0/819280890 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800e070 con 0x7f6af006b8c0 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af4db1700 1 -- 172.21.15.153:0/819280890 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae800b910 con 0x7f6af006b8c0 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af4db1700 1 -- 172.21.15.153:0/819280890 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800bcc0 con 0x7f6af006b8c0 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 -- 172.21.15.153:0/819280890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 msgr2=0x7f6af0071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 --2- 172.21.15.153:0/819280890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0071040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6af00723f0 tx=0x7f6ae800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 -- 172.21.15.153:0/819280890 shutdown_connections 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 --2- 172.21.15.153:0/819280890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0071040 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 --2- 172.21.15.153:0/819280890 >> v2:172.21.15.177:3300/0 conn(0x7f6af015cef0 0x7f6af00af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 --2- 172.21.15.153:0/819280890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af00cbdf0 0x7f6af006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 -- 172.21.15.153:0/819280890 >> 172.21.15.153:0/819280890 conn(0x7f6af00d0000 msgr2=0x7f6af00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 -- 172.21.15.153:0/819280890 shutdown_connections 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.792+0000 7f6af8017700 1 -- 172.21.15.153:0/819280890 wait complete. 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.793+0000 7f6af8017700 1 Processor -- start 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.793+0000 7f6af8017700 1 -- start start 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af8017700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af8017700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6af00cbdf0 0x7f6af00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af8017700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af015cef0 0x7f6af00c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af8017700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af0154990 con 0x7f6af006b8c0 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af8017700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00b45b0 con 0x7f6af00cbdf0 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af8017700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af009aaf0 con 0x7f6af015cef0 2023-11-28T13:01:41.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40696/0 (socket says 172.21.15.153:40696) 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 -- 172.21.15.153:0/645598770 learned_addr learned my addr 172.21.15.153:0/645598770 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af55b2700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6af00cbdf0 0x7f6af00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af65b4700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af015cef0 0x7f6af00c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 -- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af015cef0 msgr2=0x7f6af00c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af015cef0 0x7f6af00c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 -- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6af00cbdf0 msgr2=0x7f6af00c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6af00cbdf0 0x7f6af00c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af5db3700 1 -- 172.21.15.153:0/645598770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8009b30 con 0x7f6af006b8c0 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af65b4700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af015cef0 0x7f6af00c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.794+0000 7f6af55b2700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6af00cbdf0 0x7f6af00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.795+0000 7f6af5db3700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0152f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9b0 tx=0x7f6aec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.795+0000 7f6ae67fc700 1 -- 172.21.15.153:0/645598770 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec009710 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.795+0000 7f6ae67fc700 1 -- 172.21.15.153:0/645598770 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec00c850 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.795+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af015dd10 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.795+0000 7f6ae67fc700 1 -- 172.21.15.153:0/645598770 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00cbc0 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.795+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af015da30 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.796+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af015dff0 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.796+0000 7f6ae67fc700 1 -- 172.21.15.153:0/645598770 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6aec019030 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.796+0000 7f6ae67fc700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6ad40682f0 0x7f6ad406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.797+0000 7f6ae67fc700 1 -- 172.21.15.153:0/645598770 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6aec085ca0 con 0x7f6af006b8c0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.797+0000 7f6af55b2700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6ad40682f0 0x7f6ad406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.797+0000 7f6af55b2700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6ad40682f0 0x7f6ad406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6af00b5350 tx=0x7f6ae0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.801+0000 7f6ae67fc700 1 -- 172.21.15.153:0/645598770 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6aec08a070 con 0x7f6af006b8c0 2023-11-28T13:01:41.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.982+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6af00d0e60 con 0x7f6af006b8c0 2023-11-28T13:01:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.983+0000 7f6ae67fc700 1 -- 172.21.15.153:0/645598770 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6aec0210a0 con 0x7f6af006b8c0 2023-11-28T13:01:41.984 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:01:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6ad40682f0 msgr2=0x7f6ad406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6ad40682f0 0x7f6ad406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6af00b5350 tx=0x7f6ae0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 msgr2=0x7f6af0152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0152f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00e9b0 tx=0x7f6aec00ed70 comp rx=0 tx=0).stop 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 shutdown_connections 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6ad40682f0 0x7f6ad406a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6af006b8c0 0x7f6af0152f40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af015cef0 0x7f6af00c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 --2- 172.21.15.153:0/645598770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6af00cbdf0 0x7f6af00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 >> 172.21.15.153:0/645598770 conn(0x7f6af00d0000 msgr2=0x7f6af0155730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 shutdown_connections 2023-11-28T13:01:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:41.985+0000 7f6af8017700 1 -- 172.21.15.153:0/645598770 wait complete. 2023-11-28T13:01:41.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:42.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.121+0000 7f9f0ac0b700 1 Processor -- start 2023-11-28T13:01:42.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.121+0000 7f9f0ac0b700 1 -- start start 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f0ac0b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f0406ba90 0x7f9f040af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f0ac0b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f040cc5f0 con 0x7f9f040cbdf0 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f0ac0b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0415d470 con 0x7f9f0415cef0 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f0ac0b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0415d750 con 0x7f9f0406ba90 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f091a8700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9f040cbdf0 0x7f9f040ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58872/0 (socket says 172.21.15.153:58872) 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f091a8700 1 -- 172.21.15.153:0/1613284866 learned_addr learned my addr 172.21.15.153:0/1613284866 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842064738 0 0) 0x7f9f040cc5f0 con 0x7f9f040cbdf0 2023-11-28T13:01:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f03fff700 1 --2- 172.21.15.153:0/1613284866 >> v2:172.21.15.177:3300/0 conn(0x7f9f0406ba90 0x7f9f040af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef40035b0 con 0x7f9f040cbdf0 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2837206978 0 0) 0x7f9ef40035b0 con 0x7f9f040cbdf0 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 >> v2:172.21.15.177:3300/0 conn(0x7f9f0406ba90 msgr2=0x7f9f040af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 --2- 172.21.15.153:0/1613284866 >> v2:172.21.15.177:3300/0 conn(0x7f9f0406ba90 0x7f9f040af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 >> v1:172.21.15.177:6789/0 conn(0x7f9f0415cef0 legacy=0x7f9f0406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0415da30 con 0x7f9f040cbdf0 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f9efc0019f0 con 0x7f9f040cbdf0 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 >> v1:172.21.15.153:6789/0 conn(0x7f9f040cbdf0 legacy=0x7f9f040ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ef4003a00 0x7f9ef4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.122+0000 7f9f037fe700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9ef4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f037fe700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ef4005200 0x7f9ef400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0415d750 con 0x7f9ef4004850 2023-11-28T13:01:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef400c080 con 0x7f9ef4005200 2023-11-28T13:01:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef400c240 con 0x7f9ef4003a00 2023-11-28T13:01:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f03fff700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9ef4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f089a7700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ef4003a00 0x7f9ef4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f091a8700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ef4005200 0x7f9ef400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f03fff700 1 -- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ef4003a00 msgr2=0x7f9ef4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f03fff700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ef4003a00 0x7f9ef4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f03fff700 1 -- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ef4005200 msgr2=0x7f9ef400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f03fff700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ef4005200 0x7f9ef400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f091a8700 1 -- 172.21.15.153:0/1613284866 reap_dead start 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f03fff700 1 -- 172.21.15.153:0/1613284866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40035b0 con 0x7f9ef4004850 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f03fff700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9ef4004cc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009b60 tx=0x7f9ef4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b259a4116e2799 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000e6c0 con 0x7f9ef4004850 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef000e820 con 0x7f9ef4004850 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.123+0000 7f9f037fe700 1 -- 172.21.15.153:0/1613284866 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000eb90 con 0x7f9ef4004850 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.124+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/1613284866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 msgr2=0x7f9ef4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.124+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9ef4004cc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009b60 tx=0x7f9ef4005880 comp rx=0 tx=0).stop 2023-11-28T13:01:42.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.124+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/1613284866 shutdown_connections 2023-11-28T13:01:42.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.124+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/1613284866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9ef4004cc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.124+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/1613284866 >> 172.21.15.153:0/1613284866 conn(0x7f9f040d0000 msgr2=0x7f9f040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.124+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/1613284866 shutdown_connections 2023-11-28T13:01:42.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.124+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/1613284866 wait complete. 2023-11-28T13:01:42.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.125+0000 7f9f0ac0b700 1 Processor -- start 2023-11-28T13:01:42.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.125+0000 7f9f0ac0b700 1 -- start start 2023-11-28T13:01:42.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f0ac0b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9f04150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f0ac0b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f04150950 0x7f9f041e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f0ac0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f041e5140 0x7f9f0409b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f0ac0b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0409b1b0 con 0x7f9ef4004850 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f0ac0b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f041e5680 con 0x7f9f04150950 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f0ac0b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f041e5960 con 0x7f9f041e5140 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9f04150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9f04150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40716/0 (socket says 172.21.15.153:40716) 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 -- 172.21.15.153:0/2599949185 learned_addr learned my addr 172.21.15.153:0/2599949185 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f03fff700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f04150950 0x7f9f041e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f091a8700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f041e5140 0x7f9f0409b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 -- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f041e5140 msgr2=0x7f9f0409b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f041e5140 0x7f9f0409b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 -- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f04150950 msgr2=0x7f9f041e4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f04150950 0x7f9f041e4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 -- 172.21.15.153:0/2599949185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0415dff0 con 0x7f9ef4004850 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f089a7700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9f04150410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800f7f0 tx=0x7f9ef800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.126+0000 7f9f017fa700 1 -- 172.21.15.153:0/2599949185 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80052e0 con 0x7f9ef4004850 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.127+0000 7f9f017fa700 1 -- 172.21.15.153:0/2599949185 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8005440 con 0x7f9ef4004850 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.127+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0415dd10 con 0x7f9ef4004850 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.127+0000 7f9f017fa700 1 -- 172.21.15.153:0/2599949185 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80038c0 con 0x7f9ef4004850 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.127+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f040af400 con 0x7f9ef4004850 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.127+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0409af80 con 0x7f9ef4004850 2023-11-28T13:01:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.128+0000 7f9f017fa700 1 -- 172.21.15.153:0/2599949185 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ef8003a20 con 0x7f9ef4004850 2023-11-28T13:01:42.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.129+0000 7f9f017fa700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ee80682f0 0x7f9ee806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.129+0000 7f9f017fa700 1 -- 172.21.15.153:0/2599949185 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ef80866e0 con 0x7f9ef4004850 2023-11-28T13:01:42.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.129+0000 7f9f03fff700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ee80682f0 0x7f9ee806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.129+0000 7f9f03fff700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ee80682f0 0x7f9ee806a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f040cbab0 tx=0x7f9ef0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.132+0000 7f9f017fa700 1 -- 172.21.15.153:0/2599949185 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ef8052f50 con 0x7f9ef4004850 2023-11-28T13:01:42.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.313+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f040d0e60 con 0x7f9ef4004850 2023-11-28T13:01:42.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.314+0000 7f9f017fa700 1 -- 172.21.15.153:0/2599949185 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ef8003cb0 con 0x7f9ef4004850 2023-11-28T13:01:42.315 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ee80682f0 msgr2=0x7f9ee806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ee80682f0 0x7f9ee806a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f040cbab0 tx=0x7f9ef0008030 comp rx=0 tx=0).stop 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 msgr2=0x7f9f04150410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9f04150410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800f7f0 tx=0x7f9ef800fbb0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 shutdown_connections 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ee80682f0 0x7f9ee806a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ef4004850 0x7f9f04150410 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.317 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f041e5140 0x7f9f0409b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 --2- 172.21.15.153:0/2599949185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f04150950 0x7f9f041e4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 >> 172.21.15.153:0/2599949185 conn(0x7f9f040d0000 msgr2=0x7f9f0406e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 shutdown_connections 2023-11-28T13:01:42.318 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.316+0000 7f9f0ac0b700 1 -- 172.21.15.153:0/2599949185 wait complete. 2023-11-28T13:01:42.329 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:42.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.451+0000 7fa7afa54700 1 Processor -- start 2023-11-28T13:01:42.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.452+0000 7fa7afa54700 1 -- start start 2023-11-28T13:01:42.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.452+0000 7fa7afa54700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80d01b0 0x7fa7a80d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.452+0000 7fa7afa54700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa7a80b20b0 0x7fa7a80d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.452+0000 7fa7afa54700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a80b0230 con 0x7fa7a80b29b0 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.452+0000 7fa7afa54700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a815cec0 con 0x7fa7a80b20b0 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.452+0000 7fa7afa54700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a80b4570 con 0x7fa7a80d01b0 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa7a80b20b0 0x7fa7a80d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa7a80b20b0 0x7fa7a80d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40724/0 (socket says 172.21.15.153:40724) 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 -- 172.21.15.153:0/1477644911 learned_addr learned my addr 172.21.15.153:0/1477644911 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7ad7f0700 1 --2- 172.21.15.153:0/1477644911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80d01b0 0x7fa7a80d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 -- 172.21.15.153:0/1477644911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80d01b0 msgr2=0x7fa7a80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 --2- 172.21.15.153:0/1477644911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80d01b0 0x7fa7a80d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 -- 172.21.15.153:0/1477644911 >> v1:172.21.15.177:6789/0 conn(0x7fa7a80b29b0 legacy=0x7fa7a80b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 -- 172.21.15.153:0/1477644911 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80b4850 con 0x7fa7a80b20b0 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7ad7f0700 1 --2- 172.21.15.153:0/1477644911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80d01b0 0x7fa7a80d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa7acfef700 1 --2- 172.21.15.153:0/1477644911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80b20b0 0x7fa7a80d1160 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40098f0 tx=0x7fa7a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f9d0dabe17c9f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa79ffff700 1 -- 172.21.15.153:0/1477644911 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400b8a0 con 0x7fa7a80b20b0 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa79ffff700 1 -- 172.21.15.153:0/1477644911 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a400ba00 con 0x7fa7a80b20b0 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.453+0000 7fa79ffff700 1 -- 172.21.15.153:0/1477644911 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400bd90 con 0x7fa7a80b20b0 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 -- 172.21.15.153:0/1477644911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80b20b0 msgr2=0x7fa7a80d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 --2- 172.21.15.153:0/1477644911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80b20b0 0x7fa7a80d1160 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40098f0 tx=0x7fa7a4009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 -- 172.21.15.153:0/1477644911 shutdown_connections 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 --2- 172.21.15.153:0/1477644911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80d01b0 0x7fa7a80d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 --2- 172.21.15.153:0/1477644911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80b20b0 0x7fa7a80d1160 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 -- 172.21.15.153:0/1477644911 >> 172.21.15.153:0/1477644911 conn(0x7fa7a809c3f0 msgr2=0x7fa7a80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 -- 172.21.15.153:0/1477644911 shutdown_connections 2023-11-28T13:01:42.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.454+0000 7fa7afa54700 1 -- 172.21.15.153:0/1477644911 wait complete. 2023-11-28T13:01:42.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.455+0000 7fa7afa54700 1 Processor -- start 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.455+0000 7fa7afa54700 1 -- start start 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7afa54700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80b20b0 0x7fa7a806b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7afa54700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a80b29b0 0x7fa7a81f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7afa54700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80d01b0 0x7fa7a80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7afa54700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a80cbd30 con 0x7fa7a80d01b0 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7afa54700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81f2810 con 0x7fa7a80b29b0 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7afa54700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81f2af0 con 0x7fa7a80b20b0 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80d01b0 0x7fa7a80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80d01b0 0x7fa7a80ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40732/0 (socket says 172.21.15.153:40732) 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 -- 172.21.15.153:0/3767432912 learned_addr learned my addr 172.21.15.153:0/3767432912 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:42.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7acfef700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a80b29b0 0x7fa7a81f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7ad7f0700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80b20b0 0x7fa7a806b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 -- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80b20b0 msgr2=0x7fa7a806b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80b20b0 0x7fa7a806b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 -- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a80b29b0 msgr2=0x7fa7a81f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a80b29b0 0x7fa7a81f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 -- 172.21.15.153:0/3767432912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4009550 con 0x7fa7a80d01b0 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa7adff1700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80d01b0 0x7fa7a80ca450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000cc50 tx=0x7fa7a000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.456+0000 7fa79dffb700 1 -- 172.21.15.153:0/3767432912 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0003c00 con 0x7fa7a80d01b0 2023-11-28T13:01:42.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.457+0000 7fa79dffb700 1 -- 172.21.15.153:0/3767432912 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a0003d60 con 0x7fa7a80d01b0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.457+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a80afec0 con 0x7fa7a80d01b0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.457+0000 7fa79dffb700 1 -- 172.21.15.153:0/3767432912 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000aab0 con 0x7fa7a80d01b0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.457+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a80afbe0 con 0x7fa7a80d01b0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.458+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a80b2600 con 0x7fa7a80d01b0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.458+0000 7fa79dffb700 1 -- 172.21.15.153:0/3767432912 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7a0010050 con 0x7fa7a80d01b0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.458+0000 7fa79dffb700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa78c0682f0 0x7fa78c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.458+0000 7fa79dffb700 1 -- 172.21.15.153:0/3767432912 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7a0085ac0 con 0x7fa7a80d01b0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.458+0000 7fa7ad7f0700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa78c0682f0 0x7fa78c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.459+0000 7fa7ad7f0700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa78c0682f0 0x7fa78c06a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa798006fd0 tx=0x7fa798008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.462+0000 7fa79dffb700 1 -- 172.21.15.153:0/3767432912 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7a0052320 con 0x7fa7a80d01b0 2023-11-28T13:01:42.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.644+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7a815f010 con 0x7fa7a80d01b0 2023-11-28T13:01:42.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.644+0000 7fa79dffb700 1 -- 172.21.15.153:0/3767432912 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7a815f010 con 0x7fa7a80d01b0 2023-11-28T13:01:42.645 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:01:42.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.646+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa78c0682f0 msgr2=0x7fa78c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.646+0000 7fa7afa54700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa78c0682f0 0x7fa78c06a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa798006fd0 tx=0x7fa798008040 comp rx=0 tx=0).stop 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.646+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80d01b0 msgr2=0x7fa7a80ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.646+0000 7fa7afa54700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80d01b0 0x7fa7a80ca450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000cc50 tx=0x7fa7a000cf60 comp rx=0 tx=0).stop 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 shutdown_connections 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa78c0682f0 0x7fa78c06a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7a80d01b0 0x7fa7a80ca450 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a80b20b0 0x7fa7a806b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 --2- 172.21.15.153:0/3767432912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a80b29b0 0x7fa7a81f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 >> 172.21.15.153:0/3767432912 conn(0x7fa7a809c3f0 msgr2=0x7fa7a80b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 shutdown_connections 2023-11-28T13:01:42.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.647+0000 7fa7afa54700 1 -- 172.21.15.153:0/3767432912 wait complete. 2023-11-28T13:01:42.659 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:42.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.779+0000 7fbbdcd63700 1 Processor -- start 2023-11-28T13:01:42.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.779+0000 7fbbdcd63700 1 -- start start 2023-11-28T13:01:42.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.780+0000 7fbbdcd63700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbbd8071040 0x7fbbd8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.780+0000 7fbbdcd63700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd8071950 con 0x7fbbd806dee0 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.780+0000 7fbbdcd63700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd815d060 con 0x7fbbd8160540 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.780+0000 7fbbdcd63700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd815d340 con 0x7fbbd8071040 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd5d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fbbd806dee0 0x7fbbd806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58900/0 (socket says 172.21.15.153:58900) 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd5d9b700 1 -- 172.21.15.153:0/89543119 learned_addr learned my addr 172.21.15.153:0/89543119 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd659c700 1 --2- 172.21.15.153:0/89543119 >> v2:172.21.15.177:3300/0 conn(0x7fbbd8071040 0x7fbbd8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289844293 0 0) 0x7fbbd8071950 con 0x7fbbd806dee0 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc00035b0 con 0x7fbbd806dee0 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 773399416 0 0) 0x7fbbc00035b0 con 0x7fbbd806dee0 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 >> v2:172.21.15.177:3300/0 conn(0x7fbbd8071040 msgr2=0x7fbbd8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 --2- 172.21.15.153:0/89543119 >> v2:172.21.15.177:3300/0 conn(0x7fbbd8071040 0x7fbbd8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 >> v1:172.21.15.177:6789/0 conn(0x7fbbd8160540 legacy=0x7fbbd8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd815d620 con 0x7fbbd806dee0 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fbbc8001670 con 0x7fbbd806dee0 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 >> v1:172.21.15.153:6789/0 conn(0x7fbbd806dee0 legacy=0x7fbbd806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbc0004850 0x7fbbc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbc000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd815d340 con 0x7fbbc0005320 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc000c250 con 0x7fbbc0003a00 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc000c410 con 0x7fbbc0004850 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd6d9d700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbc000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd659c700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.781+0000 7fbbd5d9b700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbc0004850 0x7fbbc0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd6d9d700 1 -- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbc0004850 msgr2=0x7fbbc0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd6d9d700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbc0004850 0x7fbbc0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd6d9d700 1 -- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbc0003a00 msgr2=0x7fbbc0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd6d9d700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd6d9d700 1 -- 172.21.15.153:0/89543119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00035b0 con 0x7fbbc0005320 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd6d9d700 1 -- 172.21.15.153:0/89543119 reap_dead start 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd6d9d700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbc000ba10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c240 tx=0x7fbbc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81c5e811c931483 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0001df0 con 0x7fbbc0005320 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd000cda0 con 0x7fbbc0005320 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbd559a700 1 -- 172.21.15.153:0/89543119 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0003c40 con 0x7fbbc0005320 2023-11-28T13:01:42.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbdcd63700 1 -- 172.21.15.153:0/89543119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 msgr2=0x7fbbc000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbc000ba10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c240 tx=0x7fbbc0005880 comp rx=0 tx=0).stop 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbdcd63700 1 -- 172.21.15.153:0/89543119 shutdown_connections 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/89543119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbc000ba10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.782+0000 7fbbdcd63700 1 -- 172.21.15.153:0/89543119 >> 172.21.15.153:0/89543119 conn(0x7fbbd80d0000 msgr2=0x7fbbd80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.783+0000 7fbbdcd63700 1 -- 172.21.15.153:0/89543119 shutdown_connections 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.783+0000 7fbbdcd63700 1 -- 172.21.15.153:0/89543119 wait complete. 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.784+0000 7fbbdcd63700 1 Processor -- start 2023-11-28T13:01:42.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.784+0000 7fbbdcd63700 1 -- start start 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbd81f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbd8157200 0x7fbbd80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbd80b5a60 0x7fbbd80c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80b5ed0 con 0x7fbbc0005320 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81561f0 con 0x7fbbd80b5a60 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8156440 con 0x7fbbd8157200 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbd81f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbd81f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40748/0 (socket says 172.21.15.153:40748) 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 -- 172.21.15.153:0/3182345931 learned_addr learned my addr 172.21.15.153:0/3182345931 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd6d9d700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbd80b5a60 0x7fbbd80c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd5d9b700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbd8157200 0x7fbbd80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 -- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbd8157200 msgr2=0x7fbbd80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbd8157200 0x7fbbd80b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 -- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbd80b5a60 msgr2=0x7fbbd80c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbd80b5a60 0x7fbbd80c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 -- 172.21.15.153:0/3182345931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd815ccd0 con 0x7fbbc0005320 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd5d9b700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbd8157200 0x7fbbd80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbd659c700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbd81f1f30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0119d0 tx=0x7fbbcc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbc6ffd700 1 -- 172.21.15.153:0/3182345931 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc00f8e0 con 0x7fbbc0005320 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbc6ffd700 1 -- 172.21.15.153:0/3182345931 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc00fa40 con 0x7fbbc0005320 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd815d900 con 0x7fbbc0005320 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbc6ffd700 1 -- 172.21.15.153:0/3182345931 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc00fdd0 con 0x7fbbc0005320 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.785+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd806a270 con 0x7fbbc0005320 2023-11-28T13:01:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.786+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd80720a0 con 0x7fbbc0005320 2023-11-28T13:01:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.787+0000 7fbbc6ffd700 1 -- 172.21.15.153:0/3182345931 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbbcc014440 con 0x7fbbc0005320 2023-11-28T13:01:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.787+0000 7fbbc6ffd700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbbb80682f0 0x7fbbb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.787+0000 7fbbc6ffd700 1 -- 172.21.15.153:0/3182345931 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbcc0863c0 con 0x7fbbc0005320 2023-11-28T13:01:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.787+0000 7fbbd5d9b700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbbb80682f0 0x7fbbb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.787+0000 7fbbd5d9b700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbbb80682f0 0x7fbbb806a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80aede0 tx=0x7fbbc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:42.791+0000 7fbbc6ffd700 1 -- 172.21.15.153:0/3182345931 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbbcc052bc0 con 0x7fbbc0005320 2023-11-28T13:01:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.015+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbd806f440 con 0x7fbbc0005320 2023-11-28T13:01:43.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.015+0000 7fbbc6ffd700 1 -- 172.21.15.153:0/3182345931 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7fbbcc0561e0 con 0x7fbbc0005320 2023-11-28T13:01:43.016 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:43.016 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbbb80682f0 msgr2=0x7fbbb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbbb80682f0 0x7fbbb806a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80aede0 tx=0x7fbbc8009040 comp rx=0 tx=0).stop 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 msgr2=0x7fbbd81f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbd81f1f30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0119d0 tx=0x7fbbcc011ce0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 shutdown_connections 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbbb80682f0 0x7fbbb806a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbbc0005320 0x7fbbd81f1f30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbd80b5a60 0x7fbbd80c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 --2- 172.21.15.153:0/3182345931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbd8157200 0x7fbbd80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 >> 172.21.15.153:0/3182345931 conn(0x7fbbd80d0000 msgr2=0x7fbbd815fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 shutdown_connections 2023-11-28T13:01:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.017+0000 7fbbdcd63700 1 -- 172.21.15.153:0/3182345931 wait complete. 2023-11-28T13:01:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:43.030 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:43.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.151+0000 7f1a3a178700 1 Processor -- start 2023-11-28T13:01:43.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.151+0000 7f1a3a178700 1 -- start start 2023-11-28T13:01:43.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.152+0000 7f1a3a178700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.152+0000 7f1a3a178700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a340d1950 con 0x7f1a34161340 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.152+0000 7f1a3a178700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a340d24f0 con 0x7f1a340cd7c0 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.152+0000 7f1a3a178700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340b5800 con 0x7f1a3415c9b0 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.152+0000 7f1a32ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1a34161340 0x7f1a340d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:58920/0 (socket says 172.21.15.153:58920) 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.152+0000 7f1a32ffd700 1 -- 172.21.15.153:0/1201246981 learned_addr learned my addr 172.21.15.153:0/1201246981 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.152+0000 7f1a337fe700 1 --2- 172.21.15.153:0/1201246981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a327fc700 1 -- 172.21.15.153:0/1201246981 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673673723 0 0) 0x7f1a340d1950 con 0x7f1a34161340 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a327fc700 1 -- 172.21.15.153:0/1201246981 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a200035b0 con 0x7f1a34161340 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a327fc700 1 -- 172.21.15.153:0/1201246981 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197236594 0 0) 0x7f1a340d24f0 con 0x7f1a340cd7c0 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a327fc700 1 -- 172.21.15.153:0/1201246981 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a340d1950 con 0x7f1a340cd7c0 2023-11-28T13:01:43.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a337fe700 1 -- 172.21.15.153:0/1201246981 >> v1:172.21.15.177:6789/0 conn(0x7f1a340cd7c0 legacy=0x7f1a340b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a337fe700 1 -- 172.21.15.153:0/1201246981 >> v1:172.21.15.153:6789/0 conn(0x7f1a34161340 legacy=0x7f1a340d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a337fe700 1 -- 172.21.15.153:0/1201246981 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340b5520 con 0x7f1a3415c9b0 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a337fe700 1 --2- 172.21.15.153:0/1201246981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009870 tx=0x7f1a1c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878e2b436f3e629a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a327fc700 1 -- 172.21.15.153:0/1201246981 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011830 con 0x7f1a3415c9b0 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a327fc700 1 -- 172.21.15.153:0/1201246981 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c011990 con 0x7f1a3415c9b0 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.153+0000 7f1a327fc700 1 -- 172.21.15.153:0/1201246981 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c011d00 con 0x7f1a3415c9b0 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.154+0000 7f1a3a178700 1 -- 172.21.15.153:0/1201246981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415c9b0 msgr2=0x7f1a340b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.154+0000 7f1a3a178700 1 --2- 172.21.15.153:0/1201246981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009870 tx=0x7f1a1c009b80 comp rx=0 tx=0).stop 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.154+0000 7f1a3a178700 1 -- 172.21.15.153:0/1201246981 shutdown_connections 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.154+0000 7f1a3a178700 1 --2- 172.21.15.153:0/1201246981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415c9b0 0x7f1a340b4bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.154+0000 7f1a3a178700 1 -- 172.21.15.153:0/1201246981 >> 172.21.15.153:0/1201246981 conn(0x7f1a34072600 msgr2=0x7f1a340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.154+0000 7f1a3a178700 1 -- 172.21.15.153:0/1201246981 shutdown_connections 2023-11-28T13:01:43.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.154+0000 7f1a3a178700 1 -- 172.21.15.153:0/1201246981 wait complete. 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.155+0000 7f1a3a178700 1 Processor -- start 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.155+0000 7f1a3a178700 1 -- start start 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a3a178700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a340cd7c0 0x7f1a3414a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a3a178700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3415c9b0 0x7f1a341e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a3a178700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34161340 0x7f1a341e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a3a178700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340cc990 con 0x7f1a3415c9b0 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a3a178700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340cbd10 con 0x7f1a340cd7c0 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a3a178700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340cbff0 con 0x7f1a34161340 2023-11-28T13:01:43.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3415c9b0 0x7f1a341e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3415c9b0 0x7f1a341e6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40764/0 (socket says 172.21.15.153:40764) 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 -- 172.21.15.153:0/3557604990 learned_addr learned my addr 172.21.15.153:0/3557604990 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a337fe700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a340cd7c0 0x7f1a3414a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a33fff700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34161340 0x7f1a341e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 -- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34161340 msgr2=0x7f1a341e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34161340 0x7f1a341e73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 -- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a340cd7c0 msgr2=0x7f1a3414a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a340cd7c0 0x7f1a3414a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 -- 172.21.15.153:0/3557604990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c0094a0 con 0x7f1a3415c9b0 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a32ffd700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3415c9b0 0x7f1a341e6e70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c4d0 tx=0x7f1a2400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.156+0000 7f1a2bfff700 1 -- 172.21.15.153:0/3557604990 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24005d70 con 0x7f1a3415c9b0 2023-11-28T13:01:43.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.157+0000 7f1a2bfff700 1 -- 172.21.15.153:0/3557604990 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a240070d0 con 0x7f1a3415c9b0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.157+0000 7f1a2bfff700 1 -- 172.21.15.153:0/3557604990 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24007480 con 0x7f1a3415c9b0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.157+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a340d15e0 con 0x7f1a3415c9b0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.157+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a340d1300 con 0x7f1a3415c9b0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.158+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a340b5130 con 0x7f1a3415c9b0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.158+0000 7f1a2bfff700 1 -- 172.21.15.153:0/3557604990 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a24019030 con 0x7f1a3415c9b0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.158+0000 7f1a2bfff700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a140682f0 0x7f1a1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.158+0000 7f1a2bfff700 1 -- 172.21.15.153:0/3557604990 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a240851b0 con 0x7f1a3415c9b0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.158+0000 7f1a337fe700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a140682f0 0x7f1a1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.159+0000 7f1a337fe700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a140682f0 0x7f1a1406a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0090b0 tx=0x7f1a1c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.163+0000 7f1a2bfff700 1 -- 172.21.15.153:0/3557604990 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a24051920 con 0x7f1a3415c9b0 2023-11-28T13:01:43.347 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.347+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a340ccad0 con 0x7f1a3415c9b0 2023-11-28T13:01:43.348 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.348+0000 7f1a2bfff700 1 -- 172.21.15.153:0/3557604990 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f1a340ccad0 con 0x7f1a3415c9b0 2023-11-28T13:01:43.349 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:43.349 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":39,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:14.078173+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:11.364224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:12.036978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:43.351 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a140682f0 msgr2=0x7f1a1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.351 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a140682f0 0x7f1a1406a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0090b0 tx=0x7f1a1c01c040 comp rx=0 tx=0).stop 2023-11-28T13:01:43.351 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3415c9b0 msgr2=0x7f1a341e6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.351 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3415c9b0 0x7f1a341e6e70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c4d0 tx=0x7f1a2400c890 comp rx=0 tx=0).stop 2023-11-28T13:01:43.351 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 shutdown_connections 2023-11-28T13:01:43.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34161340 0x7f1a341e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a340cd7c0 0x7f1a3414a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1a140682f0 0x7f1a1406a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 --2- 172.21.15.153:0/3557604990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a3415c9b0 0x7f1a341e6e70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 >> 172.21.15.153:0/3557604990 conn(0x7f1a34072600 msgr2=0x7f1a340d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 shutdown_connections 2023-11-28T13:01:43.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.350+0000 7f1a3a178700 1 -- 172.21.15.153:0/3557604990 wait complete. 2023-11-28T13:01:43.363 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:01:43.363 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.484+0000 7f9c979f9700 1 Processor -- start 2023-11-28T13:01:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.485+0000 7f9c979f9700 1 -- start start 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.485+0000 7f9c979f9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9c90071040 0x7f9c90071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.485+0000 7f9c979f9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c9006dee0 0x7f9c9006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.485+0000 7f9c979f9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9006cdc0 0x7f9c9006d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.485+0000 7f9c979f9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c90071950 con 0x7f9c9006dee0 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.485+0000 7f9c979f9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9015d060 con 0x7f9c90071040 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.485+0000 7f9c979f9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9015d340 con 0x7f9c9006cdc0 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c9006dee0 0x7f9c9006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c9006dee0 0x7f9c9006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40770/0 (socket says 172.21.15.153:40770) 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 -- 172.21.15.153:0/2623570994 learned_addr learned my addr 172.21.15.153:0/2623570994 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c95795700 1 --2- 172.21.15.153:0/2623570994 >> v2:172.21.15.177:3300/0 conn(0x7f9c90071040 0x7f9c90071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 -- 172.21.15.153:0/2623570994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9006cdc0 msgr2=0x7f9c9006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 --2- 172.21.15.153:0/2623570994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9006cdc0 0x7f9c9006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 -- 172.21.15.153:0/2623570994 >> v2:172.21.15.177:3300/0 conn(0x7f9c90071040 msgr2=0x7f9c90071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 --2- 172.21.15.153:0/2623570994 >> v2:172.21.15.177:3300/0 conn(0x7f9c90071040 0x7f9c90071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 -- 172.21.15.153:0/2623570994 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9015d620 con 0x7f9c9006dee0 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c94f94700 1 --2- 172.21.15.153:0/2623570994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006dee0 0x7f9c9006e330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9c900d0f50 tx=0x7f9c8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425f22bfde495488 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c87fff700 1 -- 172.21.15.153:0/2623570994 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00b870 con 0x7f9c9006dee0 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c87fff700 1 -- 172.21.15.153:0/2623570994 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c00b9d0 con 0x7f9c9006dee0 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.486+0000 7f9c87fff700 1 -- 172.21.15.153:0/2623570994 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00bd60 con 0x7f9c9006dee0 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 -- 172.21.15.153:0/2623570994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006dee0 msgr2=0x7f9c9006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 --2- 172.21.15.153:0/2623570994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006dee0 0x7f9c9006e330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9c900d0f50 tx=0x7f9c8c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 -- 172.21.15.153:0/2623570994 shutdown_connections 2023-11-28T13:01:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 --2- 172.21.15.153:0/2623570994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9006cdc0 0x7f9c9006d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 --2- 172.21.15.153:0/2623570994 >> v2:172.21.15.177:3300/0 conn(0x7f9c90071040 0x7f9c90071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 --2- 172.21.15.153:0/2623570994 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006dee0 0x7f9c9006e330 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 -- 172.21.15.153:0/2623570994 >> 172.21.15.153:0/2623570994 conn(0x7f9c900d0000 msgr2=0x7f9c900d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 -- 172.21.15.153:0/2623570994 shutdown_connections 2023-11-28T13:01:43.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.487+0000 7f9c979f9700 1 -- 172.21.15.153:0/2623570994 wait complete. 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.488+0000 7f9c979f9700 1 Processor -- start 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c979f9700 1 -- start start 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c979f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006cdc0 0x7f9c900c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c979f9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9006dee0 0x7f9c900b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c979f9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c90071040 0x7f9c90156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c979f9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900c49c0 con 0x7f9c9006cdc0 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c979f9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b54e0 con 0x7f9c9006dee0 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c979f9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900c4bb0 con 0x7f9c90071040 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c95795700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006cdc0 0x7f9c900c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.489+0000 7f9c95795700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006cdc0 0x7f9c900c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40776/0 (socket says 172.21.15.153:40776) 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95795700 1 -- 172.21.15.153:0/645117878 learned_addr learned my addr 172.21.15.153:0/645117878 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c94f94700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9006dee0 0x7f9c900b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95f96700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c90071040 0x7f9c90156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95795700 1 -- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c90071040 msgr2=0x7f9c90156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95795700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c90071040 0x7f9c90156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95795700 1 -- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9006dee0 msgr2=0x7f9c900b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95795700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9006dee0 0x7f9c900b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95795700 1 -- 172.21.15.153:0/645117878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c009550 con 0x7f9c9006cdc0 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c94f94700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9006dee0 0x7f9c900b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c95795700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006cdc0 0x7f9c900c4480 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c820 tx=0x7f9c8000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c85ffb700 1 -- 172.21.15.153:0/645117878 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80011800 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c85ffb700 1 -- 172.21.15.153:0/645117878 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c80011960 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c85ffb700 1 -- 172.21.15.153:0/645117878 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80011cf0 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9006e870 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.490+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9015d900 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.491+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9015ccd0 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.492+0000 7f9c85ffb700 1 -- 172.21.15.153:0/645117878 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c80021070 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.492+0000 7f9c85ffb700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9c740682f0 0x7f9c7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.492+0000 7f9c85ffb700 1 -- 172.21.15.153:0/645117878 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c80085a80 con 0x7f9c9006cdc0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.492+0000 7f9c94f94700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9c740682f0 0x7f9c7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.492+0000 7f9c94f94700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9c740682f0 0x7f9c7406a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9c90159140 tx=0x7f9c8c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.496+0000 7f9c85ffb700 1 -- 172.21.15.153:0/645117878 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c800522e0 con 0x7f9c9006cdc0 2023-11-28T13:01:43.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.724+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c900720a0 con 0x7f9c9006cdc0 2023-11-28T13:01:43.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.725+0000 7f9c85ffb700 1 -- 172.21.15.153:0/645117878 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f9c8001f090 con 0x7f9c9006cdc0 2023-11-28T13:01:43.726 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:43.726 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:43.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9c740682f0 msgr2=0x7f9c7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9c740682f0 0x7f9c7406a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9c90159140 tx=0x7f9c8c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:43.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006cdc0 msgr2=0x7f9c900c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006cdc0 0x7f9c900c4480 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c820 tx=0x7f9c8000cbe0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 shutdown_connections 2023-11-28T13:01:43.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9c740682f0 0x7f9c7406a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c9006cdc0 0x7f9c900c4480 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c90071040 0x7f9c90156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 --2- 172.21.15.153:0/645117878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9006dee0 0x7f9c900b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 >> 172.21.15.153:0/645117878 conn(0x7f9c900d0000 msgr2=0x7f9c9006d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 shutdown_connections 2023-11-28T13:01:43.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.727+0000 7f9c979f9700 1 -- 172.21.15.153:0/645117878 wait complete. 2023-11-28T13:01:43.731 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:43.742 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:43.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.864+0000 7f6c749aa700 1 Processor -- start 2023-11-28T13:01:43.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.864+0000 7f6c749aa700 1 -- start start 2023-11-28T13:01:43.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c749aa700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c70140950 0x7f6c700b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c749aa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c7006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c749aa700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7013ff00 con 0x7f6c700cd8c0 2023-11-28T13:01:43.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c749aa700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c701404c0 con 0x7f6c70140950 2023-11-28T13:01:43.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c749aa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c70140dc0 con 0x7f6c700b2e70 2023-11-28T13:01:43.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c70140950 0x7f6c700b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6c70140950 0x7f6c700b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40786/0 (socket says 172.21.15.153:40786) 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 -- 172.21.15.153:0/2930129154 learned_addr learned my addr 172.21.15.153:0/2930129154 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6ed9d700 1 --2- 172.21.15.153:0/2930129154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c7006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 -- 172.21.15.153:0/2930129154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 msgr2=0x7f6c7006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 --2- 172.21.15.153:0/2930129154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c7006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 -- 172.21.15.153:0/2930129154 >> v1:172.21.15.177:6789/0 conn(0x7f6c700cd8c0 legacy=0x7f6c700cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 -- 172.21.15.153:0/2930129154 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7015aa80 con 0x7f6c70140950 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6dd9b700 1 --2- 172.21.15.153:0/2930129154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c700b2930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098f0 tx=0x7f6c64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f840a190d5814fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.865+0000 7f6c6d59a700 1 -- 172.21.15.153:0/2930129154 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400b8a0 con 0x7f6c70140950 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c6d59a700 1 -- 172.21.15.153:0/2930129154 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6400ba00 con 0x7f6c70140950 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c6d59a700 1 -- 172.21.15.153:0/2930129154 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400bd90 con 0x7f6c70140950 2023-11-28T13:01:43.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 -- 172.21.15.153:0/2930129154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 msgr2=0x7f6c700b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 --2- 172.21.15.153:0/2930129154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c700b2930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098f0 tx=0x7f6c64009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:43.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 -- 172.21.15.153:0/2930129154 shutdown_connections 2023-11-28T13:01:43.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 --2- 172.21.15.153:0/2930129154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c7006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 --2- 172.21.15.153:0/2930129154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c700b2930 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 -- 172.21.15.153:0/2930129154 >> 172.21.15.153:0/2930129154 conn(0x7f6c7006aed0 msgr2=0x7f6c7006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 -- 172.21.15.153:0/2930129154 shutdown_connections 2023-11-28T13:01:43.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.866+0000 7f6c749aa700 1 -- 172.21.15.153:0/2930129154 wait complete. 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.867+0000 7f6c749aa700 1 Processor -- start 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c749aa700 1 -- start start 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c749aa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c701f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c749aa700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c700cd8c0 0x7f6c7013ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c749aa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c7013d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c749aa700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c83f0 con 0x7f6c70140950 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c749aa700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c66a0 con 0x7f6c700cd8c0 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c749aa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c6980 con 0x7f6c700b2e70 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c6ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c7013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c6ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c7013d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40794/0 (socket says 172.21.15.153:40794) 2023-11-28T13:01:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c6ed9d700 1 -- 172.21.15.153:0/460450921 learned_addr learned my addr 172.21.15.153:0/460450921 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c6dd9b700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c700cd8c0 0x7f6c7013ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.868+0000 7f6c6e59c700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c701f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c6ed9d700 1 -- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 msgr2=0x7f6c701f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c6ed9d700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c701f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c6ed9d700 1 -- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c700cd8c0 msgr2=0x7f6c7013ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c6ed9d700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c700cd8c0 0x7f6c7013ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c6ed9d700 1 -- 172.21.15.153:0/460450921 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009550 con 0x7f6c70140950 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c6ed9d700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c7013d230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c68007e30 tx=0x7f6c6800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c5effd700 1 -- 172.21.15.153:0/460450921 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800ed30 con 0x7f6c70140950 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c5effd700 1 -- 172.21.15.153:0/460450921 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6800ee90 con 0x7f6c70140950 2023-11-28T13:01:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c700b35f0 con 0x7f6c70140950 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c5effd700 1 -- 172.21.15.153:0/460450921 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68014660 con 0x7f6c70140950 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.869+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c700b3310 con 0x7f6c70140950 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.870+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c70140150 con 0x7f6c70140950 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.870+0000 7f6c5effd700 1 -- 172.21.15.153:0/460450921 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c68012050 con 0x7f6c70140950 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.871+0000 7f6c5effd700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6c500682f0 0x7f6c5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.871+0000 7f6c5effd700 1 -- 172.21.15.153:0/460450921 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c68086220 con 0x7f6c70140950 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.871+0000 7f6c6e59c700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6c500682f0 0x7f6c5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.871+0000 7f6c6e59c700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6c500682f0 0x7f6c5006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000fb0 tx=0x7f6c60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:43.875+0000 7f6c5effd700 1 -- 172.21.15.153:0/460450921 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6c68052a80 con 0x7f6c70140950 2023-11-28T13:01:44.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.102+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c700c7dd0 con 0x7f6c70140950 2023-11-28T13:01:44.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.103+0000 7f6c5effd700 1 -- 172.21.15.153:0/460450921 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4654 (secure 0 0 0) 0x7f6c680560a0 con 0x7f6c70140950 2023-11-28T13:01:44.104 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:44.104 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:13.076520+0000","modified":"2023-11-28T13:01:16.090678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/122658331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":122658331},{"type":"v1","addr":"172.21.15.153:6837","nonce":122658331}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:44.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6c500682f0 msgr2=0x7f6c5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6c500682f0 0x7f6c5006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000fb0 tx=0x7f6c60009040 comp rx=0 tx=0).stop 2023-11-28T13:01:44.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 msgr2=0x7f6c7013d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c7013d230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c68007e30 tx=0x7f6c6800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 shutdown_connections 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6c500682f0 0x7f6c5006a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c70140950 0x7f6c7013d230 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c700b2e70 0x7f6c701f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 --2- 172.21.15.153:0/460450921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c700cd8c0 0x7f6c7013ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 >> 172.21.15.153:0/460450921 conn(0x7f6c7006aed0 msgr2=0x7f6c700cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 shutdown_connections 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.105+0000 7f6c749aa700 1 -- 172.21.15.153:0/460450921 wait complete. 2023-11-28T13:01:44.107 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 16 2023-11-28T13:01:44.118 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.239+0000 7eff927d7700 1 Processor -- start 2023-11-28T13:01:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.239+0000 7eff927d7700 1 -- start start 2023-11-28T13:01:44.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff927d7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff927d7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7eff8c0ce4b0 0x7eff8c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff927d7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8c06bf10 con 0x7eff8c06fb30 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff927d7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c15d730 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff927d7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c1590b0 con 0x7eff8c158ce0 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff8bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff90d74700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7eff8c0ce4b0 0x7eff8c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff8bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c15d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:58408/0 (socket says 172.21.15.153:58408) 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff8bfff700 1 -- 172.21.15.153:0/3963748828 learned_addr learned my addr 172.21.15.153:0/3963748828 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.240+0000 7eff8b7fe700 1 --1- 172.21.15.153:0/3963748828 >> v1:172.21.15.177:6789/0 conn(0x7eff8c06fb30 0x7eff8c0cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:35576/0 (socket says 172.21.15.153:35576) 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff90d74700 1 -- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 msgr2=0x7eff8c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:44.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff90d74700 1 --2- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff90d74700 1 -- 172.21.15.153:0/3963748828 >> v1:172.21.15.177:6789/0 conn(0x7eff8c06fb30 legacy=0x7eff8c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff90d74700 1 -- 172.21.15.153:0/3963748828 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c0cb0d0 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff8affd700 1 -- 172.21.15.153:0/3963748828 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895078610 0 0) 0x7eff8c06bf10 con 0x7eff8c06fb30 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff8bfff700 1 --2- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff90d74700 1 --2- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c06c3a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff84009d90 tx=0x7eff8400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845bd4b5c066b046 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff8affd700 1 -- 172.21.15.153:0/3963748828 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400e070 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff8affd700 1 -- 172.21.15.153:0/3963748828 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8400bb20 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff8affd700 1 -- 172.21.15.153:0/3963748828 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84012430 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.241+0000 7eff927d7700 1 -- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c0ce4b0 msgr2=0x7eff8c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.242+0000 7eff927d7700 1 --2- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c06c3a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff84009d90 tx=0x7eff8400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:44.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.242+0000 7eff927d7700 1 -- 172.21.15.153:0/3963748828 shutdown_connections 2023-11-28T13:01:44.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.242+0000 7eff927d7700 1 --2- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c06c3a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.242+0000 7eff927d7700 1 --2- 172.21.15.153:0/3963748828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.242+0000 7eff927d7700 1 -- 172.21.15.153:0/3963748828 >> 172.21.15.153:0/3963748828 conn(0x7eff8c0cb670 msgr2=0x7eff8c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:44.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.242+0000 7eff927d7700 1 -- 172.21.15.153:0/3963748828 shutdown_connections 2023-11-28T13:01:44.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.242+0000 7eff927d7700 1 -- 172.21.15.153:0/3963748828 wait complete. 2023-11-28T13:01:44.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.243+0000 7eff927d7700 1 Processor -- start 2023-11-28T13:01:44.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.243+0000 7eff927d7700 1 -- start start 2023-11-28T13:01:44.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff927d7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c06fb30 0x7eff8c14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff927d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff927d7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff927d7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0b2430 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff927d7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0b3290 con 0x7eff8c06fb30 2023-11-28T13:01:44.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff927d7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0b2680 con 0x7eff8c158ce0 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40806/0 (socket says 172.21.15.153:40806) 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 -- 172.21.15.153:0/2178212798 learned_addr learned my addr 172.21.15.153:0/2178212798 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8bfff700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c06fb30 0x7eff8c14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff90d74700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 -- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 msgr2=0x7eff8c0b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c0b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 -- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c06fb30 msgr2=0x7eff8c14bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c06fb30 0x7eff8c14bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 -- 172.21.15.153:0/2178212798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8000b7f0 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff8b7fe700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c0b3f30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0011e0 tx=0x7eff7c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff88ff9700 1 -- 172.21.15.153:0/2178212798 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c004c80 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff88ff9700 1 -- 172.21.15.153:0/2178212798 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c004e00 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff84009a70 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.244+0000 7eff88ff9700 1 -- 172.21.15.153:0/2178212798 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c007310 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.245+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c06b8c0 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.246+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c0cb3b0 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.246+0000 7eff88ff9700 1 -- 172.21.15.153:0/2178212798 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff7c01a020 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.246+0000 7eff88ff9700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff780682f0 0x7eff7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.247+0000 7eff88ff9700 1 -- 172.21.15.153:0/2178212798 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff7c085800 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.247+0000 7eff8bfff700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff780682f0 0x7eff7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.247+0000 7eff8bfff700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff780682f0 0x7eff7806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff80006d30 tx=0x7eff8000f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.251+0000 7eff88ff9700 1 -- 172.21.15.153:0/2178212798 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff7c052000 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.435+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff8c06bba0 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.437+0000 7eff88ff9700 1 -- 172.21.15.153:0/2178212798 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18027 (secure 0 0 0) 0x7eff7c055620 con 0x7eff8c0ce4b0 2023-11-28T13:01:44.438 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:44.438 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":39,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:14.078173+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:11.364224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:12.036978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff780682f0 msgr2=0x7eff7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff780682f0 0x7eff7806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff80006d30 tx=0x7eff8000f040 comp rx=0 tx=0).stop 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff8c0ce4b0 msgr2=0x7eff8c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c0b3f30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0011e0 tx=0x7eff7c00ca30 comp rx=0 tx=0).stop 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 shutdown_connections 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff8c158ce0 0x7eff8c0b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff8c06fb30 0x7eff8c14bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7eff780682f0 0x7eff7806a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 --2- 172.21.15.153:0/2178212798 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7eff8c0ce4b0 0x7eff8c0b3f30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 >> 172.21.15.153:0/2178212798 conn(0x7eff8c0cb670 msgr2=0x7eff8c06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:44.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 shutdown_connections 2023-11-28T13:01:44.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.439+0000 7eff927d7700 1 -- 172.21.15.153:0/2178212798 wait complete. 2023-11-28T13:01:44.452 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:01:44.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.573+0000 7f52fe94d700 1 Processor -- start 2023-11-28T13:01:44.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.573+0000 7f52fe94d700 1 -- start start 2023-11-28T13:01:44.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fe94d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f52f806dee0 0x7f52f806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fe94d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f8160540 0x7f52f81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fe94d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f815d060 con 0x7f52f8071040 2023-11-28T13:01:44.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fe94d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f815d340 con 0x7f52f806dee0 2023-11-28T13:01:44.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fe94d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f815d620 con 0x7f52f8160540 2023-11-28T13:01:44.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52f77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f52f806dee0 0x7f52f806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fceea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f8160540 0x7f52f81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52f7fff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f52f8071040 0x7f52f8071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:35590/0 (socket says 172.21.15.153:35590) 2023-11-28T13:01:44.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52f7fff700 1 -- 172.21.15.153:0/3251045481 learned_addr learned my addr 172.21.15.153:0/3251045481 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:44.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52f6ffd700 1 -- 172.21.15.153:0/3251045481 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685747152 0 0) 0x7f52f815d060 con 0x7f52f8071040 2023-11-28T13:01:44.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52f6ffd700 1 -- 172.21.15.153:0/3251045481 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e00035b0 con 0x7f52f8071040 2023-11-28T13:01:44.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fceea700 1 -- 172.21.15.153:0/3251045481 >> v2:172.21.15.177:3300/0 conn(0x7f52f806dee0 msgr2=0x7f52f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:44.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fceea700 1 --2- 172.21.15.153:0/3251045481 >> v2:172.21.15.177:3300/0 conn(0x7f52f806dee0 0x7f52f806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fceea700 1 -- 172.21.15.153:0/3251045481 >> v1:172.21.15.177:6789/0 conn(0x7f52f8071040 legacy=0x7f52f8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52fceea700 1 -- 172.21.15.153:0/3251045481 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f815d900 con 0x7f52f8160540 2023-11-28T13:01:44.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.574+0000 7f52f77fe700 1 --2- 172.21.15.153:0/3251045481 >> v2:172.21.15.177:3300/0 conn(0x7f52f806dee0 0x7f52f806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:44.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fceea700 1 --2- 172.21.15.153:0/3251045481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f8160540 0x7f52f81609c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009d90 tx=0x7f52f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb63016e345d7761 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52f6ffd700 1 -- 172.21.15.153:0/3251045481 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f000e070 con 0x7f52f8160540 2023-11-28T13:01:44.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52f6ffd700 1 -- 172.21.15.153:0/3251045481 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f000bb20 con 0x7f52f8160540 2023-11-28T13:01:44.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52f6ffd700 1 -- 172.21.15.153:0/3251045481 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f000be90 con 0x7f52f8160540 2023-11-28T13:01:44.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 -- 172.21.15.153:0/3251045481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f8160540 msgr2=0x7f52f81609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 --2- 172.21.15.153:0/3251045481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f8160540 0x7f52f81609c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009d90 tx=0x7f52f000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:44.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 -- 172.21.15.153:0/3251045481 shutdown_connections 2023-11-28T13:01:44.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 --2- 172.21.15.153:0/3251045481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f8160540 0x7f52f81609c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 --2- 172.21.15.153:0/3251045481 >> v2:172.21.15.177:3300/0 conn(0x7f52f806dee0 0x7f52f806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 -- 172.21.15.153:0/3251045481 >> 172.21.15.153:0/3251045481 conn(0x7f52f80d0000 msgr2=0x7f52f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:44.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 -- 172.21.15.153:0/3251045481 shutdown_connections 2023-11-28T13:01:44.581 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.575+0000 7f52fe94d700 1 -- 172.21.15.153:0/3251045481 wait complete. 2023-11-28T13:01:44.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.576+0000 7f52fe94d700 1 Processor -- start 2023-11-28T13:01:44.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.577+0000 7f52fe94d700 1 -- start start 2023-11-28T13:01:44.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.577+0000 7f52fe94d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f806dee0 0x7f52f81f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.577+0000 7f52fe94d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f8071040 0x7f52f80b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.577+0000 7f52fe94d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52f8160540 0x7f52f8156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.577+0000 7f52fe94d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b5b60 con 0x7f52f8160540 2023-11-28T13:01:44.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.577+0000 7f52fe94d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b2e30 con 0x7f52f8071040 2023-11-28T13:01:44.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.577+0000 7f52fe94d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f8156680 con 0x7f52f806dee0 2023-11-28T13:01:44.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52f8160540 0x7f52f8156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52f8160540 0x7f52f8156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40822/0 (socket says 172.21.15.153:40822) 2023-11-28T13:01:44.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52f77fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f8071040 0x7f52f80b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 -- 172.21.15.153:0/1878824449 learned_addr learned my addr 172.21.15.153:0/1878824449 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:44.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52f7fff700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f806dee0 0x7f52f81f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 -- 172.21.15.153:0/1878824449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f806dee0 msgr2=0x7f52f81f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f806dee0 0x7f52f81f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 -- 172.21.15.153:0/1878824449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f8071040 msgr2=0x7f52f80b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f8071040 0x7f52f80b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 -- 172.21.15.153:0/1878824449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0094a0 con 0x7f52f8160540 2023-11-28T13:01:44.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fceea700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52f8160540 0x7f52f8156140 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52f815d760 tx=0x7f52f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52f4ff9700 1 -- 172.21.15.153:0/1878824449 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0004880 con 0x7f52f8160540 2023-11-28T13:01:44.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52f4ff9700 1 -- 172.21.15.153:0/1878824449 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0020070 con 0x7f52f8160540 2023-11-28T13:01:44.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f0009a70 con 0x7f52f8160540 2023-11-28T13:01:44.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52f4ff9700 1 -- 172.21.15.153:0/1878824449 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00276c0 con 0x7f52f8160540 2023-11-28T13:01:44.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.578+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f815ccd0 con 0x7f52f8160540 2023-11-28T13:01:44.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.579+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f806e870 con 0x7f52f8160540 2023-11-28T13:01:44.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.580+0000 7f52f4ff9700 1 -- 172.21.15.153:0/1878824449 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52f0027820 con 0x7f52f8160540 2023-11-28T13:01:44.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.580+0000 7f52f4ff9700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52e40682f0 0x7f52e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.580+0000 7f52f4ff9700 1 -- 172.21.15.153:0/1878824449 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52f0022070 con 0x7f52f8160540 2023-11-28T13:01:44.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.580+0000 7f52f7fff700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52e40682f0 0x7f52e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.580+0000 7f52f7fff700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52e40682f0 0x7f52e406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52e8007fd0 tx=0x7f52e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.584+0000 7f52f4ff9700 1 -- 172.21.15.153:0/1878824449 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52f0054690 con 0x7f52f8160540 2023-11-28T13:01:44.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:44.811+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f52f806a270 con 0x7f52f8160540 2023-11-28T13:01:45.431 INFO:tasks.ceph.mon.a.smithi153.stderr:2023-11-28T13:01:45.430+0000 7feb34d4f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:01:45.437 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.436+0000 7f52f4ff9700 1 -- 172.21.15.153:0/1878824449 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f52f002c090 con 0x7f52f8160540 2023-11-28T13:01:45.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.438+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52e40682f0 msgr2=0x7f52e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.438+0000 7f52fe94d700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52e40682f0 0x7f52e406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52e8007fd0 tx=0x7f52e8009040 comp rx=0 tx=0).stop 2023-11-28T13:01:45.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.438+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52f8160540 msgr2=0x7f52f8156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.438+0000 7f52fe94d700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52f8160540 0x7f52f8156140 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52f815d760 tx=0x7f52f0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 shutdown_connections 2023-11-28T13:01:45.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f52e40682f0 0x7f52e406a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52f8160540 0x7f52f8156140 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52f806dee0 0x7f52f81f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 --2- 172.21.15.153:0/1878824449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52f8071040 0x7f52f80b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 >> 172.21.15.153:0/1878824449 conn(0x7f52f80d0000 msgr2=0x7f52f815f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:45.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 shutdown_connections 2023-11-28T13:01:45.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.439+0000 7f52fe94d700 1 -- 172.21.15.153:0/1878824449 wait complete. 2023-11-28T13:01:45.443 INFO:teuthology.orchestra.run.smithi153.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:01:45.443 INFO:tasks.ceph.mds.a.smithi153.stderr: -7361> 2023-11-28T13:01:16.602+0000 7f72e99b8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:01:45.454 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:01:45.477 INFO:tasks.ceph.mds.a.smithi153.stdout:starting mds.a at 2023-11-28T13:01:45.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.578+0000 7fcaa0d5e700 1 Processor -- start 2023-11-28T13:01:45.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.578+0000 7fcaa0d5e700 1 -- start start 2023-11-28T13:01:45.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.579+0000 7fcaa0d5e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fca9c06ba90 0x7fca9c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.579+0000 7fcaa0d5e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c0cc5f0 con 0x7fca9c0cbdf0 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.579+0000 7fcaa0d5e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c15d470 con 0x7fca9c15cef0 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.579+0000 7fcaa0d5e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c15d750 con 0x7fca9c06ba90 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.579+0000 7fca99d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fca9c06ba90 0x7fca9c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.579+0000 7fca9ad9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fca9c0cbdf0 0x7fca9c0ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59012/0 (socket says 172.21.15.153:59012) 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.579+0000 7fca9ad9d700 1 -- 172.21.15.153:0/105252113 learned_addr learned my addr 172.21.15.153:0/105252113 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.580+0000 7fca99d9b700 1 -- 172.21.15.153:0/105252113 >> v1:172.21.15.177:6789/0 conn(0x7fca9c15cef0 legacy=0x7fca9c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.580+0000 7fca99d9b700 1 -- 172.21.15.153:0/105252113 >> v1:172.21.15.153:6789/0 conn(0x7fca9c0cbdf0 legacy=0x7fca9c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.580+0000 7fca99d9b700 1 -- 172.21.15.153:0/105252113 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c15da30 con 0x7fca9c06ba90 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.580+0000 7fca99d9b700 1 --2- 172.21.15.153:0/105252113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c0af400 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009840 tx=0x7fca8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb759a503df94ec8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.580+0000 7fca9959a700 1 -- 172.21.15.153:0/105252113 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c011830 con 0x7fca9c06ba90 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.580+0000 7fca9959a700 1 -- 172.21.15.153:0/105252113 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c011990 con 0x7fca9c06ba90 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.580+0000 7fca9959a700 1 -- 172.21.15.153:0/105252113 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c011d20 con 0x7fca9c06ba90 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.581+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/105252113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 msgr2=0x7fca9c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.581+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/105252113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c0af400 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009840 tx=0x7fca8c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.581+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/105252113 shutdown_connections 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.581+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/105252113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c0af400 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.581+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/105252113 >> 172.21.15.153:0/105252113 conn(0x7fca9c0d0000 msgr2=0x7fca9c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.581+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/105252113 shutdown_connections 2023-11-28T13:01:45.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.581+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/105252113 wait complete. 2023-11-28T13:01:45.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.582+0000 7fcaa0d5e700 1 Processor -- start 2023-11-28T13:01:45.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.582+0000 7fcaa0d5e700 1 -- start start 2023-11-28T13:01:45.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fcaa0d5e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fcaa0d5e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca9c0cbdf0 0x7fca9c0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fcaa0d5e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca9c15cef0 0x7fca9c0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fcaa0d5e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1f2400 con 0x7fca9c06ba90 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fcaa0d5e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0c50a0 con 0x7fca9c15cef0 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fcaa0d5e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0c5340 con 0x7fca9c0cbdf0 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c1f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40850/0 (socket says 172.21.15.153:40850) 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 -- 172.21.15.153:0/2050041900 learned_addr learned my addr 172.21.15.153:0/2050041900 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9ad9d700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca9c15cef0 0x7fca9c0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca99d9b700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca9c0cbdf0 0x7fca9c0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 -- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca9c0cbdf0 msgr2=0x7fca9c0c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca9c0cbdf0 0x7fca9c0c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 -- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca9c15cef0 msgr2=0x7fca9c0c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca9c15cef0 0x7fca9c0c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 -- 172.21.15.153:0/2050041900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c0094a0 con 0x7fca9c06ba90 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca9a59c700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c1f1ec0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c450 tx=0x7fca9000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.583+0000 7fca8affd700 1 -- 172.21.15.153:0/2050041900 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900052d0 con 0x7fca9c06ba90 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.584+0000 7fca8affd700 1 -- 172.21.15.153:0/2050041900 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90005430 con 0x7fca9c06ba90 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.584+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c15dff0 con 0x7fca9c06ba90 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.584+0000 7fca8affd700 1 -- 172.21.15.153:0/2050041900 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900038c0 con 0x7fca9c06ba90 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.584+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c15dd10 con 0x7fca9c06ba90 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.584+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c0cad60 con 0x7fca9c06ba90 2023-11-28T13:01:45.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.585+0000 7fca8affd700 1 -- 172.21.15.153:0/2050041900 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fca90019030 con 0x7fca9c06ba90 2023-11-28T13:01:45.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.585+0000 7fca8affd700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca7c0682f0 0x7fca7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.585+0000 7fca99d9b700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca7c0682f0 0x7fca7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.585+0000 7fca8affd700 1 -- 172.21.15.153:0/2050041900 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fca90084e90 con 0x7fca9c06ba90 2023-11-28T13:01:45.590 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.586+0000 7fca99d9b700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca7c0682f0 0x7fca7c06a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0090b0 tx=0x7fca8c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:45.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.590+0000 7fca8affd700 1 -- 172.21.15.153:0/2050041900 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fca90051690 con 0x7fca9c06ba90 2023-11-28T13:01:45.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:45.813+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca9c1e93c0 con 0x7fca9c06ba90 2023-11-28T13:01:46.218 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:46.221 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:46.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.439+0000 7fca8affd700 1 -- 172.21.15.153:0/2050041900 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fca9000ce60 con 0x7fca9c06ba90 2023-11-28T13:01:46.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca7c0682f0 msgr2=0x7fca7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca7c0682f0 0x7fca7c06a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0090b0 tx=0x7fca8c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:46.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 msgr2=0x7fca9c1f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c1f1ec0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c450 tx=0x7fca9000c760 comp rx=0 tx=0).stop 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 shutdown_connections 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca7c0682f0 0x7fca7c06a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca9c06ba90 0x7fca9c1f1ec0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca9c15cef0 0x7fca9c0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 --2- 172.21.15.153:0/2050041900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca9c0cbdf0 0x7fca9c0c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.441+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 >> 172.21.15.153:0/2050041900 conn(0x7fca9c0d0000 msgr2=0x7fca9c0cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.442+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 shutdown_connections 2023-11-28T13:01:46.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.442+0000 7fcaa0d5e700 1 -- 172.21.15.153:0/2050041900 wait complete. 2023-11-28T13:01:46.456 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:01:46.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.583+0000 7fceef453700 1 Processor -- start 2023-11-28T13:01:46.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.583+0000 7fceef453700 1 -- start start 2023-11-28T13:01:46.593 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceef453700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee81596b0 0x7fcee8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceef453700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcee80cc260 0x7fcee80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceef453700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcee80d02a0 0x7fcee815d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceef453700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80d0720 con 0x7fcee80cc260 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceef453700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80cc7b0 con 0x7fcee80d02a0 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceef453700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80cd610 con 0x7fcee81596b0 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcee80cc260 0x7fcee80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcee80cc260 0x7fcee80cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40862/0 (socket says 172.21.15.153:40862) 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 -- 172.21.15.153:0/1640521998 learned_addr learned my addr 172.21.15.153:0/1640521998 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:46.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceed9f0700 1 --2- 172.21.15.153:0/1640521998 >> v2:172.21.15.177:3300/0 conn(0x7fcee80d02a0 0x7fcee815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceed1ef700 1 --2- 172.21.15.153:0/1640521998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee81596b0 0x7fcee8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 -- 172.21.15.153:0/1640521998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee81596b0 msgr2=0x7fcee8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 --2- 172.21.15.153:0/1640521998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee81596b0 0x7fcee8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 -- 172.21.15.153:0/1640521998 >> v2:172.21.15.177:3300/0 conn(0x7fcee80d02a0 msgr2=0x7fcee815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 --2- 172.21.15.153:0/1640521998 >> v2:172.21.15.177:3300/0 conn(0x7fcee80d02a0 0x7fcee815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 -- 172.21.15.153:0/1640521998 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee8159a80 con 0x7fcee80cc260 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fceec9ee700 1 --2- 172.21.15.153:0/1640521998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80cc260 0x7fcee80cfd60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcee8072960 tx=0x7fced8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0541ccfc3122a4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fcedffff700 1 -- 172.21.15.153:0/1640521998 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800b870 con 0x7fcee80cc260 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.584+0000 7fcedffff700 1 -- 172.21.15.153:0/1640521998 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced800b9d0 con 0x7fcee80cc260 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fcedffff700 1 -- 172.21.15.153:0/1640521998 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800bd60 con 0x7fcee80cc260 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 -- 172.21.15.153:0/1640521998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80cc260 msgr2=0x7fcee80cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.595 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 --2- 172.21.15.153:0/1640521998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80cc260 0x7fcee80cfd60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcee8072960 tx=0x7fced8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 -- 172.21.15.153:0/1640521998 shutdown_connections 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 --2- 172.21.15.153:0/1640521998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee81596b0 0x7fcee8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 --2- 172.21.15.153:0/1640521998 >> v2:172.21.15.177:3300/0 conn(0x7fcee80d02a0 0x7fcee815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 --2- 172.21.15.153:0/1640521998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80cc260 0x7fcee80cfd60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 -- 172.21.15.153:0/1640521998 >> 172.21.15.153:0/1640521998 conn(0x7fcee806b380 msgr2=0x7fcee806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 -- 172.21.15.153:0/1640521998 shutdown_connections 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.585+0000 7fceef453700 1 -- 172.21.15.153:0/1640521998 wait complete. 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.587+0000 7fceef453700 1 Processor -- start 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.587+0000 7fceef453700 1 -- start start 2023-11-28T13:01:46.596 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceef453700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee80cc260 0x7fcee8152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceef453700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80d02a0 0x7fcee80c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceef453700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcee81596b0 0x7fcee80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceef453700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80c8b30 con 0x7fcee80d02a0 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceef453700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80c8290 con 0x7fcee81596b0 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceef453700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80c8570 con 0x7fcee80cc260 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80d02a0 0x7fcee80c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80d02a0 0x7fcee80c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40868/0 (socket says 172.21.15.153:40868) 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 -- 172.21.15.153:0/2855317623 learned_addr learned my addr 172.21.15.153:0/2855317623 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceed1ef700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee80cc260 0x7fcee8152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceed9f0700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcee81596b0 0x7fcee80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 -- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee80cc260 msgr2=0x7fcee8152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee80cc260 0x7fcee8152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 -- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcee81596b0 msgr2=0x7fcee80c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcee81596b0 0x7fcee80c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 -- 172.21.15.153:0/2855317623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009550 con 0x7fcee80d02a0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceec9ee700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80d02a0 0x7fcee80c78b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceddffb700 1 -- 172.21.15.153:0/2855317623 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8004520 con 0x7fcee80d02a0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceddffb700 1 -- 172.21.15.153:0/2855317623 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced801b6c0 con 0x7fcee80d02a0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee80ccce0 con 0x7fcee80d02a0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.588+0000 7fceddffb700 1 -- 172.21.15.153:0/2855317623 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced801ba50 con 0x7fcee80d02a0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.589+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee80cca00 con 0x7fcee80d02a0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.590+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee80ccfc0 con 0x7fcee80d02a0 2023-11-28T13:01:46.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.590+0000 7fceddffb700 1 -- 172.21.15.153:0/2855317623 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fced802a030 con 0x7fcee80d02a0 2023-11-28T13:01:46.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.590+0000 7fceddffb700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcecc068340 0x7fcecc06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.590+0000 7fceddffb700 1 -- 172.21.15.153:0/2855317623 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fced8088c50 con 0x7fcee80d02a0 2023-11-28T13:01:46.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.590+0000 7fceed1ef700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcecc068340 0x7fcecc06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.591+0000 7fceed1ef700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcecc068340 0x7fcecc06a7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009950 tx=0x7fcee4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.599 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.595+0000 7fceddffb700 1 -- 172.21.15.153:0/2855317623 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fced80554f0 con 0x7fcee80d02a0 2023-11-28T13:01:46.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:46.776+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcee8068690 con 0x7fcee80d02a0 2023-11-28T13:01:47.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.497+0000 7fceddffb700 1 -- 172.21.15.153:0/2855317623 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7fced8058b10 con 0x7fcee80d02a0 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.501+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcecc068340 msgr2=0x7fcecc06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.501+0000 7fceef453700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcecc068340 0x7fcecc06a7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009950 tx=0x7fcee4008040 comp rx=0 tx=0).stop 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80d02a0 msgr2=0x7fcee80c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80d02a0 0x7fcee80c78b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced8004750 comp rx=0 tx=0).stop 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 shutdown_connections 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcee81596b0 0x7fcee80c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcee80cc260 0x7fcee8152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcecc068340 0x7fcecc06a7f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 --2- 172.21.15.153:0/2855317623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcee80d02a0 0x7fcee80c78b0 secure :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fced8000c00 tx=0x7fced8004750 comp rx=0 tx=0).stop 2023-11-28T13:01:47.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 >> 172.21.15.153:0/2855317623 conn(0x7fcee806b380 msgr2=0x7fcee80cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:47.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 shutdown_connections 2023-11-28T13:01:47.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.502+0000 7fceef453700 1 -- 172.21.15.153:0/2855317623 wait complete. 2023-11-28T13:01:47.504 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:01:47.518 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:01:47.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.648+0000 7fc7597f4700 1 Processor -- start 2023-11-28T13:01:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.648+0000 7fc7597f4700 1 -- start start 2023-11-28T13:01:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.648+0000 7fc7597f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc7540cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.648+0000 7fc7597f4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc75406ba90 0x7fc7540cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.648+0000 7fc7597f4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7540cbea0 con 0x7fc7540d08b0 2023-11-28T13:01:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.648+0000 7fc7597f4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540cb5b0 con 0x7fc75406ba90 2023-11-28T13:01:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7597f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415cf60 con 0x7fc75415cb90 2023-11-28T13:01:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7537fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc75406ba90 0x7fc7540cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7527fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc7540d08b0 0x7fc75406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59038/0 (socket says 172.21.15.153:59038) 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7527fc700 1 -- 172.21.15.153:0/4215502567 learned_addr learned my addr 172.21.15.153:0/4215502567 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc752ffd700 1 --2- 172.21.15.153:0/4215502567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc7540cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7537fe700 1 -- 172.21.15.153:0/4215502567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 msgr2=0x7fc7540cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7537fe700 1 --2- 172.21.15.153:0/4215502567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc7540cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7537fe700 1 -- 172.21.15.153:0/4215502567 >> v1:172.21.15.153:6789/0 conn(0x7fc7540d08b0 legacy=0x7fc75406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7537fe700 1 -- 172.21.15.153:0/4215502567 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540cfbd0 con 0x7fc75406ba90 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc751ffb700 1 -- 172.21.15.153:0/4215502567 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400200042 0 0) 0x7fc7540cbea0 con 0x7fc7540d08b0 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc752ffd700 1 --2- 172.21.15.153:0/4215502567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc7540cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc7537fe700 1 --2- 172.21.15.153:0/4215502567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc7540cc330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009d90 tx=0x7fc74c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef9afa9b9112691 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc751ffb700 1 -- 172.21.15.153:0/4215502567 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00e070 con 0x7fc75406ba90 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc751ffb700 1 -- 172.21.15.153:0/4215502567 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c00bbd0 con 0x7fc75406ba90 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.649+0000 7fc751ffb700 1 -- 172.21.15.153:0/4215502567 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c012430 con 0x7fc75406ba90 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 -- 172.21.15.153:0/4215502567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 msgr2=0x7fc7540cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 --2- 172.21.15.153:0/4215502567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc7540cc330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009d90 tx=0x7fc74c00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 -- 172.21.15.153:0/4215502567 shutdown_connections 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 --2- 172.21.15.153:0/4215502567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc7540cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 --2- 172.21.15.153:0/4215502567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc7540cc330 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 -- 172.21.15.153:0/4215502567 >> 172.21.15.153:0/4215502567 conn(0x7fc75409a230 msgr2=0x7fc75408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 -- 172.21.15.153:0/4215502567 shutdown_connections 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.650+0000 7fc7597f4700 1 -- 172.21.15.153:0/4215502567 wait complete. 2023-11-28T13:01:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.651+0000 7fc7597f4700 1 Processor -- start 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.651+0000 7fc7597f4700 1 -- start start 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7597f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc754148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7597f4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7540d08b0 0x7fc75415e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7597f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc75415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7597f4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415f9a0 con 0x7fc75406ba90 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7597f4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415ed20 con 0x7fc7540d08b0 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7597f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75415f000 con 0x7fc75415cb90 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc754148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc754148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40880/0 (socket says 172.21.15.153:40880) 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 -- 172.21.15.153:0/1703875453 learned_addr learned my addr 172.21.15.153:0/1703875453 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:47.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7537fe700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc75415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 -- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 msgr2=0x7fc75415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc75415e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7527fc700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7540d08b0 0x7fc75415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 -- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7540d08b0 msgr2=0x7fc75415e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7540d08b0 0x7fc75415e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 -- 172.21.15.153:0/1703875453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c009a70 con 0x7fc75406ba90 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc7527fc700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7540d08b0 0x7fc75415e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.652+0000 7fc752ffd700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc754148490 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e810 tx=0x7fc74400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.653+0000 7fc73b7fe700 1 -- 172.21.15.153:0/1703875453 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744004230 con 0x7fc75406ba90 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.653+0000 7fc73b7fe700 1 -- 172.21.15.153:0/1703875453 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744004390 con 0x7fc75406ba90 2023-11-28T13:01:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.653+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7540cbb30 con 0x7fc75406ba90 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.653+0000 7fc73b7fe700 1 -- 172.21.15.153:0/1703875453 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440036a0 con 0x7fc75406ba90 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.653+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7540cb850 con 0x7fc75406ba90 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.654+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7540d0d00 con 0x7fc75406ba90 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.654+0000 7fc73b7fe700 1 -- 172.21.15.153:0/1703875453 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc744022070 con 0x7fc75406ba90 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.654+0000 7fc73b7fe700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7400682f0 0x7fc74006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.654+0000 7fc73b7fe700 1 -- 172.21.15.153:0/1703875453 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fc744085660 con 0x7fc75406ba90 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.654+0000 7fc7527fc700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7400682f0 0x7fc74006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.655+0000 7fc7527fc700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7400682f0 0x7fc74006a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7540ce7c0 tx=0x7fc748009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.659+0000 7fc73b7fe700 1 -- 172.21.15.153:0/1703875453 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc744051fc0 con 0x7fc75406ba90 2023-11-28T13:01:47.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:47.840+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc7541e5200 con 0x7fc75406ba90 2023-11-28T13:01:48.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.502+0000 7fc73b7fe700 1 -- 172.21.15.153:0/1703875453 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc7440094c0 con 0x7fc75406ba90 2023-11-28T13:01:48.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7400682f0 msgr2=0x7fc74006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7400682f0 0x7fc74006a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7540ce7c0 tx=0x7fc748009040 comp rx=0 tx=0).stop 2023-11-28T13:01:48.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 msgr2=0x7fc754148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc754148490 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e810 tx=0x7fc74400ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 shutdown_connections 2023-11-28T13:01:48.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7400682f0 0x7fc74006a7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc75406ba90 0x7fc754148490 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc75415cb90 0x7fc75415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 --2- 172.21.15.153:0/1703875453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7540d08b0 0x7fc75415e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 >> 172.21.15.153:0/1703875453 conn(0x7fc75409a230 msgr2=0x7fc7540aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 shutdown_connections 2023-11-28T13:01:48.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.505+0000 7fc7597f4700 1 -- 172.21.15.153:0/1703875453 wait complete. 2023-11-28T13:01:48.508 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' removed 2023-11-28T13:01:48.520 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:01:48.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.648+0000 7f03ad52b700 1 Processor -- start 2023-11-28T13:01:48.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.649+0000 7f03ad52b700 1 -- start start 2023-11-28T13:01:48.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.649+0000 7f03ad52b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a8158ce0 0x7f03a815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.649+0000 7f03ad52b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a806bf10 con 0x7f03a806fb30 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.649+0000 7f03ad52b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a815d730 con 0x7f03a80ce4b0 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.649+0000 7f03ad52b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81590b0 con 0x7f03a8158ce0 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.649+0000 7f03a67fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f03a806fb30 0x7f03a80cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59052/0 (socket says 172.21.15.153:59052) 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.649+0000 7f03a67fc700 1 -- 172.21.15.153:0/2910939884 learned_addr learned my addr 172.21.15.153:0/2910939884 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392196773 0 0) 0x7f03a806bf10 con 0x7f03a806fb30 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03900035b0 con 0x7f03a806fb30 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a6ffd700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a8158ce0 0x7f03a815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1934530559 0 0) 0x7f03900035b0 con 0x7f03a806fb30 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a8158ce0 msgr2=0x7f03a815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a8158ce0 0x7f03a815d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 >> v1:172.21.15.177:6789/0 conn(0x7f03a80ce4b0 legacy=0x7f03a806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a80cb0d0 con 0x7f03a806fb30 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f0398001670 con 0x7f03a806fb30 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 >> v1:172.21.15.153:6789/0 conn(0x7f03a806fb30 legacy=0x7f03a80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0390003a30 0x7f0390003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0390004850 0x7f0390004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390005200 0x7f039000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81590b0 con 0x7f0390004850 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039000c000 con 0x7f0390003a30 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039000c1c0 con 0x7f0390005200 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a6ffd700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03a8158ce0 0x7f03a815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a67fc700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0390004850 0x7f0390004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a77fe700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390005200 0x7f039000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a6ffd700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0390003a30 0x7f0390003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a67fc700 1 -- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390005200 msgr2=0x7f039000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a67fc700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390005200 0x7f039000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a67fc700 1 -- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0390003a30 msgr2=0x7f0390003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a67fc700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0390003a30 0x7f0390003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.650+0000 7f03a67fc700 1 -- 172.21.15.153:0/2910939884 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03900035b0 con 0x7f0390004850 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03a77fe700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390005200 0x7f039000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03a6ffd700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0390003a30 0x7f0390003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03a67fc700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0390004850 0x7f0390004cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f039800c2f0 tx=0x7f0390005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9980f9455f039a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800ce30 con 0x7f0390004850 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0398005960 con 0x7f0390004850 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03a5ffb700 1 -- 172.21.15.153:0/2910939884 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398005d10 con 0x7f0390004850 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03ad52b700 1 -- 172.21.15.153:0/2910939884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0390004850 msgr2=0x7f0390004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03ad52b700 1 --2- 172.21.15.153:0/2910939884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0390004850 0x7f0390004cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f039800c2f0 tx=0x7f0390005880 comp rx=0 tx=0).stop 2023-11-28T13:01:48.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03a77fe700 1 -- 172.21.15.153:0/2910939884 reap_dead start 2023-11-28T13:01:48.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03ad52b700 1 -- 172.21.15.153:0/2910939884 shutdown_connections 2023-11-28T13:01:48.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03ad52b700 1 -- 172.21.15.153:0/2910939884 >> 172.21.15.153:0/2910939884 conn(0x7f03a80cb670 msgr2=0x7f03a80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03ad52b700 1 -- 172.21.15.153:0/2910939884 shutdown_connections 2023-11-28T13:01:48.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.651+0000 7f03ad52b700 1 -- 172.21.15.153:0/2910939884 wait complete. 2023-11-28T13:01:48.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 Processor -- start 2023-11-28T13:01:48.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 -- start start 2023-11-28T13:01:48.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390004850 0x7f03a8146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a8146680 0x7f03a81f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a80cd270 0x7f03a81f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81502d0 con 0x7f03a80cd270 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80c1740 con 0x7f03a8146680 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.653+0000 7f03ad52b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80b3d40 con 0x7f0390004850 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a80cd270 0x7f03a81f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a80cd270 0x7f03a81f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40904/0 (socket says 172.21.15.153:40904) 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 -- 172.21.15.153:0/1349766723 learned_addr learned my addr 172.21.15.153:0/1349766723 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a67fc700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a8146680 0x7f03a81f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a6ffd700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390004850 0x7f03a8146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 -- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390004850 msgr2=0x7f03a8146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390004850 0x7f03a8146140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 -- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a8146680 msgr2=0x7f03a81f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a8146680 0x7f03a81f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 -- 172.21.15.153:0/1349766723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a806b8c0 con 0x7f03a80cd270 2023-11-28T13:01:48.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03a77fe700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a80cd270 0x7f03a81f64d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f03a80c2660 tx=0x7f03a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f038f7fe700 1 -- 172.21.15.153:0/1349766723 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0003c10 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f038f7fe700 1 -- 172.21.15.153:0/1349766723 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a0003d90 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a80cb3b0 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f038f7fe700 1 -- 172.21.15.153:0/1349766723 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0005310 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.654+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a806bba0 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.655+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a80745c0 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.656+0000 7f038f7fe700 1 -- 172.21.15.153:0/1349766723 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f03a0005470 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.656+0000 7f038f7fe700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0394068320 0x7f039406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.656+0000 7f038f7fe700 1 -- 172.21.15.153:0/1349766723 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f03a0085b40 con 0x7f03a80cd270 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.656+0000 7f03a6ffd700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0394068320 0x7f039406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.656+0000 7f03a6ffd700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0394068320 0x7f039406a7d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f039c006f10 tx=0x7f039c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.660+0000 7f038f7fe700 1 -- 172.21.15.153:0/1349766723 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f03a00526a0 con 0x7f03a80cd270 2023-11-28T13:01:48.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.851+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f03a80b4b70 con 0x7f03a80cd270 2023-11-28T13:01:48.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.855+0000 7f038f7fe700 1 -- 172.21.15.153:0/1349766723 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f03a0001cc0 con 0x7f03a80cd270 2023-11-28T13:01:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0394068320 msgr2=0x7f039406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0394068320 0x7f039406a7d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f039c006f10 tx=0x7f039c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a80cd270 msgr2=0x7f03a81f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a80cd270 0x7f03a81f64d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f03a80c2660 tx=0x7f03a000ef20 comp rx=0 tx=0).stop 2023-11-28T13:01:48.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 shutdown_connections 2023-11-28T13:01:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0394068320 0x7f039406a7d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f03a80cd270 0x7f03a81f64d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0390004850 0x7f03a8146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 --2- 172.21.15.153:0/1349766723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03a8146680 0x7f03a81f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 >> 172.21.15.153:0/1349766723 conn(0x7f03a80cb670 msgr2=0x7f03a80af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 shutdown_connections 2023-11-28T13:01:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.857+0000 7f03ad52b700 1 -- 172.21.15.153:0/1349766723 wait complete. 2023-11-28T13:01:48.870 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T13:01:48.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.990+0000 7f4700999700 1 Processor -- start 2023-11-28T13:01:48.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.990+0000 7f4700999700 1 -- start start 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f4700999700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46fc071040 0x7f46fc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f4700999700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f46fc160540 0x7f46fc160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f4700999700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc071950 con 0x7f46fc06dee0 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f4700999700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc15d060 con 0x7f46fc071040 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f4700999700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc15d340 con 0x7f46fc160540 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46fc071040 0x7f46fc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46fc071040 0x7f46fc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40908/0 (socket says 172.21.15.153:40908) 2023-11-28T13:01:48.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 -- 172.21.15.153:0/840991894 learned_addr learned my addr 172.21.15.153:0/840991894 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fad9d700 1 --2- 172.21.15.153:0/840991894 >> v2:172.21.15.177:3300/0 conn(0x7f46fc160540 0x7f46fc160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 -- 172.21.15.153:0/840991894 >> v2:172.21.15.177:3300/0 conn(0x7f46fc160540 msgr2=0x7f46fc160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 --2- 172.21.15.153:0/840991894 >> v2:172.21.15.177:3300/0 conn(0x7f46fc160540 0x7f46fc160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 -- 172.21.15.153:0/840991894 >> v1:172.21.15.177:6789/0 conn(0x7f46fc06dee0 legacy=0x7f46fc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 -- 172.21.15.153:0/840991894 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc15d620 con 0x7f46fc071040 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46fa59c700 1 --2- 172.21.15.153:0/840991894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc071040 0x7f46fc071410 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0c3a10 tx=0x7f46ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1fef29ce9e0e07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.991+0000 7f46f959a700 1 -- 172.21.15.153:0/840991894 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec004230 con 0x7f46fc071040 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f46f959a700 1 -- 172.21.15.153:0/840991894 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec004390 con 0x7f46fc071040 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f46f959a700 1 -- 172.21.15.153:0/840991894 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0036a0 con 0x7f46fc071040 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 -- 172.21.15.153:0/840991894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc071040 msgr2=0x7f46fc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 --2- 172.21.15.153:0/840991894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc071040 0x7f46fc071410 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0c3a10 tx=0x7f46ec009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 -- 172.21.15.153:0/840991894 shutdown_connections 2023-11-28T13:01:48.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 --2- 172.21.15.153:0/840991894 >> v2:172.21.15.177:3300/0 conn(0x7f46fc160540 0x7f46fc160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 --2- 172.21.15.153:0/840991894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc071040 0x7f46fc071410 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 -- 172.21.15.153:0/840991894 >> 172.21.15.153:0/840991894 conn(0x7f46fc0d0000 msgr2=0x7f46fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 -- 172.21.15.153:0/840991894 shutdown_connections 2023-11-28T13:01:48.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.992+0000 7f4700999700 1 -- 172.21.15.153:0/840991894 wait complete. 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.993+0000 7f4700999700 1 Processor -- start 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f4700999700 1 -- start start 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f4700999700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46fc06dee0 0x7f46fc1f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f4700999700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc071040 0x7f46fc159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f4700999700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc160540 0x7f46fc159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f4700999700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc1f37f0 con 0x7f46fc160540 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f4700999700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15ab00 con 0x7f46fc071040 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f4700999700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15ade0 con 0x7f46fc06dee0 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f46fad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc160540 0x7f46fc159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f46fad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc160540 0x7f46fc159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40918/0 (socket says 172.21.15.153:40918) 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f46fad9d700 1 -- 172.21.15.153:0/1064398773 learned_addr learned my addr 172.21.15.153:0/1064398773 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.994+0000 7f46f9d9b700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc071040 0x7f46fc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46fa59c700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46fc06dee0 0x7f46fc1f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46fad9d700 1 -- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46fc06dee0 msgr2=0x7f46fc1f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46fad9d700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46fc06dee0 0x7f46fc1f32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46fad9d700 1 -- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc071040 msgr2=0x7f46fc159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46fad9d700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc071040 0x7f46fc159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46fad9d700 1 -- 172.21.15.153:0/1064398773 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec009550 con 0x7f46fc160540 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46f9d9b700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc071040 0x7f46fc159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46fad9d700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc160540 0x7f46fc159640 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f46f4007e60 tx=0x7f46f400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46eaffd700 1 -- 172.21.15.153:0/1064398773 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f400ede0 con 0x7f46fc160540 2023-11-28T13:01:48.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46eaffd700 1 -- 172.21.15.153:0/1064398773 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f4014420 con 0x7f46fc160540 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc15ccd0 con 0x7f46fc160540 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f46eaffd700 1 -- 172.21.15.153:0/1064398773 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f40147d0 con 0x7f46fc160540 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.995+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc15d900 con 0x7f46fc160540 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.996+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc06a270 con 0x7f46fc160540 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.997+0000 7f46eaffd700 1 -- 172.21.15.153:0/1064398773 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46f4012050 con 0x7f46fc160540 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.997+0000 7f46eaffd700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46dc0682f0 0x7f46dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.997+0000 7f46eaffd700 1 -- 172.21.15.153:0/1064398773 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f46f4086dc0 con 0x7f46fc160540 2023-11-28T13:01:49.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.997+0000 7f46fa59c700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46dc0682f0 0x7f46dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:49.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:48.998+0000 7f46fa59c700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46dc0682f0 0x7f46dc06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009840 tx=0x7f46ec008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:49.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.001+0000 7f46eaffd700 1 -- 172.21.15.153:0/1064398773 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46f40539e0 con 0x7f46fc160540 2023-11-28T13:01:49.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.260+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f46fc0720a0 con 0x7f46fc160540 2023-11-28T13:01:49.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.503+0000 7f46eaffd700 1 -- 172.21.15.153:0/1064398773 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v102) v1 ==== 142+0+0 (secure 0 0 0) 0x7f46f4057000 con 0x7f46fc160540 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46dc0682f0 msgr2=0x7f46dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46dc0682f0 0x7f46dc06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009840 tx=0x7f46ec008030 comp rx=0 tx=0).stop 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc160540 msgr2=0x7f46fc159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc160540 0x7f46fc159640 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f46f4007e60 tx=0x7f46f400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 shutdown_connections 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46dc0682f0 0x7f46dc06a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46fc160540 0x7f46fc159640 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46fc06dee0 0x7f46fc1f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 --2- 172.21.15.153:0/1064398773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc071040 0x7f46fc159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 >> 172.21.15.153:0/1064398773 conn(0x7f46fc0d0000 msgr2=0x7f46fc15e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:49.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 shutdown_connections 2023-11-28T13:01:49.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.505+0000 7f4700999700 1 -- 172.21.15.153:0/1064398773 wait complete. 2023-11-28T13:01:49.518 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T13:01:49.518 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T13:01:49.519 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:01:49.519 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T13:01:49.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.639+0000 7fabdf00f700 1 Processor -- start 2023-11-28T13:01:49.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.639+0000 7fabdf00f700 1 -- start start 2023-11-28T13:01:49.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdf00f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fabd806dee0 0x7fabd806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:49.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdf00f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:49.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdf00f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd815d060 con 0x7fabd8071040 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdf00f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd815d340 con 0x7fabd806dee0 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdf00f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd815d620 con 0x7fabd8160540 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdcdab700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fabd8071040 0x7fabd8071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59086/0 (socket says 172.21.15.153:59086) 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdcdab700 1 -- 172.21.15.153:0/3148608338 learned_addr learned my addr 172.21.15.153:0/3148608338 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabdd5ac700 1 --2- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabd7fff700 1 --2- 172.21.15.153:0/3148608338 >> v2:172.21.15.177:3300/0 conn(0x7fabd806dee0 0x7fabd806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabd77fe700 1 -- 172.21.15.153:0/3148608338 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474167296 0 0) 0x7fabd815d060 con 0x7fabd8071040 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.640+0000 7fabd77fe700 1 -- 172.21.15.153:0/3148608338 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc80035b0 con 0x7fabd8071040 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd7fff700 1 -- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 msgr2=0x7fabd81609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd7fff700 1 --2- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd81609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd7fff700 1 -- 172.21.15.153:0/3148608338 >> v1:172.21.15.153:6789/0 conn(0x7fabd8071040 legacy=0x7fabd8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd77fe700 1 -- 172.21.15.153:0/3148608338 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 971664841 0 0) 0x7fabc80035b0 con 0x7fabd8071040 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd7fff700 1 -- 172.21.15.153:0/3148608338 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd815d900 con 0x7fabd806dee0 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabdd5ac700 1 --2- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd81609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd7fff700 1 --2- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd806dee0 0x7fabd806e330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009840 tx=0x7fabc4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e2cfecd1f97634 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd77fe700 1 -- 172.21.15.153:0/3148608338 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc001670 con 0x7fabd806dee0 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd77fe700 1 -- 172.21.15.153:0/3148608338 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc4004030 con 0x7fabd806dee0 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabd77fe700 1 -- 172.21.15.153:0/3148608338 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc40043c0 con 0x7fabd806dee0 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabdf00f700 1 -- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd806dee0 msgr2=0x7fabd806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.641+0000 7fabdf00f700 1 --2- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd806dee0 0x7fabd806e330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009840 tx=0x7fabc4009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.642+0000 7fabdf00f700 1 -- 172.21.15.153:0/3148608338 shutdown_connections 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.642+0000 7fabdf00f700 1 --2- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd81609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.642+0000 7fabdf00f700 1 --2- 172.21.15.153:0/3148608338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd806dee0 0x7fabd806e330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.642+0000 7fabdf00f700 1 -- 172.21.15.153:0/3148608338 >> 172.21.15.153:0/3148608338 conn(0x7fabd80d0000 msgr2=0x7fabd80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:49.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.642+0000 7fabdf00f700 1 -- 172.21.15.153:0/3148608338 shutdown_connections 2023-11-28T13:01:49.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.642+0000 7fabdf00f700 1 -- 172.21.15.153:0/3148608338 wait complete. 2023-11-28T13:01:49.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.643+0000 7fabdf00f700 1 Processor -- start 2023-11-28T13:01:49.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.643+0000 7fabdf00f700 1 -- start start 2023-11-28T13:01:49.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd806dee0 0x7fabd80b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd8071040 0x7fabd81f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c5460 con 0x7fabd806dee0 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81f6d70 con 0x7fabd8071040 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80b4740 con 0x7fabd8160540 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd806dee0 0x7fabd80b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd806dee0 0x7fabd80b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40928/0 (socket says 172.21.15.153:40928) 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 -- 172.21.15.153:0/2409441381 learned_addr learned my addr 172.21.15.153:0/2409441381 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabd7fff700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd8071040 0x7fabd81f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdd5ac700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:49.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 -- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 msgr2=0x7fabd814d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd814d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 -- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd8071040 msgr2=0x7fabd81f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd8071040 0x7fabd81f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 -- 172.21.15.153:0/2409441381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009a70 con 0x7fabd806dee0 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdcdab700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd806dee0 0x7fabd80b3a70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c750 tx=0x7fabcc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabd57fa700 1 -- 172.21.15.153:0/2409441381 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc004c80 con 0x7fabd806dee0 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabd57fa700 1 -- 172.21.15.153:0/2409441381 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc004de0 con 0x7fabd806dee0 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc40094a0 con 0x7fabd806dee0 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabd57fa700 1 -- 172.21.15.153:0/2409441381 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc0072d0 con 0x7fabd806dee0 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.644+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd815ccd0 con 0x7fabd806dee0 2023-11-28T13:01:49.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.645+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd806e870 con 0x7fabd806dee0 2023-11-28T13:01:49.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.646+0000 7fabd57fa700 1 -- 172.21.15.153:0/2409441381 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fabcc01a020 con 0x7fabd806dee0 2023-11-28T13:01:49.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.646+0000 7fabd57fa700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fabbc0682c0 0x7fabbc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:49.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.646+0000 7fabd57fa700 1 -- 172.21.15.153:0/2409441381 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fabcc0854f0 con 0x7fabd806dee0 2023-11-28T13:01:49.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.646+0000 7fabd7fff700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fabbc0682c0 0x7fabbc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:49.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.647+0000 7fabd7fff700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fabbc0682c0 0x7fabbc06a770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fabc40090b0 tx=0x7fabc4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:49.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.651+0000 7fabd57fa700 1 -- 172.21.15.153:0/2409441381 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fabcc052050 con 0x7fabd806dee0 2023-11-28T13:01:49.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:49.908+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fabd806a270 con 0x7fabd806dee0 2023-11-28T13:01:50.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.511+0000 7fabd57fa700 1 -- 172.21.15.153:0/2409441381 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v103) v1 ==== 152+0+0 (secure 0 0 0) 0x7fabcc055670 con 0x7fabd806dee0 2023-11-28T13:01:50.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fabbc0682c0 msgr2=0x7fabbc06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:50.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fabbc0682c0 0x7fabbc06a770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fabc40090b0 tx=0x7fabc4008030 comp rx=0 tx=0).stop 2023-11-28T13:01:50.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd806dee0 msgr2=0x7fabd80b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:50.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd806dee0 0x7fabd80b3a70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c750 tx=0x7fabcc00ca60 comp rx=0 tx=0).stop 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 shutdown_connections 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fabbc0682c0 0x7fabbc06a770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabd806dee0 0x7fabd80b3a70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd8160540 0x7fabd814d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 --2- 172.21.15.153:0/2409441381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd8071040 0x7fabd81f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 >> 172.21.15.153:0/2409441381 conn(0x7fabd80d0000 msgr2=0x7fabd80aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 shutdown_connections 2023-11-28T13:01:50.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.513+0000 7fabdf00f700 1 -- 172.21.15.153:0/2409441381 wait complete. 2023-11-28T13:01:50.527 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:01:50.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.653+0000 7f35b2f2e700 1 Processor -- start 2023-11-28T13:01:50.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.653+0000 7f35b2f2e700 1 -- start start 2023-11-28T13:01:50.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35b2f2e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06e9f0 0x7f35ac06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:50.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35b2f2e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f35ac06b830 0x7f35ac06c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35b2f2e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac06ae60 con 0x7f35ac06ced0 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35b2f2e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac06d330 con 0x7f35ac06b830 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35b2f2e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac06a270 con 0x7f35ac06e9f0 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35abfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f35ac06b830 0x7f35ac06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35abfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f35ac06b830 0x7f35ac06c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40936/0 (socket says 172.21.15.153:40936) 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35abfff700 1 -- 172.21.15.153:0/317380154 learned_addr learned my addr 172.21.15.153:0/317380154 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35abfff700 1 -- 172.21.15.153:0/317380154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06e9f0 msgr2=0x7f35ac06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35abfff700 1 --2- 172.21.15.153:0/317380154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06e9f0 0x7f35ac06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35abfff700 1 -- 172.21.15.153:0/317380154 >> v1:172.21.15.177:6789/0 conn(0x7f35ac06ced0 legacy=0x7f35ac0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.654+0000 7f35abfff700 1 -- 172.21.15.153:0/317380154 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac06a530 con 0x7f35ac06b830 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35abfff700 1 --2- 172.21.15.153:0/317380154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06b830 0x7f35ac06c990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f35980098f0 tx=0x7f3598009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6582cc084638c2ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35ab7fe700 1 -- 172.21.15.153:0/317380154 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598004230 con 0x7f35ac06b830 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35ab7fe700 1 -- 172.21.15.153:0/317380154 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598004390 con 0x7f35ac06b830 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35ab7fe700 1 -- 172.21.15.153:0/317380154 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980036a0 con 0x7f35ac06b830 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 -- 172.21.15.153:0/317380154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06b830 msgr2=0x7f35ac06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/317380154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06b830 0x7f35ac06c990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f35980098f0 tx=0x7f3598009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 -- 172.21.15.153:0/317380154 shutdown_connections 2023-11-28T13:01:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/317380154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06e9f0 0x7f35ac06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/317380154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06b830 0x7f35ac06c990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 -- 172.21.15.153:0/317380154 >> 172.21.15.153:0/317380154 conn(0x7f35ac0b12b0 msgr2=0x7f35ac0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:50.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 -- 172.21.15.153:0/317380154 shutdown_connections 2023-11-28T13:01:50.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.655+0000 7f35b2f2e700 1 -- 172.21.15.153:0/317380154 wait complete. 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 Processor -- start 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 -- start start 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06b830 0x7f35ac1642c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06ced0 0x7f35ac162a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac06e9f0 0x7f35ac14a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0c6e00 con 0x7f35ac06ced0 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac163500 con 0x7f35ac06e9f0 2023-11-28T13:01:50.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.657+0000 7f35b2f2e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1637e0 con 0x7f35ac06b830 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06ced0 0x7f35ac162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06ced0 0x7f35ac162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40944/0 (socket says 172.21.15.153:40944) 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 -- 172.21.15.153:0/3647737366 learned_addr learned my addr 172.21.15.153:0/3647737366 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35b0cca700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06b830 0x7f35ac1642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35b14cb700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac06e9f0 0x7f35ac14a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 -- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06b830 msgr2=0x7f35ac1642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06b830 0x7f35ac1642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 -- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac06e9f0 msgr2=0x7f35ac14a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac06e9f0 0x7f35ac14a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 -- 172.21.15.153:0/3647737366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598009550 con 0x7f35ac06ced0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35b0cca700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06b830 0x7f35ac1642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35abfff700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06ced0 0x7f35ac162a60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0704f0 tx=0x7f3598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35a97fa700 1 -- 172.21.15.153:0/3647737366 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800b660 con 0x7f35ac06ced0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35a97fa700 1 -- 172.21.15.153:0/3647737366 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359800b7c0 con 0x7f35ac06ced0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35a97fa700 1 -- 172.21.15.153:0/3647737366 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800bb50 con 0x7f35ac06ced0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac06aaf0 con 0x7f35ac06ced0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.658+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac06a810 con 0x7f35ac06ced0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.659+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac06edc0 con 0x7f35ac06ced0 2023-11-28T13:01:50.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.659+0000 7f35a97fa700 1 -- 172.21.15.153:0/3647737366 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3598017020 con 0x7f35ac06ced0 2023-11-28T13:01:50.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.660+0000 7f35a97fa700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f35900682f0 0x7f359006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:50.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.660+0000 7f35b0cca700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f35900682f0 0x7f359006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:50.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.660+0000 7f35a97fa700 1 -- 172.21.15.153:0/3647737366 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f35980872e0 con 0x7f35ac06ced0 2023-11-28T13:01:50.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.660+0000 7f35b0cca700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f35900682f0 0x7f359006a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f35ac164890 tx=0x7f35a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:50.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.664+0000 7f35a97fa700 1 -- 172.21.15.153:0/3647737366 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3598053e40 con 0x7f35ac06ced0 2023-11-28T13:01:50.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.848+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f35ac06b2d0 con 0x7f35ac06ced0 2023-11-28T13:01:50.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.853+0000 7f35a97fa700 1 -- 172.21.15.153:0/3647737366 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3598057460 con 0x7f35ac06ced0 2023-11-28T13:01:50.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f35900682f0 msgr2=0x7f359006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:50.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f35900682f0 0x7f359006a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f35ac164890 tx=0x7f35a0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:50.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06ced0 msgr2=0x7f35ac162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:50.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06ced0 0x7f35ac162a60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0704f0 tx=0x7f3598004750 comp rx=0 tx=0).stop 2023-11-28T13:01:50.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 shutdown_connections 2023-11-28T13:01:50.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac06e9f0 0x7f35ac14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac06b830 0x7f35ac1642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f35900682f0 0x7f359006a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 --2- 172.21.15.153:0/3647737366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ac06ced0 0x7f35ac162a60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:50.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 >> 172.21.15.153:0/3647737366 conn(0x7f35ac0b12b0 msgr2=0x7f35ac0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:50.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 shutdown_connections 2023-11-28T13:01:50.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:50.856+0000 7f35b2f2e700 1 -- 172.21.15.153:0/3647737366 wait complete. 2023-11-28T13:01:50.869 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:01:50.869 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:01:50.910 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:01:50.953 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:01:50.953 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:01:50.954 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:50.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:01:50.971 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:01:50.971 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:51.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:01:51.014 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:01:51.014 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:51.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:01:51.031 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:01:51.031 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:51.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:01:51.075 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:01:51.075 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:51.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd32a469700 1 Processor -- start 2023-11-28T13:01:51.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd32a469700 1 -- start start 2023-11-28T13:01:51.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd32a469700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 0x7fd3240d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd32a469700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3240d1950 con 0x7fd32415c9b0 2023-11-28T13:01:51.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd32a469700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3240b5520 con 0x7fd3240d2320 2023-11-28T13:01:51.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd32a469700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3240b5800 con 0x7fd324161340 2023-11-28T13:01:51.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd323fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd32415c9b0 0x7fd3240b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59114/0 (socket says 172.21.15.153:59114) 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.195+0000 7fd323fff700 1 -- 172.21.15.153:0/2771282901 learned_addr learned my addr 172.21.15.153:0/2771282901 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd3237fe700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 0x7fd3240d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127187819 0 0) 0x7fd3240d1950 con 0x7fd32415c9b0 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3100035b0 con 0x7fd32415c9b0 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 603142611 0 0) 0x7fd3100035b0 con 0x7fd32415c9b0 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 msgr2=0x7fd3240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 0x7fd3240d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 >> v1:172.21.15.177:6789/0 conn(0x7fd3240d2320 legacy=0x7fd3240cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240d1300 con 0x7fd32415c9b0 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fd30c001670 con 0x7fd32415c9b0 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 >> v1:172.21.15.153:6789/0 conn(0x7fd32415c9b0 legacy=0x7fd3240b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 0x7fd310003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310003fb0 0x7fd310004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd310005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3240b5800 con 0x7fd310004d90 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31000bba0 con 0x7fd310003fb0 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31000bd60 con 0x7fd324161340 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd328a06700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd310005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd323fff700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 0x7fd310003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd3237fe700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310003fb0 0x7fd310004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd328a06700 1 -- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 msgr2=0x7fd310003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd328a06700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd324161340 0x7fd310003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd328a06700 1 -- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310003fb0 msgr2=0x7fd310004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:51.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd328a06700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310003fb0 0x7fd310004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.196+0000 7fd328a06700 1 -- 172.21.15.153:0/2771282901 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100035b0 con 0x7fd310004d90 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd328a06700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd310005200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c160 tx=0x7fd310005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedb405bdd538626 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c001db0 con 0x7fd310004d90 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c00cca0 con 0x7fd310004d90 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd322ffd700 1 -- 172.21.15.153:0/2771282901 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c003c40 con 0x7fd310004d90 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd32a469700 1 -- 172.21.15.153:0/2771282901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 msgr2=0x7fd310005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd32a469700 1 --2- 172.21.15.153:0/2771282901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd310005200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c160 tx=0x7fd310005880 comp rx=0 tx=0).stop 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd328a06700 1 -- 172.21.15.153:0/2771282901 reap_dead start 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd32a469700 1 -- 172.21.15.153:0/2771282901 shutdown_connections 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd32a469700 1 -- 172.21.15.153:0/2771282901 >> 172.21.15.153:0/2771282901 conn(0x7fd324072600 msgr2=0x7fd3240cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd32a469700 1 -- 172.21.15.153:0/2771282901 shutdown_connections 2023-11-28T13:01:51.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.197+0000 7fd32a469700 1 -- 172.21.15.153:0/2771282901 wait complete. 2023-11-28T13:01:51.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.199+0000 7fd32a469700 1 Processor -- start 2023-11-28T13:01:51.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.199+0000 7fd32a469700 1 -- start start 2023-11-28T13:01:51.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd32a469700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd32415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd32a469700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd32415fdc0 0x7fd3241e7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd32a469700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3240c8570 0x7fd324156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd32a469700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3241e7dd0 con 0x7fd310004d90 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd32a469700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32415ed20 con 0x7fd3240c8570 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd32a469700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32415f000 con 0x7fd32415fdc0 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd32415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd32415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40962/0 (socket says 172.21.15.153:40962) 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 -- 172.21.15.153:0/1557391347 learned_addr learned my addr 172.21.15.153:0/1557391347 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd328a06700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3240c8570 0x7fd324156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd3237fe700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd32415fdc0 0x7fd3241e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 -- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd32415fdc0 msgr2=0x7fd3241e7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd32415fdc0 0x7fd3241e7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 -- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3240c8570 msgr2=0x7fd324156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3240c8570 0x7fd324156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd323fff700 1 -- 172.21.15.153:0/1557391347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32415cd80 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.200+0000 7fd3237fe700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd32415fdc0 0x7fd3241e7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.201+0000 7fd323fff700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd32415e690 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f60 tx=0x7fd30c007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.201+0000 7fd320ff9700 1 -- 172.21.15.153:0/1557391347 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c00f850 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.201+0000 7fd320ff9700 1 -- 172.21.15.153:0/1557391347 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c00f9d0 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.201+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3240d15e0 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.201+0000 7fd320ff9700 1 -- 172.21.15.153:0/1557391347 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c00fd80 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.201+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32415cf10 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.202+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3240745c0 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.202+0000 7fd320ff9700 1 -- 172.21.15.153:0/1557391347 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd30c01d460 con 0x7fd310004d90 2023-11-28T13:01:51.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.202+0000 7fd320ff9700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3040682f0 0x7fd30406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.203+0000 7fd320ff9700 1 -- 172.21.15.153:0/1557391347 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd30c085bf0 con 0x7fd310004d90 2023-11-28T13:01:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.203+0000 7fd3237fe700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3040682f0 0x7fd30406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.203+0000 7fd3237fe700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3040682f0 0x7fd30406a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd3240b6f90 tx=0x7fd314008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.207+0000 7fd320ff9700 1 -- 172.21.15.153:0/1557391347 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd30c08c080 con 0x7fd310004d90 2023-11-28T13:01:51.429 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.426+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd32406b5f0 con 0x7fd310004d90 2023-11-28T13:01:51.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.427+0000 7fd320ff9700 1 -- 172.21.15.153:0/1557391347 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fd30c0526f0 con 0x7fd310004d90 2023-11-28T13:01:51.430 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:51.430 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/2527007552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2527007552},{"type":"v1","addr":"172.21.15.177:6833","nonce":2527007552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2976651071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2976651071},{"type":"v1","addr":"172.21.15.177:6835","nonce":2976651071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5070,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/814911244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":814911244},{"type":"v1","addr":"172.21.15.153:6836","nonce":814911244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/791392525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":791392525},{"type":"v1","addr":"172.21.15.153:6837","nonce":791392525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-11-28T13:01:51.431 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3040682f0 msgr2=0x7fd30406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.431 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3040682f0 0x7fd30406a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd3240b6f90 tx=0x7fd314008040 comp rx=0 tx=0).stop 2023-11-28T13:01:51.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 msgr2=0x7fd32415e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd32415e690 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd30c000f60 tx=0x7fd30c007690 comp rx=0 tx=0).stop 2023-11-28T13:01:51.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 shutdown_connections 2023-11-28T13:01:51.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3040682f0 0x7fd30406a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310004d90 0x7fd32415e690 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3240c8570 0x7fd324156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 --2- 172.21.15.153:0/1557391347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd32415fdc0 0x7fd3241e7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 >> 172.21.15.153:0/1557391347 conn(0x7fd324072600 msgr2=0x7fd3240b6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:51.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 shutdown_connections 2023-11-28T13:01:51.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.429+0000 7fd32a469700 1 -- 172.21.15.153:0/1557391347 wait complete. 2023-11-28T13:01:51.433 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 18 2023-11-28T13:01:51.442 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:01:51.442 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:01:51.443 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:01:51.443 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:51.443 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:01:51.443 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:01:51.444 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:01:51.444 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:51.444 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:01:51.444 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:01:51.445 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:01:51.445 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:51.445 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:01:51.445 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:01:51.445 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:01:51.445 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:51.452 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:01:51.452 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:01:51.453 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:01:51.453 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:01:51.457 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:01:51.457 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:01:51.457 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:01:51.457 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:01:51.495 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:01:51.495 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:51.496 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:01:51.496 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:51.499 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:01:51.499 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:51.500 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:01:51.500 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:51.500 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:01:51.576 INFO:tasks.ceph.mds.a.smithi153.stdout:starting mds.a at 2023-11-28T13:01:51.576 INFO:tasks.ceph.mds.c.smithi153.stdout:starting mds.c at 2023-11-28T13:01:51.590 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T13:01:51.591 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T13:01:51.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.630+0000 7f100f69e700 1 Processor -- start 2023-11-28T13:01:51.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.630+0000 7f100f69e700 1 -- start start 2023-11-28T13:01:51.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100f69e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f10080d01b0 0x7f10080d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100f69e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10080b16d0 0x7f10080d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100f69e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10080b0230 con 0x7f10080b20b0 2023-11-28T13:01:51.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100f69e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100815cec0 con 0x7f10080b16d0 2023-11-28T13:01:51.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100f69e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10080b4570 con 0x7f10080d01b0 2023-11-28T13:01:51.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100dc3b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10080b16d0 0x7f10080d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100dc3b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10080b16d0 0x7f10080d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40998/0 (socket says 172.21.15.153:40998) 2023-11-28T13:01:51.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100dc3b700 1 -- 172.21.15.153:0/4265901349 learned_addr learned my addr 172.21.15.153:0/4265901349 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:51.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100d43a700 1 --2- 172.21.15.153:0/4265901349 >> v2:172.21.15.177:3300/0 conn(0x7f10080d01b0 0x7f10080d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100dc3b700 1 -- 172.21.15.153:0/4265901349 >> v2:172.21.15.177:3300/0 conn(0x7f10080d01b0 msgr2=0x7f10080d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:51.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100dc3b700 1 --2- 172.21.15.153:0/4265901349 >> v2:172.21.15.177:3300/0 conn(0x7f10080d01b0 0x7f10080d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100dc3b700 1 -- 172.21.15.153:0/4265901349 >> v1:172.21.15.177:6789/0 conn(0x7f10080b20b0 legacy=0x7f10080b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:51.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100dc3b700 1 -- 172.21.15.153:0/4265901349 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10080b4850 con 0x7f10080b16d0 2023-11-28T13:01:51.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.631+0000 7f100d43a700 1 --2- 172.21.15.153:0/4265901349 >> v2:172.21.15.177:3300/0 conn(0x7f10080d01b0 0x7f10080d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:51.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100dc3b700 1 --2- 172.21.15.153:0/4265901349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b16d0 0x7f10080d1160 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f10080b5bb0 tx=0x7f100000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0878950e281fac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:51.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f0ffffff700 1 -- 172.21.15.153:0/4265901349 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000e070 con 0x7f10080b16d0 2023-11-28T13:01:51.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f0ffffff700 1 -- 172.21.15.153:0/4265901349 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000003a20 con 0x7f10080b16d0 2023-11-28T13:01:51.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f0ffffff700 1 -- 172.21.15.153:0/4265901349 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000003dd0 con 0x7f10080b16d0 2023-11-28T13:01:51.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 -- 172.21.15.153:0/4265901349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b16d0 msgr2=0x7f10080d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 --2- 172.21.15.153:0/4265901349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b16d0 0x7f10080d1160 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f10080b5bb0 tx=0x7f100000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:51.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 -- 172.21.15.153:0/4265901349 shutdown_connections 2023-11-28T13:01:51.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 --2- 172.21.15.153:0/4265901349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b16d0 0x7f10080d1160 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 --2- 172.21.15.153:0/4265901349 >> v2:172.21.15.177:3300/0 conn(0x7f10080d01b0 0x7f10080d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 -- 172.21.15.153:0/4265901349 >> 172.21.15.153:0/4265901349 conn(0x7f100809c3f0 msgr2=0x7f10080d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:51.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 -- 172.21.15.153:0/4265901349 shutdown_connections 2023-11-28T13:01:51.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.632+0000 7f100f69e700 1 -- 172.21.15.153:0/4265901349 wait complete. 2023-11-28T13:01:51.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.633+0000 7f100f69e700 1 Processor -- start 2023-11-28T13:01:51.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.633+0000 7f100f69e700 1 -- start start 2023-11-28T13:01:51.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100f69e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10080b16d0 0x7f10080cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100f69e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b20b0 0x7f100809ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100f69e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10080d01b0 0x7f100806b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100f69e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100814eba0 con 0x7f10080b20b0 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100f69e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10080ca120 con 0x7f10080b16d0 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100f69e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10080cb670 con 0x7f10080d01b0 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100cc39700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b20b0 0x7f100809ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100cc39700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b20b0 0x7f100809ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41002/0 (socket says 172.21.15.153:41002) 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.634+0000 7f100cc39700 1 -- 172.21.15.153:0/4121313903 learned_addr learned my addr 172.21.15.153:0/4121313903 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100d43a700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10080b16d0 0x7f10080cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100dc3b700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10080d01b0 0x7f100806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.643 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100cc39700 1 -- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10080d01b0 msgr2=0x7f100806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100cc39700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10080d01b0 0x7f100806b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100cc39700 1 -- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10080b16d0 msgr2=0x7f10080cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100cc39700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10080b16d0 0x7f10080cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100cc39700 1 -- 172.21.15.153:0/4121313903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000009b30 con 0x7f10080b20b0 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100d43a700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10080b16d0 0x7f10080cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100cc39700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b20b0 0x7f100809ee90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f100400c4d0 tx=0x7f100400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f0ffdffb700 1 -- 172.21.15.153:0/4121313903 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004011790 con 0x7f10080b20b0 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f0ffdffb700 1 -- 172.21.15.153:0/4121313903 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10040118f0 con 0x7f10080b20b0 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f0ffdffb700 1 -- 172.21.15.153:0/4121313903 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004011ca0 con 0x7f10080b20b0 2023-11-28T13:01:51.644 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10080afec0 con 0x7f10080b20b0 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.635+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10080afbe0 con 0x7f10080b20b0 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.636+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10080b2600 con 0x7f10080b20b0 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.636+0000 7f0ffdffb700 1 -- 172.21.15.153:0/4121313903 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1004010050 con 0x7f10080b20b0 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.637+0000 7f0ffdffb700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fec0682f0 0x7f0fec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.637+0000 7f0ffdffb700 1 -- 172.21.15.153:0/4121313903 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1004085710 con 0x7f10080b20b0 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.637+0000 7f100d43a700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fec0682f0 0x7f0fec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.637+0000 7f100d43a700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fec0682f0 0x7f0fec06a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8008910 tx=0x7f0ff8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:51.645 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.641+0000 7f0ffdffb700 1 -- 172.21.15.153:0/4121313903 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1004052270 con 0x7f10080b20b0 2023-11-28T13:01:51.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:51.837+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1008068360 con 0x7f10080b20b0 2023-11-28T13:01:52.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.628+0000 7f0ffdffb700 1 -- 172.21.15.153:0/4121313903 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1004055890 con 0x7f10080b20b0 2023-11-28T13:01:52.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fec0682f0 msgr2=0x7f0fec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fec0682f0 0x7f0fec06a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8008910 tx=0x7f0ff8008040 comp rx=0 tx=0).stop 2023-11-28T13:01:52.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b20b0 msgr2=0x7f100809ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b20b0 0x7f100809ee90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f100400c4d0 tx=0x7f100400c890 comp rx=0 tx=0).stop 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 shutdown_connections 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10080d01b0 0x7f100806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10080b16d0 0x7f10080cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fec0682f0 0x7f0fec06a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 --2- 172.21.15.153:0/4121313903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10080b20b0 0x7f100809ee90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 >> 172.21.15.153:0/4121313903 conn(0x7f100809c3f0 msgr2=0x7f10080b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 shutdown_connections 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.631+0000 7f100f69e700 1 -- 172.21.15.153:0/4121313903 wait complete. 2023-11-28T13:01:52.633 INFO:teuthology.orchestra.run.smithi153.stderr: removed all blocklist entries 2023-11-28T13:01:52.645 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.766+0000 7fb8219c7700 1 Processor -- start 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.767+0000 7fb8219c7700 1 -- start start 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.767+0000 7fb8219c7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb81c159860 0x7fb81c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.767+0000 7fb8219c7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0b0d80 0x7fb81c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.767+0000 7fb8219c7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c0ccc00 con 0x7fb81c159490 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.767+0000 7fb8219c7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c0ccec0 con 0x7fb81c159860 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.767+0000 7fb8219c7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0cd1a0 con 0x7fb81c0b0d80 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb81c159860 0x7fb81c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81b7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0b0d80 0x7fb81c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb81c159860 0x7fb81c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:50480/0 (socket says 172.21.15.153:50480) 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 -- 172.21.15.153:0/1556416148 learned_addr learned my addr 172.21.15.153:0/1556416148 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81affd700 1 --1- 172.21.15.153:0/1556416148 >> v1:172.21.15.177:6789/0 conn(0x7fb81c159490 0x7fb81c0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:35790/0 (socket says 172.21.15.153:35790) 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb819ffb700 1 -- 172.21.15.153:0/1556416148 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979392904 0 0) 0x7fb81c0ccc00 con 0x7fb81c159490 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb819ffb700 1 -- 172.21.15.153:0/1556416148 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8040035b0 con 0x7fb81c159490 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 -- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0b0d80 msgr2=0x7fb81c0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 --2- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0b0d80 0x7fb81c0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 -- 172.21.15.153:0/1556416148 >> v1:172.21.15.177:6789/0 conn(0x7fb81c159490 legacy=0x7fb81c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 -- 172.21.15.153:0/1556416148 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0cd480 con 0x7fb81c159860 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81b7fe700 1 --2- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0b0d80 0x7fb81c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb81a7fc700 1 --2- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159860 0x7fb81c0c94c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098f0 tx=0x7fb80c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bee158167d3da3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb819ffb700 1 -- 172.21.15.153:0/1556416148 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c011830 con 0x7fb81c159860 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb819ffb700 1 -- 172.21.15.153:0/1556416148 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c011990 con 0x7fb81c159860 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.768+0000 7fb819ffb700 1 -- 172.21.15.153:0/1556416148 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c011d00 con 0x7fb81c159860 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 -- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159860 msgr2=0x7fb81c0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 --2- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159860 0x7fb81c0c94c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098f0 tx=0x7fb80c009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 -- 172.21.15.153:0/1556416148 shutdown_connections 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 --2- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0b0d80 0x7fb81c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 --2- 172.21.15.153:0/1556416148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159860 0x7fb81c0c94c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 -- 172.21.15.153:0/1556416148 >> 172.21.15.153:0/1556416148 conn(0x7fb81c0cb670 msgr2=0x7fb81c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 -- 172.21.15.153:0/1556416148 shutdown_connections 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.769+0000 7fb8219c7700 1 -- 172.21.15.153:0/1556416148 wait complete. 2023-11-28T13:01:52.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.770+0000 7fb8219c7700 1 Processor -- start 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.770+0000 7fb8219c7700 1 -- start start 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb8219c7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81c0b0d80 0x7fb81c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb8219c7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159490 0x7fb81c0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb8219c7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0ac930 0x7fb81c0c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb8219c7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c157bb0 con 0x7fb81c0b0d80 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb8219c7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1f2500 con 0x7fb81c159490 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb8219c7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c156f30 con 0x7fb81c0ac930 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb81affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81c0b0d80 0x7fb81c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb81affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81c0b0d80 0x7fb81c0712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41018/0 (socket says 172.21.15.153:41018) 2023-11-28T13:01:52.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb81affd700 1 -- 172.21.15.153:0/2375068672 learned_addr learned my addr 172.21.15.153:0/2375068672 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb81a7fc700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159490 0x7fb81c0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.771+0000 7fb81b7fe700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0ac930 0x7fb81c0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb81affd700 1 -- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0ac930 msgr2=0x7fb81c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb81affd700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0ac930 0x7fb81c0c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb81affd700 1 -- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159490 msgr2=0x7fb81c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb81affd700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159490 0x7fb81c0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb81affd700 1 -- 172.21.15.153:0/2375068672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb814009a70 con 0x7fb81c0b0d80 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb81affd700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81c0b0d80 0x7fb81c0712c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb81000c500 tx=0x7fb81000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb803fff700 1 -- 172.21.15.153:0/2375068672 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8100064e0 con 0x7fb81c0b0d80 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb803fff700 1 -- 172.21.15.153:0/2375068672 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb810006640 con 0x7fb81c0b0d80 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c009550 con 0x7fb81c0b0d80 2023-11-28T13:01:52.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb803fff700 1 -- 172.21.15.153:0/2375068672 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810004a90 con 0x7fb81c0b0d80 2023-11-28T13:01:52.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.772+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c0c9f40 con 0x7fb81c0b0d80 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.773+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c0cd760 con 0x7fb81c0b0d80 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.773+0000 7fb803fff700 1 -- 172.21.15.153:0/2375068672 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb810019030 con 0x7fb81c0b0d80 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.774+0000 7fb803fff700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8080682f0 0x7fb80806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.774+0000 7fb803fff700 1 -- 172.21.15.153:0/2375068672 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb810084cf0 con 0x7fb81c0b0d80 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.774+0000 7fb81a7fc700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8080682f0 0x7fb80806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.774+0000 7fb81a7fc700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8080682f0 0x7fb80806a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb80c014040 tx=0x7fb80c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:52.778+0000 7fb803fff700 1 -- 172.21.15.153:0/2375068672 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb810051820 con 0x7fb81c0b0d80 2023-11-28T13:01:53.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.032+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb81c0cdc80 con 0x7fb81c0b0d80 2023-11-28T13:01:53.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.033+0000 7fb803fff700 1 -- 172.21.15.153:0/2375068672 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fb810054e40 con 0x7fb81c0b0d80 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "auth_dump": [ 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.a", 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlYBAHIBAA4jaMav4CAatojxXY6AkGPw==", 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.034 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.b", 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVl9vBaLBAAD84m+iSSScY6ETZRBpBUEg==", 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:53.035 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.c", 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlG5hJJxAAtFV9HSiLJClNn3nmB7ORSg==", 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.036 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.d", 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlvAC6MxAAX0N4lN+NKADdjjjROw+Zbw==", 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.037 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.0", 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAq5GVlatZbJhAA2RQenckwVd7P0kBpzW08Lg==", 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.1", 2023-11-28T13:01:53.038 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAt5GVlf73BJRAA51B4aDKKKVO+yo0+jQ20Yw==", 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.2", 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAw5GVl3M9qJRAAa2Jhr7Cp6vN/fRKJpMzCOg==", 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.039 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.3", 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAz5GVlWVgHJRAAlP6Gc1QZkzufchXRrkLkbA==", 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.040 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.4", 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA55GVlLb5hJRAAMMymIqHNw8oeVW8s3BrtIg==", 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.5", 2023-11-28T13:01:53.041 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA85GVlwsLjJBAAb/1bWARS8ILpFdBGemlcoA==", 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.6", 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA/5GVlwXx5JBAAXOS8JqPaXyQ7sj+2t9tsPQ==", 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.042 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.7", 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBC5GVl5q+qJBAAMc7ZUInNCMJmf0lC22s1QQ==", 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.0", 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlR2IBOBAAJBEAQ3PnTN6KaU44zoHW7Q==", 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow r", 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.1", 2023-11-28T13:01:53.044 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAn5GVlTvYJAxAA68QZff0NcXXeMV7aGZoBfQ==", 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.2", 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlsT5YOxAAZ6ep21AlkI2znaE9wyskxA==", 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.045 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.3", 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAn5GVlxW63CBAApiPmiN4tN3S2dZH/bCKS/Q==", 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:01:53.046 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.admin", 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlE1RYARAA5UHDpYgXbOEuWdAhlYnmbA==", 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.047 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mgr.x", 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlKLmuHBAANErUDgd4g3Yysqk/sPAbrA==", 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile mgr", 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:53.048 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mgr.y", 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVld7eyFhAAafZalFXR/4jRd5fUct7T2Q==", 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile mgr", 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2023-11-28T13:01:53.049 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8080682f0 msgr2=0x7fb80806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8080682f0 0x7fb80806a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb80c014040 tx=0x7fb80c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81c0b0d80 msgr2=0x7fb81c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81c0b0d80 0x7fb81c0712c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb81000c500 tx=0x7fb81000c810 comp rx=0 tx=0).stop 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 shutdown_connections 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb8080682f0 0x7fb80806a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb81c0b0d80 0x7fb81c0712c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb81c0ac930 0x7fb81c0c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 --2- 172.21.15.153:0/2375068672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb81c159490 0x7fb81c0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 >> 172.21.15.153:0/2375068672 conn(0x7fb81c0cb670 msgr2=0x7fb81c15d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 shutdown_connections 2023-11-28T13:01:53.052 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.035+0000 7fb8219c7700 1 -- 172.21.15.153:0/2375068672 wait complete. 2023-11-28T13:01:53.052 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:53.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.173+0000 7fabbb3d1700 1 Processor -- start 2023-11-28T13:01:53.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.173+0000 7fabbb3d1700 1 -- start start 2023-11-28T13:01:53.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabbb3d1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabbb3d1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabb406dee0 0x7fabb406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabbb3d1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb4071950 con 0x7fabb406cdc0 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabbb3d1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb4161550 con 0x7fabb406dee0 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabbb3d1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415d060 con 0x7fabb4071040 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabb406dee0 0x7fabb406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fabb406dee0 0x7fabb406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41026/0 (socket says 172.21.15.153:41026) 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 -- 172.21.15.153:0/210667616 learned_addr learned my addr 172.21.15.153:0/210667616 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb916d700 1 --2- 172.21.15.153:0/210667616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 -- 172.21.15.153:0/210667616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 msgr2=0x7fabb4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 --2- 172.21.15.153:0/210667616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 -- 172.21.15.153:0/210667616 >> v1:172.21.15.177:6789/0 conn(0x7fabb406cdc0 legacy=0x7fabb406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 -- 172.21.15.153:0/210667616 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb415d340 con 0x7fabb406dee0 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fabb896c700 1 --2- 172.21.15.153:0/210667616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb406e350 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faba40098f0 tx=0x7faba4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d107de537374b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.174+0000 7fababfff700 1 -- 172.21.15.153:0/210667616 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba400b8a0 con 0x7fabb406dee0 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fababfff700 1 -- 172.21.15.153:0/210667616 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba400ba00 con 0x7fabb406dee0 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fababfff700 1 -- 172.21.15.153:0/210667616 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba400bd90 con 0x7fabb406dee0 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 -- 172.21.15.153:0/210667616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 msgr2=0x7fabb406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/210667616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb406e350 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faba40098f0 tx=0x7faba4009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 -- 172.21.15.153:0/210667616 shutdown_connections 2023-11-28T13:01:53.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/210667616 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/210667616 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb406e350 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 -- 172.21.15.153:0/210667616 >> 172.21.15.153:0/210667616 conn(0x7fabb40d0000 msgr2=0x7fabb40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 -- 172.21.15.153:0/210667616 shutdown_connections 2023-11-28T13:01:53.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.175+0000 7fabbb3d1700 1 -- 172.21.15.153:0/210667616 wait complete. 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.176+0000 7fabbb3d1700 1 Processor -- start 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.176+0000 7fabbb3d1700 1 -- start start 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabbb3d1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabb406cdc0 0x7fabb40c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabbb3d1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb40ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabbb3d1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabbb3d1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4147950 con 0x7fabb406dee0 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabbb3d1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40ad930 con 0x7fabb406cdc0 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabbb3d1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40adc10 con 0x7fabb4071040 2023-11-28T13:01:53.182 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb40ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb40ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41034/0 (socket says 172.21.15.153:41034) 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 -- 172.21.15.153:0/708900695 learned_addr learned my addr 172.21.15.153:0/708900695 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb916d700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabb406cdc0 0x7fabb40c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb996e700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 -- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 msgr2=0x7fabb4147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 -- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabb406cdc0 msgr2=0x7fabb40c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabb406cdc0 0x7fabb40c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb896c700 1 -- 172.21.15.153:0/708900695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4009550 con 0x7fabb406dee0 2023-11-28T13:01:53.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.177+0000 7fabb996e700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.178+0000 7fabb896c700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb40ad230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faba4014040 tx=0x7faba4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.178+0000 7faba9ffb700 1 -- 172.21.15.153:0/708900695 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4004590 con 0x7fabb406dee0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.178+0000 7faba9ffb700 1 -- 172.21.15.153:0/708900695 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba401e070 con 0x7fabb406dee0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.178+0000 7faba9ffb700 1 -- 172.21.15.153:0/708900695 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba40038e0 con 0x7fabb406dee0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.178+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb415d900 con 0x7fabb406dee0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.178+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb415d620 con 0x7fabb406dee0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.179+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb406e890 con 0x7fabb406dee0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.179+0000 7faba9ffb700 1 -- 172.21.15.153:0/708900695 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faba400f050 con 0x7fabb406dee0 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.180+0000 7faba9ffb700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab980682f0 0x7fab9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.180+0000 7fabb916d700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab980682f0 0x7fab9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.180+0000 7faba9ffb700 1 -- 172.21.15.153:0/708900695 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faba4020070 con 0x7fabb406dee0 2023-11-28T13:01:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.180+0000 7fabb916d700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab980682f0 0x7fab9806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006b20 tx=0x7fabb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.184+0000 7faba9ffb700 1 -- 172.21.15.153:0/708900695 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faba408c020 con 0x7fabb406dee0 2023-11-28T13:01:53.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.365+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabb40c6ad0 con 0x7fabb406dee0 2023-11-28T13:01:53.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.366+0000 7faba9ffb700 1 -- 172.21.15.153:0/708900695 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14315 (secure 0 0 0) 0x7faba4053970 con 0x7fabb406dee0 2023-11-28T13:01:53.367 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:53.367 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":47,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:52.623923+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:53.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.368+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab980682f0 msgr2=0x7fab9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.368+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab980682f0 0x7fab9806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006b20 tx=0x7fabb0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.368+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 msgr2=0x7fabb40ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.368+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb40ad230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faba4014040 tx=0x7faba4004750 comp rx=0 tx=0).stop 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 shutdown_connections 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabb4071040 0x7fabb4147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabb406cdc0 0x7fabb40c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fab980682f0 0x7fab9806a7a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 --2- 172.21.15.153:0/708900695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabb406dee0 0x7fabb40ad230 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 >> 172.21.15.153:0/708900695 conn(0x7fabb40d0000 msgr2=0x7fabb40cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 shutdown_connections 2023-11-28T13:01:53.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.369+0000 7fabbb3d1700 1 -- 172.21.15.153:0/708900695 wait complete. 2023-11-28T13:01:53.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:01:53.382 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:53.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.505+0000 7f79da97c700 1 Processor -- start 2023-11-28T13:01:53.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.505+0000 7f79da97c700 1 -- start start 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.505+0000 7f79da97c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f79d4071040 0x7f79d4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79da97c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d4071950 con 0x7f79d4160540 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79da97c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d415d060 con 0x7f79d406dee0 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79da97c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d415d340 con 0x7f79d4071040 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f79d4071040 0x7f79d4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d8f19700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f79d4160540 0x7f79d4160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59206/0 (socket says 172.21.15.153:59206) 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d8f19700 1 -- 172.21.15.153:0/1171072641 learned_addr learned my addr 172.21.15.153:0/1171072641 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d3fff700 1 -- 172.21.15.153:0/1171072641 >> v1:172.21.15.177:6789/0 conn(0x7f79d406dee0 legacy=0x7f79d406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.508 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d3fff700 1 -- 172.21.15.153:0/1171072641 >> v1:172.21.15.153:6789/0 conn(0x7f79d4160540 legacy=0x7f79d4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d3fff700 1 -- 172.21.15.153:0/1171072641 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d415d620 con 0x7f79d4071040 2023-11-28T13:01:53.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d3fff700 1 --2- 172.21.15.153:0/1171072641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4071040 0x7f79d4071410 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009840 tx=0x7f79c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852964ecac9e85ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d2ffd700 1 -- 172.21.15.153:0/1171072641 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4004230 con 0x7f79d4071040 2023-11-28T13:01:53.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d2ffd700 1 -- 172.21.15.153:0/1171072641 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c4004390 con 0x7f79d4071040 2023-11-28T13:01:53.509 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.506+0000 7f79d2ffd700 1 -- 172.21.15.153:0/1171072641 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c40036a0 con 0x7f79d4071040 2023-11-28T13:01:53.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.507+0000 7f79da97c700 1 -- 172.21.15.153:0/1171072641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4071040 msgr2=0x7f79d4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.507+0000 7f79da97c700 1 --2- 172.21.15.153:0/1171072641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4071040 0x7f79d4071410 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009840 tx=0x7f79c4009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:53.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.507+0000 7f79da97c700 1 -- 172.21.15.153:0/1171072641 shutdown_connections 2023-11-28T13:01:53.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.507+0000 7f79da97c700 1 --2- 172.21.15.153:0/1171072641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d4071040 0x7f79d4071410 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.507+0000 7f79da97c700 1 -- 172.21.15.153:0/1171072641 >> 172.21.15.153:0/1171072641 conn(0x7f79d40d0000 msgr2=0x7f79d40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.507+0000 7f79da97c700 1 -- 172.21.15.153:0/1171072641 shutdown_connections 2023-11-28T13:01:53.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.507+0000 7f79da97c700 1 -- 172.21.15.153:0/1171072641 wait complete. 2023-11-28T13:01:53.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.508+0000 7f79da97c700 1 Processor -- start 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.508+0000 7f79da97c700 1 -- start start 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79da97c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d406dee0 0x7f79d40c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79da97c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d4071040 0x7f79d40ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79da97c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d4160540 0x7f79d40ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79da97c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40c54e0 con 0x7f79d406dee0 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79da97c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40c5780 con 0x7f79d4160540 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79da97c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40c5a60 con 0x7f79d4071040 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d406dee0 0x7f79d40c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d406dee0 0x7f79d40c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41044/0 (socket says 172.21.15.153:41044) 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d3fff700 1 -- 172.21.15.153:0/3662281754 learned_addr learned my addr 172.21.15.153:0/3662281754 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d8f19700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d4160540 0x7f79d40ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d3fff700 1 -- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d4071040 msgr2=0x7f79d40ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d37fe700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d4071040 0x7f79d40ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d3fff700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d4071040 0x7f79d40ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d3fff700 1 -- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d4160540 msgr2=0x7f79d40ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.509+0000 7f79d3fff700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d4160540 0x7f79d40ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79d3fff700 1 -- 172.21.15.153:0/3662281754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40094a0 con 0x7f79d406dee0 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79d8f19700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d4160540 0x7f79d40ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79d37fe700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d4071040 0x7f79d40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79d3fff700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d406dee0 0x7f79d40c4fa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009b90 tx=0x7f79c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79d0ff9700 1 -- 172.21.15.153:0/3662281754 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4011630 con 0x7f79d406dee0 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79d0ff9700 1 -- 172.21.15.153:0/3662281754 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c4011790 con 0x7f79d406dee0 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d415ccd0 con 0x7f79d406dee0 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79d0ff9700 1 -- 172.21.15.153:0/3662281754 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4011b40 con 0x7f79d406dee0 2023-11-28T13:01:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.510+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d415d900 con 0x7f79d406dee0 2023-11-28T13:01:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.511+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d406a270 con 0x7f79d406dee0 2023-11-28T13:01:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.511+0000 7f79d0ff9700 1 -- 172.21.15.153:0/3662281754 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f79c4017020 con 0x7f79d406dee0 2023-11-28T13:01:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.511+0000 7f79d0ff9700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f79c00682f0 0x7f79c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.512+0000 7f79d0ff9700 1 -- 172.21.15.153:0/3662281754 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f79c4087130 con 0x7f79d406dee0 2023-11-28T13:01:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.512+0000 7f79d37fe700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f79c00682f0 0x7f79c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.512+0000 7f79d37fe700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f79c00682f0 0x7f79c006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79c8006fd0 tx=0x7f79c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.516+0000 7f79d0ff9700 1 -- 172.21.15.153:0/3662281754 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f79c4053c60 con 0x7f79d406dee0 2023-11-28T13:01:53.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.699+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79d406ed40 con 0x7f79d406dee0 2023-11-28T13:01:53.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.699+0000 7f79d0ff9700 1 -- 172.21.15.153:0/3662281754 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79c4057280 con 0x7f79d406dee0 2023-11-28T13:01:53.700 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f79c00682f0 msgr2=0x7f79c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f79c00682f0 0x7f79c006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79c8006fd0 tx=0x7f79c8008040 comp rx=0 tx=0).stop 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d406dee0 msgr2=0x7f79d40c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d406dee0 0x7f79d40c4fa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009b90 tx=0x7f79c40047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 shutdown_connections 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f79c00682f0 0x7f79c006a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79d406dee0 0x7f79d40c4fa0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79d4160540 0x7f79d40ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.701+0000 7f79da97c700 1 --2- 172.21.15.153:0/3662281754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79d4071040 0x7f79d40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.702+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 >> 172.21.15.153:0/3662281754 conn(0x7f79d40d0000 msgr2=0x7f79d415f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.702+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 shutdown_connections 2023-11-28T13:01:53.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.702+0000 7f79da97c700 1 -- 172.21.15.153:0/3662281754 wait complete. 2023-11-28T13:01:53.714 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:53.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.837+0000 7f91f5a16700 1 Processor -- start 2023-11-28T13:01:53.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.837+0000 7f91f5a16700 1 -- start start 2023-11-28T13:01:53.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91f5a16700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91f015c9b0 0x7f91f00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91f5a16700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f91f0161340 0x7f91f00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91f5a16700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f00d1950 con 0x7f91f00d2320 2023-11-28T13:01:53.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91f5a16700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00b5800 con 0x7f91f015c9b0 2023-11-28T13:01:53.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91f5a16700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00b5520 con 0x7f91f0161340 2023-11-28T13:01:53.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91eeffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f91f015c9b0 0x7f91f00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91ef7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f91f00d2320 0x7f91f00cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59220/0 (socket says 172.21.15.153:59220) 2023-11-28T13:01:53.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91ef7fe700 1 -- 172.21.15.153:0/3772461902 learned_addr learned my addr 172.21.15.153:0/3772461902 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:53.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91ee7fc700 1 --2- 172.21.15.153:0/3772461902 >> v2:172.21.15.177:3300/0 conn(0x7f91f0161340 0x7f91f00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91eeffd700 1 -- 172.21.15.153:0/3772461902 >> v2:172.21.15.177:3300/0 conn(0x7f91f0161340 msgr2=0x7f91f00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91eeffd700 1 --2- 172.21.15.153:0/3772461902 >> v2:172.21.15.177:3300/0 conn(0x7f91f0161340 0x7f91f00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91eeffd700 1 -- 172.21.15.153:0/3772461902 >> v1:172.21.15.153:6789/0 conn(0x7f91f00d2320 legacy=0x7f91f00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91eeffd700 1 -- 172.21.15.153:0/3772461902 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00d1300 con 0x7f91f015c9b0 2023-11-28T13:01:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91edffb700 1 -- 172.21.15.153:0/3772461902 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188305546 0 0) 0x7f91f00d1950 con 0x7f91f00d2320 2023-11-28T13:01:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.838+0000 7f91ee7fc700 1 --2- 172.21.15.153:0/3772461902 >> v2:172.21.15.177:3300/0 conn(0x7f91f0161340 0x7f91f00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91eeffd700 1 --2- 172.21.15.153:0/3772461902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f015c9b0 0x7f91f00b4bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f91f00685c0 tx=0x7f91e4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe5c32ceb4c2d08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91edffb700 1 -- 172.21.15.153:0/3772461902 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4011830 con 0x7f91f015c9b0 2023-11-28T13:01:53.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91edffb700 1 -- 172.21.15.153:0/3772461902 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e4011990 con 0x7f91f015c9b0 2023-11-28T13:01:53.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91edffb700 1 -- 172.21.15.153:0/3772461902 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4011d20 con 0x7f91f015c9b0 2023-11-28T13:01:53.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 -- 172.21.15.153:0/3772461902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f015c9b0 msgr2=0x7f91f00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 --2- 172.21.15.153:0/3772461902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f015c9b0 0x7f91f00b4bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f91f00685c0 tx=0x7f91e4009b20 comp rx=0 tx=0).stop 2023-11-28T13:01:53.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 -- 172.21.15.153:0/3772461902 shutdown_connections 2023-11-28T13:01:53.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 --2- 172.21.15.153:0/3772461902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f015c9b0 0x7f91f00b4bf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 --2- 172.21.15.153:0/3772461902 >> v2:172.21.15.177:3300/0 conn(0x7f91f0161340 0x7f91f00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 -- 172.21.15.153:0/3772461902 >> 172.21.15.153:0/3772461902 conn(0x7f91f0072600 msgr2=0x7f91f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 -- 172.21.15.153:0/3772461902 shutdown_connections 2023-11-28T13:01:53.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.839+0000 7f91f5a16700 1 -- 172.21.15.153:0/3772461902 wait complete. 2023-11-28T13:01:53.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.841+0000 7f91f5a16700 1 Processor -- start 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.841+0000 7f91f5a16700 1 -- start start 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91f5a16700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91f00d2320 0x7f91f0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91f5a16700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91f015c9b0 0x7f91f00c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91f5a16700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0161340 0x7f91f0154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91f5a16700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0156bc0 con 0x7f91f0161340 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91f5a16700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01555e0 con 0x7f91f00d2320 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91f5a16700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f01558c0 con 0x7f91f015c9b0 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91ef7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0161340 0x7f91f0154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91ef7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0161340 0x7f91f0154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41062/0 (socket says 172.21.15.153:41062) 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91ef7fe700 1 -- 172.21.15.153:0/2900656318 learned_addr learned my addr 172.21.15.153:0/2900656318 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91eeffd700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91f00d2320 0x7f91f0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91ee7fc700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91f015c9b0 0x7f91f00c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.842+0000 7f91ef7fe700 1 -- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91f015c9b0 msgr2=0x7f91f00c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91ef7fe700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91f015c9b0 0x7f91f00c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91ef7fe700 1 -- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91f00d2320 msgr2=0x7f91f0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91ef7fe700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91f00d2320 0x7f91f0156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91ef7fe700 1 -- 172.21.15.153:0/2900656318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40094a0 con 0x7f91f0161340 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91ef7fe700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0161340 0x7f91f0154c00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91e800cb30 tx=0x7f91e800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91d7fff700 1 -- 172.21.15.153:0/2900656318 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800ab50 con 0x7f91f0161340 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91d7fff700 1 -- 172.21.15.153:0/2900656318 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e800acb0 con 0x7f91f0161340 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f00b5130 con 0x7f91f0161340 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91d7fff700 1 -- 172.21.15.153:0/2900656318 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8007310 con 0x7f91f0161340 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.843+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f00d15e0 con 0x7f91f0161340 2023-11-28T13:01:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.844+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f00b52c0 con 0x7f91f0161340 2023-11-28T13:01:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.844+0000 7f91d7fff700 1 -- 172.21.15.153:0/2900656318 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91e8010070 con 0x7f91f0161340 2023-11-28T13:01:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.845+0000 7f91d7fff700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91dc0682f0 0x7f91dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.845+0000 7f91d7fff700 1 -- 172.21.15.153:0/2900656318 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f91e80852a0 con 0x7f91f0161340 2023-11-28T13:01:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.845+0000 7f91eeffd700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91dc0682f0 0x7f91dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.845+0000 7f91eeffd700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91dc0682f0 0x7f91dc06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91e40090b0 tx=0x7f91e4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.854 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:53.849+0000 7f91d7fff700 1 -- 172.21.15.153:0/2900656318 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91e8051d70 con 0x7f91f0161340 2023-11-28T13:01:54.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.030+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f91f0066320 con 0x7f91f0161340 2023-11-28T13:01:54.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.030+0000 7f91d7fff700 1 -- 172.21.15.153:0/2900656318 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7f91e8018020 con 0x7f91f0161340 2023-11-28T13:01:54.031 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:01:54.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91dc0682f0 msgr2=0x7f91dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91dc0682f0 0x7f91dc06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91e40090b0 tx=0x7f91e4008030 comp rx=0 tx=0).stop 2023-11-28T13:01:54.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0161340 msgr2=0x7f91f0154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0161340 0x7f91f0154c00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91e800cb30 tx=0x7f91e800cef0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 shutdown_connections 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91dc0682f0 0x7f91dc06a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91f0161340 0x7f91f0154c00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91f00d2320 0x7f91f0156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 --2- 172.21.15.153:0/2900656318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91f015c9b0 0x7f91f00c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.032+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 >> 172.21.15.153:0/2900656318 conn(0x7f91f0072600 msgr2=0x7f91f00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.033+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 shutdown_connections 2023-11-28T13:01:54.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.033+0000 7f91f5a16700 1 -- 172.21.15.153:0/2900656318 wait complete. 2023-11-28T13:01:54.045 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.165+0000 7f20653ca700 1 Processor -- start 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.165+0000 7f20653ca700 1 -- start start 2023-11-28T13:01:54.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.166+0000 7f20653ca700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 0x7f20600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.166+0000 7f20653ca700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f20600d2320 0x7f20600cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.166+0000 7f20653ca700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600d1950 con 0x7f206015c9b0 2023-11-28T13:01:54.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.166+0000 7f20653ca700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20600b5800 con 0x7f20600d2320 2023-11-28T13:01:54.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.166+0000 7f20653ca700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20600b5300 con 0x7f2060161340 2023-11-28T13:01:54.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.166+0000 7f205effd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f206015c9b0 0x7f20600b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59230/0 (socket says 172.21.15.153:59230) 2023-11-28T13:01:54.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.166+0000 7f205effd700 1 -- 172.21.15.153:0/3952905948 learned_addr learned my addr 172.21.15.153:0/3952905948 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205f7fe700 1 --2- 172.21.15.153:0/3952905948 >> v2:172.21.15.177:3300/0 conn(0x7f20600d2320 0x7f20600cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205e7fc700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 0x7f20600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504642389 0 0) 0x7f20600d1950 con 0x7f206015c9b0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20480035b0 con 0x7f206015c9b0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517012115 0 0) 0x7f20480035b0 con 0x7f206015c9b0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 msgr2=0x7f20600d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 0x7f20600d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 >> v2:172.21.15.177:3300/0 conn(0x7f20600d2320 msgr2=0x7f20600cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 --2- 172.21.15.153:0/3952905948 >> v2:172.21.15.177:3300/0 conn(0x7f20600d2320 0x7f20600cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20600b5520 con 0x7f206015c9b0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f2054001670 con 0x7f206015c9b0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 >> v1:172.21.15.153:6789/0 conn(0x7f206015c9b0 legacy=0x7f20600b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f2048003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 0x7f2048004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2048004d90 0x7f2048005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600b5800 con 0x7f2048003a30 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600b5300 con 0x7f2048004d90 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204800ba10 con 0x7f2060161340 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205effd700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f2048003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205e7fc700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 0x7f2048004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205f7fe700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2048004d90 0x7f2048005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205effd700 1 -- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 msgr2=0x7f2048004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205effd700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2060161340 0x7f2048004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205effd700 1 -- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2048004d90 msgr2=0x7f2048005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205effd700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2048004d90 0x7f2048005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205effd700 1 -- 172.21.15.153:0/3952905948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20480035b0 con 0x7f2048003a30 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.167+0000 7f205effd700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f2048003e40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f205400c240 tx=0x7f2048005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77b1a2b67fc8c0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054007c10 con 0x7f2048003a30 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205400ce00 con 0x7f2048003a30 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f205dffb700 1 -- 172.21.15.153:0/3952905948 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054005ba0 con 0x7f2048003a30 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f20653ca700 1 -- 172.21.15.153:0/3952905948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 msgr2=0x7f2048003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f20653ca700 1 --2- 172.21.15.153:0/3952905948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f2048003e40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f205400c240 tx=0x7f2048005880 comp rx=0 tx=0).stop 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f205f7fe700 1 -- 172.21.15.153:0/3952905948 reap_dead start 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f20653ca700 1 -- 172.21.15.153:0/3952905948 shutdown_connections 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f20653ca700 1 -- 172.21.15.153:0/3952905948 >> 172.21.15.153:0/3952905948 conn(0x7f2060072600 msgr2=0x7f20600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f20653ca700 1 -- 172.21.15.153:0/3952905948 shutdown_connections 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.168+0000 7f20653ca700 1 -- 172.21.15.153:0/3952905948 wait complete. 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.169+0000 7f20653ca700 1 Processor -- start 2023-11-28T13:01:54.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f20653ca700 1 -- start start 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f20653ca700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f20600cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f20653ca700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20600c8510 0x7f20600c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f20653ca700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20600b4210 0x7f20600cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f20653ca700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c9df0 con 0x7f2048003a30 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f20653ca700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c9680 con 0x7f20600b4210 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f20653ca700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c7750 con 0x7f20600c8510 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f205effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f20600cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f205effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f20600cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41078/0 (socket says 172.21.15.153:41078) 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f205effd700 1 -- 172.21.15.153:0/1448656565 learned_addr learned my addr 172.21.15.153:0/1448656565 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:54.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f205f7fe700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20600b4210 0x7f20600cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.170+0000 7f205e7fc700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20600c8510 0x7f20600c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f205effd700 1 -- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20600c8510 msgr2=0x7f20600c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f205effd700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20600c8510 0x7f20600c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f205effd700 1 -- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20600b4210 msgr2=0x7f20600cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f205effd700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20600b4210 0x7f20600cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f205effd700 1 -- 172.21.15.153:0/1448656565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20600d15e0 con 0x7f2048003a30 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f205effd700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f20600cdb50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2048004750 tx=0x7f2054001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f20477fe700 1 -- 172.21.15.153:0/1448656565 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054004c80 con 0x7f2048003a30 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f20477fe700 1 -- 172.21.15.153:0/1448656565 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2054004e00 con 0x7f2048003a30 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20600d1300 con 0x7f2048003a30 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f20477fe700 1 -- 172.21.15.153:0/1448656565 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054007310 con 0x7f2048003a30 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.171+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206015cd80 con 0x7f2048003a30 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.172+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20600c8360 con 0x7f2048003a30 2023-11-28T13:01:54.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.173+0000 7f20477fe700 1 -- 172.21.15.153:0/1448656565 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2054010070 con 0x7f2048003a30 2023-11-28T13:01:54.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.173+0000 7f20477fe700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f204c0682f0 0x7f204c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.173+0000 7f20477fe700 1 -- 172.21.15.153:0/1448656565 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2054085ea0 con 0x7f2048003a30 2023-11-28T13:01:54.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.173+0000 7f205e7fc700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f204c0682f0 0x7f204c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.173+0000 7f205e7fc700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f204c0682f0 0x7f204c06a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f206015ae80 tx=0x7f2050009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.177+0000 7f20477fe700 1 -- 172.21.15.153:0/1448656565 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2054052a50 con 0x7f2048003a30 2023-11-28T13:01:54.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.361+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f20600745c0 con 0x7f2048003a30 2023-11-28T13:01:54.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.362+0000 7f20477fe700 1 -- 172.21.15.153:0/1448656565 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2054056070 con 0x7f2048003a30 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi153.stdout:[ 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 1, 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": ".mgr" 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:01:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 2, 2023-11-28T13:01:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": "rbd" 2023-11-28T13:01:54.364 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:01:54.364 INFO:teuthology.orchestra.run.smithi153.stdout:] 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f204c0682f0 msgr2=0x7f204c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f204c0682f0 0x7f204c06a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f206015ae80 tx=0x7f2050009210 comp rx=0 tx=0).stop 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 msgr2=0x7f20600cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f20600cdb50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2048004750 tx=0x7f2054001a00 comp rx=0 tx=0).stop 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 shutdown_connections 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f204c0682f0 0x7f204c06a7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2048003a30 0x7f20600cdb50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20600b4210 0x7f20600cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 --2- 172.21.15.153:0/1448656565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20600c8510 0x7f20600c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 >> 172.21.15.153:0/1448656565 conn(0x7f2060072600 msgr2=0x7f20600b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 shutdown_connections 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.364+0000 7f20653ca700 1 -- 172.21.15.153:0/1448656565 wait complete. 2023-11-28T13:01:54.377 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:01:54.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:01:54.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.501+0000 7ff5be2d4700 1 Processor -- start 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.501+0000 7ff5be2d4700 1 -- start start 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5be2d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b80af060 0x7ff5b8070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5be2d4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff5b806a980 0x7ff5b806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5be2d4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b80713c0 con 0x7ff5b80af430 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5be2d4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b806dee0 con 0x7ff5b806a980 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5be2d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80716a0 con 0x7ff5b80af060 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5bc871700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff5b806a980 0x7ff5b806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5b77fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff5b80af430 0x7ff5b806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59252/0 (socket says 172.21.15.153:59252) 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5b77fe700 1 -- 172.21.15.153:0/2350213879 learned_addr learned my addr 172.21.15.153:0/2350213879 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5b7fff700 1 --2- 172.21.15.153:0/2350213879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b80af060 0x7ff5b8070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5bc871700 1 -- 172.21.15.153:0/2350213879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b80af060 msgr2=0x7ff5b8070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5bc871700 1 --2- 172.21.15.153:0/2350213879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b80af060 0x7ff5b8070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5bc871700 1 -- 172.21.15.153:0/2350213879 >> v1:172.21.15.153:6789/0 conn(0x7ff5b80af430 legacy=0x7ff5b806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5bc871700 1 -- 172.21.15.153:0/2350213879 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8071980 con 0x7ff5b806a980 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5b6ffd700 1 -- 172.21.15.153:0/2350213879 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532271820 0 0) 0x7ff5b80713c0 con 0x7ff5b80af430 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5bc871700 1 --2- 172.21.15.153:0/2350213879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b806a980 0x7ff5b806d9a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009d90 tx=0x7ff5b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9de33b04ed3de3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5b6ffd700 1 -- 172.21.15.153:0/2350213879 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000e070 con 0x7ff5b806a980 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5b6ffd700 1 -- 172.21.15.153:0/2350213879 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b000bbd0 con 0x7ff5b806a980 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.502+0000 7ff5b6ffd700 1 -- 172.21.15.153:0/2350213879 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0012430 con 0x7ff5b806a980 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2350213879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b806a980 msgr2=0x7ff5b806d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2350213879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b806a980 0x7ff5b806d9a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009d90 tx=0x7ff5b000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2350213879 shutdown_connections 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2350213879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b80af060 0x7ff5b8070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2350213879 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b806a980 0x7ff5b806d9a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2350213879 >> 172.21.15.153:0/2350213879 conn(0x7ff5b806b380 msgr2=0x7ff5b806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2350213879 shutdown_connections 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.503+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2350213879 wait complete. 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.504+0000 7ff5be2d4700 1 Processor -- start 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.504+0000 7ff5be2d4700 1 -- start start 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5be2d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b806a980 0x7ff5b8159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5be2d4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5b80af060 0x7ff5b80c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5be2d4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b80af430 0x7ff5b80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5be2d4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b51f0 con 0x7ff5b80af430 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5be2d4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b5470 con 0x7ff5b80af060 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5be2d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b80b5750 con 0x7ff5b806a980 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5bc871700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b80af430 0x7ff5b80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5bc871700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b80af430 0x7ff5b80b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41096/0 (socket says 172.21.15.153:41096) 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5bc871700 1 -- 172.21.15.153:0/2152246494 learned_addr learned my addr 172.21.15.153:0/2152246494 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5b7fff700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b806a980 0x7ff5b8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.505+0000 7ff5b77fe700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5b80af060 0x7ff5b80c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5bc871700 1 -- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b806a980 msgr2=0x7ff5b8159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5bc871700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b806a980 0x7ff5b8159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5bc871700 1 -- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5b80af060 msgr2=0x7ff5b80c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5bc871700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5b80af060 0x7ff5b80c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5bc871700 1 -- 172.21.15.153:0/2152246494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b0009a70 con 0x7ff5b80af430 2023-11-28T13:01:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5b7fff700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b806a980 0x7ff5b8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5bc871700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b80af430 0x7ff5b80b4cb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0014040 tx=0x7ff5b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5b4ff9700 1 -- 172.21.15.153:0/2152246494 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0012430 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5b4ff9700 1 -- 172.21.15.153:0/2152246494 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0020070 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b8071f40 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5b4ff9700 1 -- 172.21.15.153:0/2152246494 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0004280 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.506+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b8071c60 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.507+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b80ca660 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.507+0000 7ff5b4ff9700 1 -- 172.21.15.153:0/2152246494 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff5b00043e0 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.508+0000 7ff5b4ff9700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff5a40682f0 0x7ff5a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.508+0000 7ff5b4ff9700 1 -- 172.21.15.153:0/2152246494 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff5b0022070 con 0x7ff5b80af430 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.508+0000 7ff5b7fff700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff5a40682f0 0x7ff5a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.509+0000 7ff5b7fff700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff5a40682f0 0x7ff5a406a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00aa50 tx=0x7ff5ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.512+0000 7ff5b4ff9700 1 -- 172.21.15.153:0/2152246494 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff5b0054ef0 con 0x7ff5b80af430 2023-11-28T13:01:54.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:54.697+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff5b80c4eb0 con 0x7ff5b80af430 2023-11-28T13:01:55.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.529+0000 7ff5b4ff9700 1 -- 172.21.15.153:0/2152246494 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff5b80c4eb0 con 0x7ff5b80af430 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff5a40682f0 msgr2=0x7ff5a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff5a40682f0 0x7ff5a406a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00aa50 tx=0x7ff5ac008040 comp rx=0 tx=0).stop 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b80af430 msgr2=0x7ff5b80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b80af430 0x7ff5b80b4cb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0014040 tx=0x7ff5b0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 shutdown_connections 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff5a40682f0 0x7ff5a406a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5b80af430 0x7ff5b80b4cb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5b806a980 0x7ff5b8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.536 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 --2- 172.21.15.153:0/2152246494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5b80af060 0x7ff5b80c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.537 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.535+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 >> 172.21.15.153:0/2152246494 conn(0x7ff5b806b380 msgr2=0x7ff5b806cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.537 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.536+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 shutdown_connections 2023-11-28T13:01:55.537 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.536+0000 7ff5be2d4700 1 -- 172.21.15.153:0/2152246494 wait complete. 2023-11-28T13:01:55.538 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' created 2023-11-28T13:01:55.552 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:01:55.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.685+0000 7efbf68dd700 1 Processor -- start 2023-11-28T13:01:55.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbf68dd700 1 -- start start 2023-11-28T13:01:55.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbf68dd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efbf0071040 0x7efbf0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbf68dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 0x7efbf01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbf68dd700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf0071950 con 0x7efbf006dee0 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbf68dd700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf015d060 con 0x7efbf0071040 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbf68dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf015d340 con 0x7efbf0160540 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbef7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7efbf006dee0 0x7efbf006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59264/0 (socket says 172.21.15.153:59264) 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.686+0000 7efbef7fe700 1 -- 172.21.15.153:0/2063522208 learned_addr learned my addr 172.21.15.153:0/2063522208 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbf4e7a700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 0x7efbf01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeffff700 1 --2- 172.21.15.153:0/2063522208 >> v2:172.21.15.177:3300/0 conn(0x7efbf0071040 0x7efbf0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098836629 0 0) 0x7efbf0071950 con 0x7efbf006dee0 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbd80035b0 con 0x7efbf006dee0 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4284962739 0 0) 0x7efbd80035b0 con 0x7efbf006dee0 2023-11-28T13:01:55.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 msgr2=0x7efbf01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 0x7efbf01609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 >> v2:172.21.15.177:3300/0 conn(0x7efbf0071040 msgr2=0x7efbf0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 --2- 172.21.15.153:0/2063522208 >> v2:172.21.15.177:3300/0 conn(0x7efbf0071040 0x7efbf0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf015d620 con 0x7efbf006dee0 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7efbe0001670 con 0x7efbf006dee0 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 >> v1:172.21.15.153:6789/0 conn(0x7efbf006dee0 legacy=0x7efbf006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 0x7efbd8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8003f80 0x7efbd8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbd8004d90 0x7efbd8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf015d060 con 0x7efbd8004d90 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf015d340 con 0x7efbd8003f80 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbd800baf0 con 0x7efbf0160540 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbf4e7a700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbd8004d90 0x7efbd8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbef7fe700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8003f80 0x7efbd8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbeffff700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 0x7efbd8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbf4e7a700 1 -- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 msgr2=0x7efbd8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbf4e7a700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf0160540 0x7efbd8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbf4e7a700 1 -- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8003f80 msgr2=0x7efbd8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbf4e7a700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8003f80 0x7efbd8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.687+0000 7efbf4e7a700 1 -- 172.21.15.153:0/2063522208 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbd80035b0 con 0x7efbd8004d90 2023-11-28T13:01:55.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf4e7a700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbd8004d90 0x7efbd8005200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009f90 tx=0x7efbd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4841f910a9429 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800ec30 con 0x7efbd8004d90 2023-11-28T13:01:55.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe800f040 con 0x7efbd8004d90 2023-11-28T13:01:55.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbeeffd700 1 -- 172.21.15.153:0/2063522208 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8014480 con 0x7efbd8004d90 2023-11-28T13:01:55.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf68dd700 1 -- 172.21.15.153:0/2063522208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbd8004d90 msgr2=0x7efbd8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf68dd700 1 --2- 172.21.15.153:0/2063522208 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbd8004d90 0x7efbd8005200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009f90 tx=0x7efbd8005880 comp rx=0 tx=0).stop 2023-11-28T13:01:55.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf4e7a700 1 -- 172.21.15.153:0/2063522208 reap_dead start 2023-11-28T13:01:55.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf68dd700 1 -- 172.21.15.153:0/2063522208 shutdown_connections 2023-11-28T13:01:55.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf68dd700 1 -- 172.21.15.153:0/2063522208 >> 172.21.15.153:0/2063522208 conn(0x7efbf00d0000 msgr2=0x7efbf00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf68dd700 1 -- 172.21.15.153:0/2063522208 shutdown_connections 2023-11-28T13:01:55.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.688+0000 7efbf68dd700 1 -- 172.21.15.153:0/2063522208 wait complete. 2023-11-28T13:01:55.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.689+0000 7efbf68dd700 1 Processor -- start 2023-11-28T13:01:55.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.690+0000 7efbf68dd700 1 -- start start 2023-11-28T13:01:55.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.690+0000 7efbf68dd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8004d90 0x7efbf00c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.690+0000 7efbf68dd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf01e5650 0x7efbf0158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.690+0000 7efbf68dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf00cf360 0x7efbf0154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.690+0000 7efbf68dd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf01e6760 con 0x7efbf01e5650 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.690+0000 7efbf68dd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf0155a90 con 0x7efbd8004d90 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.690+0000 7efbf68dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf01e5ae0 con 0x7efbf00cf360 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf01e5650 0x7efbf0158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf01e5650 0x7efbf0158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41114/0 (socket says 172.21.15.153:41114) 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 -- 172.21.15.153:0/2837332122 learned_addr learned my addr 172.21.15.153:0/2837332122 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbeffff700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8004d90 0x7efbf00c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbf4e7a700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf00cf360 0x7efbf0154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 -- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf00cf360 msgr2=0x7efbf0154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf00cf360 0x7efbf0154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 -- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8004d90 msgr2=0x7efbf00c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8004d90 0x7efbf00c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 -- 172.21.15.153:0/2837332122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf015ccd0 con 0x7efbf01e5650 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbef7fe700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf01e5650 0x7efbf0158ed0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000fb0 tx=0x7efbe00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbecff9700 1 -- 172.21.15.153:0/2837332122 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe000f690 con 0x7efbf01e5650 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbecff9700 1 -- 172.21.15.153:0/2837332122 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe000f810 con 0x7efbf01e5650 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf015d900 con 0x7efbf01e5650 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbecff9700 1 -- 172.21.15.153:0/2837332122 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe000fbc0 con 0x7efbf01e5650 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.691+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf006a270 con 0x7efbf01e5650 2023-11-28T13:01:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.692+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf01e68a0 con 0x7efbf01e5650 2023-11-28T13:01:55.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.693+0000 7efbecff9700 1 -- 172.21.15.153:0/2837332122 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efbe000fd20 con 0x7efbf01e5650 2023-11-28T13:01:55.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.693+0000 7efbecff9700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efbdc0682f0 0x7efbdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.693+0000 7efbecff9700 1 -- 172.21.15.153:0/2837332122 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5580+0+0 (secure 0 0 0) 0x7efbe0085a90 con 0x7efbf01e5650 2023-11-28T13:01:55.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.693+0000 7efbeffff700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efbdc0682f0 0x7efbdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.693+0000 7efbeffff700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efbdc0682f0 0x7efbdc06a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009ce0 tx=0x7efbe4009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.697+0000 7efbecff9700 1 -- 172.21.15.153:0/2837332122 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efbf01e68a0 con 0x7efbf01e5650 2023-11-28T13:01:55.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:55.879+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efbf01e68a0 con 0x7efbf01e5650 2023-11-28T13:01:56.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.542+0000 7efbecff9700 1 -- 172.21.15.153:0/2837332122 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7efbf01e68a0 con 0x7efbf01e5650 2023-11-28T13:01:56.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.548+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efbdc0682f0 msgr2=0x7efbdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.548+0000 7efbf68dd700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efbdc0682f0 0x7efbdc06a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009ce0 tx=0x7efbe4009210 comp rx=0 tx=0).stop 2023-11-28T13:01:56.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.548+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf01e5650 msgr2=0x7efbf0158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.548+0000 7efbf68dd700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf01e5650 0x7efbf0158ed0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000fb0 tx=0x7efbe00075f0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.549+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 shutdown_connections 2023-11-28T13:01:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.549+0000 7efbf68dd700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efbf00cf360 0x7efbf0154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.549+0000 7efbf68dd700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efbd8004d90 0x7efbf00c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.549+0000 7efbf68dd700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efbdc0682f0 0x7efbdc06a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.549+0000 7efbf68dd700 1 --2- 172.21.15.153:0/2837332122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf01e5650 0x7efbf0158ed0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.549+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 >> 172.21.15.153:0/2837332122 conn(0x7efbf00d0000 msgr2=0x7efbf00cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.549+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 shutdown_connections 2023-11-28T13:01:56.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.550+0000 7efbf68dd700 1 -- 172.21.15.153:0/2837332122 wait complete. 2023-11-28T13:01:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' created 2023-11-28T13:01:56.569 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:01:56.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.694+0000 7fdd6292f700 1 Processor -- start 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.694+0000 7fdd6292f700 1 -- start start 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd6292f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd6292f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c071950 con 0x7fdd5c06dee0 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd6292f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c15d060 con 0x7fdd5c160540 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd6292f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c15d340 con 0x7fdd5c071040 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd5b7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fdd5c06dee0 0x7fdd5c06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59286/0 (socket says 172.21.15.153:59286) 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd5b7fe700 1 -- 172.21.15.153:0/1801376462 learned_addr learned my addr 172.21.15.153:0/1801376462 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd5bfff700 1 --2- 172.21.15.153:0/1801376462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016084732 0 0) 0x7fdd5c071950 con 0x7fdd5c06dee0 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.695+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd440035b0 con 0x7fdd5c06dee0 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287934768 0 0) 0x7fdd5c15d060 con 0x7fdd5c160540 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd5c071950 con 0x7fdd5c160540 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 476220989 0 0) 0x7fdd440035b0 con 0x7fdd5c06dee0 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5bfff700 1 -- 172.21.15.153:0/1801376462 >> v1:172.21.15.177:6789/0 conn(0x7fdd5c160540 legacy=0x7fdd5c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5bfff700 1 -- 172.21.15.153:0/1801376462 >> v1:172.21.15.153:6789/0 conn(0x7fdd5c06dee0 legacy=0x7fdd5c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5bfff700 1 -- 172.21.15.153:0/1801376462 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c15d620 con 0x7fdd5c071040 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5bfff700 1 --2- 172.21.15.153:0/1801376462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c071410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009870 tx=0x7fdd50009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7f05c1c6adc63d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c001670 con 0x7fdd5c071040 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50004030 con 0x7fdd5c071040 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.696+0000 7fdd5affd700 1 -- 172.21.15.153:0/1801376462 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd500043a0 con 0x7fdd5c071040 2023-11-28T13:01:56.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.697+0000 7fdd6292f700 1 -- 172.21.15.153:0/1801376462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 msgr2=0x7fdd5c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.697+0000 7fdd6292f700 1 --2- 172.21.15.153:0/1801376462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c071410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009870 tx=0x7fdd50009b80 comp rx=0 tx=0).stop 2023-11-28T13:01:56.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.697+0000 7fdd6292f700 1 -- 172.21.15.153:0/1801376462 shutdown_connections 2023-11-28T13:01:56.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.697+0000 7fdd6292f700 1 --2- 172.21.15.153:0/1801376462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c071410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.697+0000 7fdd6292f700 1 -- 172.21.15.153:0/1801376462 >> 172.21.15.153:0/1801376462 conn(0x7fdd5c0d0000 msgr2=0x7fdd5c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.697+0000 7fdd6292f700 1 -- 172.21.15.153:0/1801376462 shutdown_connections 2023-11-28T13:01:56.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.697+0000 7fdd6292f700 1 -- 172.21.15.153:0/1801376462 wait complete. 2023-11-28T13:01:56.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.698+0000 7fdd6292f700 1 Processor -- start 2023-11-28T13:01:56.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.698+0000 7fdd6292f700 1 -- start start 2023-11-28T13:01:56.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd6292f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd5c06dee0 0x7fdd5c153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd6292f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd6292f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c160540 0x7fdd5c0ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd6292f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0adeb0 con 0x7fdd5c06dee0 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd6292f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1557d0 con 0x7fdd5c160540 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd6292f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0ace90 con 0x7fdd5c071040 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd5c06dee0 0x7fdd5c153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd5c06dee0 0x7fdd5c153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41126/0 (socket says 172.21.15.153:41126) 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 -- 172.21.15.153:0/972484507 learned_addr learned my addr 172.21.15.153:0/972484507 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd60ecc700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c160540 0x7fdd5c0ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5b7fe700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 -- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 msgr2=0x7fdd5c155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 -- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c160540 msgr2=0x7fdd5c0ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c160540 0x7fdd5c0ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 -- 172.21.15.153:0/972484507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd500094a0 con 0x7fdd5c06dee0 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.699+0000 7fdd5bfff700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd5c06dee0 0x7fdd5c153470 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdd500090b0 tx=0x7fdd50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.700+0000 7fdd58ff9700 1 -- 172.21.15.153:0/972484507 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003a50 con 0x7fdd5c06dee0 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.700+0000 7fdd58ff9700 1 -- 172.21.15.153:0/972484507 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5001f070 con 0x7fdd5c06dee0 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.700+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c15ccd0 con 0x7fdd5c06dee0 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.700+0000 7fdd58ff9700 1 -- 172.21.15.153:0/972484507 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003df0 con 0x7fdd5c06dee0 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.700+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c15d900 con 0x7fdd5c06dee0 2023-11-28T13:01:56.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.701+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c06a270 con 0x7fdd5c06dee0 2023-11-28T13:01:56.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.701+0000 7fdd58ff9700 1 -- 172.21.15.153:0/972484507 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd5002a020 con 0x7fdd5c06dee0 2023-11-28T13:01:56.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.702+0000 7fdd58ff9700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdd480682f0 0x7fdd4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.702+0000 7fdd58ff9700 1 -- 172.21.15.153:0/972484507 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fdd50086f90 con 0x7fdd5c06dee0 2023-11-28T13:01:56.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.702+0000 7fdd5b7fe700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdd480682f0 0x7fdd4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.702+0000 7fdd5b7fe700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdd480682f0 0x7fdd4806a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c007fd0 tx=0x7fdd4c007dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.706+0000 7fdd58ff9700 1 -- 172.21.15.153:0/972484507 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd50053850 con 0x7fdd5c06dee0 2023-11-28T13:01:56.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.930+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdd5c0af2d0 con 0x7fdd5c06dee0 2023-11-28T13:01:56.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.935+0000 7fdd58ff9700 1 -- 172.21.15.153:0/972484507 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdd50056e70 con 0x7fdd5c06dee0 2023-11-28T13:01:56.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdd480682f0 msgr2=0x7fdd4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdd480682f0 0x7fdd4806a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c007fd0 tx=0x7fdd4c007dd0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd5c06dee0 msgr2=0x7fdd5c153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd5c06dee0 0x7fdd5c153470 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdd500090b0 tx=0x7fdd50004750 comp rx=0 tx=0).stop 2023-11-28T13:01:56.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 shutdown_connections 2023-11-28T13:01:56.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdd480682f0 0x7fdd4806a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd5c06dee0 0x7fdd5c153470 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c160540 0x7fdd5c0ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.939+0000 7fdd6292f700 1 --2- 172.21.15.153:0/972484507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c071040 0x7fdd5c155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.940+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 >> 172.21.15.153:0/972484507 conn(0x7fdd5c0d0000 msgr2=0x7fdd5c15fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.942+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 shutdown_connections 2023-11-28T13:01:56.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:56.942+0000 7fdd6292f700 1 -- 172.21.15.153:0/972484507 wait complete. 2023-11-28T13:01:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T13:01:56.964 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:57.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.084+0000 7f824024f700 1 Processor -- start 2023-11-28T13:01:57.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.084+0000 7f824024f700 1 -- start start 2023-11-28T13:01:57.087 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f824024f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823815cef0 0x7f82380af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f824024f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f823806ba90 0x7f82380ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f824024f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82380cc5f0 con 0x7f82380cbdf0 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f824024f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823815d470 con 0x7f823806ba90 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f824024f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823815d750 con 0x7f823815cef0 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f823e7ec700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f823806ba90 0x7f82380ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f823e7ec700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f823806ba90 0x7f82380ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41138/0 (socket says 172.21.15.153:41138) 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.085+0000 7f823e7ec700 1 -- 172.21.15.153:0/4239895592 learned_addr learned my addr 172.21.15.153:0/4239895592 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823dfeb700 1 --2- 172.21.15.153:0/4239895592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823815cef0 0x7f82380af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823e7ec700 1 -- 172.21.15.153:0/4239895592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823815cef0 msgr2=0x7f82380af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823e7ec700 1 --2- 172.21.15.153:0/4239895592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823815cef0 0x7f82380af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823e7ec700 1 -- 172.21.15.153:0/4239895592 >> v1:172.21.15.177:6789/0 conn(0x7f82380cbdf0 legacy=0x7f823806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823e7ec700 1 -- 172.21.15.153:0/4239895592 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823815da30 con 0x7f823806ba90 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823e7ec700 1 --2- 172.21.15.153:0/4239895592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823806ba90 0x7f82380ca650 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8230009e50 tx=0x7f823000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8443c5e2eb2cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823cfe9700 1 -- 172.21.15.153:0/4239895592 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000e070 con 0x7f823806ba90 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823cfe9700 1 -- 172.21.15.153:0/4239895592 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230003a20 con 0x7f823806ba90 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.086+0000 7f823cfe9700 1 -- 172.21.15.153:0/4239895592 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230003dd0 con 0x7f823806ba90 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 -- 172.21.15.153:0/4239895592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823806ba90 msgr2=0x7f82380ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 --2- 172.21.15.153:0/4239895592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823806ba90 0x7f82380ca650 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8230009e50 tx=0x7f823000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 -- 172.21.15.153:0/4239895592 shutdown_connections 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 --2- 172.21.15.153:0/4239895592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f823806ba90 0x7f82380ca650 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 --2- 172.21.15.153:0/4239895592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823815cef0 0x7f82380af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 -- 172.21.15.153:0/4239895592 >> 172.21.15.153:0/4239895592 conn(0x7f82380d0000 msgr2=0x7f82380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 -- 172.21.15.153:0/4239895592 shutdown_connections 2023-11-28T13:01:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.087+0000 7f824024f700 1 -- 172.21.15.153:0/4239895592 wait complete. 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.088+0000 7f824024f700 1 Processor -- start 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.088+0000 7f824024f700 1 -- start start 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f824024f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806ba90 0x7f82380b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f824024f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82380cbdf0 0x7f8238152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f824024f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815cef0 0x7f823809ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f824024f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380b4de0 con 0x7f82380cbdf0 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f824024f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380b3e80 con 0x7f823815cef0 2023-11-28T13:01:57.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f824024f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82380b4160 con 0x7f823806ba90 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f823d7ea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82380cbdf0 0x7f8238152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f823d7ea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82380cbdf0 0x7f8238152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41142/0 (socket says 172.21.15.153:41142) 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.089+0000 7f823d7ea700 1 -- 172.21.15.153:0/1143280170 learned_addr learned my addr 172.21.15.153:0/1143280170 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823e7ec700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815cef0 0x7f823809ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823dfeb700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806ba90 0x7f82380b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823d7ea700 1 -- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806ba90 msgr2=0x7f82380b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823d7ea700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806ba90 0x7f82380b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823d7ea700 1 -- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815cef0 msgr2=0x7f823809ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823d7ea700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815cef0 0x7f823809ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823d7ea700 1 -- 172.21.15.153:0/1143280170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230009b30 con 0x7f82380cbdf0 2023-11-28T13:01:57.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823e7ec700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815cef0 0x7f823809ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823dfeb700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806ba90 0x7f82380b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f823d7ea700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82380cbdf0 0x7f8238152be0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f823400c4d0 tx=0x7f823400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f822e7fc700 1 -- 172.21.15.153:0/1143280170 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234011790 con 0x7f82380cbdf0 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f822e7fc700 1 -- 172.21.15.153:0/1143280170 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82340118f0 con 0x7f82380cbdf0 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823815dff0 con 0x7f82380cbdf0 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f822e7fc700 1 -- 172.21.15.153:0/1143280170 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234011ca0 con 0x7f82380cbdf0 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.090+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823815dd10 con 0x7f82380cbdf0 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.091+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82380cab90 con 0x7f82380cbdf0 2023-11-28T13:01:57.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.091+0000 7f822e7fc700 1 -- 172.21.15.153:0/1143280170 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8234010050 con 0x7f82380cbdf0 2023-11-28T13:01:57.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.092+0000 7f822e7fc700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f821c0682f0 0x7f821c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.092+0000 7f822e7fc700 1 -- 172.21.15.153:0/1143280170 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8234085a60 con 0x7f82380cbdf0 2023-11-28T13:01:57.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.092+0000 7f823dfeb700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f821c0682f0 0x7f821c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.092+0000 7f823dfeb700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f821c0682f0 0x7f821c06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f823806e0b0 tx=0x7f8228008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.096+0000 7f822e7fc700 1 -- 172.21.15.153:0/1143280170 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f82340521d0 con 0x7f82380cbdf0 2023-11-28T13:01:57.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.278+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82380af720 con 0x7f82380cbdf0 2023-11-28T13:01:57.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.279+0000 7f822e7fc700 1 -- 172.21.15.153:0/1143280170 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18040 (secure 0 0 0) 0x7f82340557f0 con 0x7f82380cbdf0 2023-11-28T13:01:57.280 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:57.280 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":50,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:56.931399+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:54.715348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:55.897576+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f821c0682f0 msgr2=0x7f821c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f821c0682f0 0x7f821c06a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f823806e0b0 tx=0x7f8228008040 comp rx=0 tx=0).stop 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82380cbdf0 msgr2=0x7f8238152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82380cbdf0 0x7f8238152be0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f823400c4d0 tx=0x7f823400c890 comp rx=0 tx=0).stop 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 shutdown_connections 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815cef0 0x7f823809ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806ba90 0x7f82380b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f821c0682f0 0x7f821c06a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 --2- 172.21.15.153:0/1143280170 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82380cbdf0 0x7f8238152be0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 >> 172.21.15.153:0/1143280170 conn(0x7f82380d0000 msgr2=0x7f82380cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:57.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 shutdown_connections 2023-11-28T13:01:57.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.281+0000 7f824024f700 1 -- 172.21.15.153:0/1143280170 wait complete. 2023-11-28T13:01:57.294 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:57.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.415+0000 7fdfcc618700 1 Processor -- start 2023-11-28T13:01:57.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.415+0000 7fdfcc618700 1 -- start start 2023-11-28T13:01:57.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfcc618700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdfc4154e70 0x7fdfc40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfcc618700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4071750 0x7fdfc415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfcc618700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc4155240 con 0x7fdfc40cf360 2023-11-28T13:01:57.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfcc618700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc415b100 con 0x7fdfc4154e70 2023-11-28T13:01:57.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfcc618700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc415b3e0 con 0x7fdfc4071750 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfcabb5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4071750 0x7fdfc415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfca3b4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdfc4154e70 0x7fdfc40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfca3b4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdfc4154e70 0x7fdfc40cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:50614/0 (socket says 172.21.15.153:50614) 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfcabb5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4071750 0x7fdfc415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:58762/0 (socket says 172.21.15.153:58762) 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfca3b4700 1 -- 172.21.15.153:0/1444927367 learned_addr learned my addr 172.21.15.153:0/1444927367 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.416+0000 7fdfc9bb3700 1 --1- 172.21.15.153:0/1444927367 >> v1:172.21.15.177:6789/0 conn(0x7fdfc40cf360 0x7fdfc4071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:35928/0 (socket says 172.21.15.153:35928) 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfcabb5700 1 -- 172.21.15.153:0/1444927367 >> v2:172.21.15.177:3300/0 conn(0x7fdfc4154e70 msgr2=0x7fdfc40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfcabb5700 1 --2- 172.21.15.153:0/1444927367 >> v2:172.21.15.177:3300/0 conn(0x7fdfc4154e70 0x7fdfc40cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfcabb5700 1 -- 172.21.15.153:0/1444927367 >> v1:172.21.15.177:6789/0 conn(0x7fdfc40cf360 legacy=0x7fdfc4071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfcabb5700 1 -- 172.21.15.153:0/1444927367 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc415b6c0 con 0x7fdfc4071750 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfca3b4700 1 --2- 172.21.15.153:0/1444927367 >> v2:172.21.15.177:3300/0 conn(0x7fdfc4154e70 0x7fdfc40cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:57.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfcabb5700 1 --2- 172.21.15.153:0/1444927367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4071750 0x7fdfc415e680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009d90 tx=0x7fdfbc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5b2d6438983d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfc93b2700 1 -- 172.21.15.153:0/1444927367 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00e070 con 0x7fdfc4071750 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfc93b2700 1 -- 172.21.15.153:0/1444927367 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc00bb20 con 0x7fdfc4071750 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfc93b2700 1 -- 172.21.15.153:0/1444927367 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc012430 con 0x7fdfc4071750 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfcc618700 1 -- 172.21.15.153:0/1444927367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4071750 msgr2=0x7fdfc415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.417+0000 7fdfcc618700 1 --2- 172.21.15.153:0/1444927367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4071750 0x7fdfc415e680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009d90 tx=0x7fdfbc00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.418+0000 7fdfcc618700 1 -- 172.21.15.153:0/1444927367 shutdown_connections 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.418+0000 7fdfcc618700 1 --2- 172.21.15.153:0/1444927367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4071750 0x7fdfc415e680 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.418+0000 7fdfcc618700 1 --2- 172.21.15.153:0/1444927367 >> v2:172.21.15.177:3300/0 conn(0x7fdfc4154e70 0x7fdfc40cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.418+0000 7fdfcc618700 1 -- 172.21.15.153:0/1444927367 >> 172.21.15.153:0/1444927367 conn(0x7fdfc40d0000 msgr2=0x7fdfc40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.418+0000 7fdfcc618700 1 -- 172.21.15.153:0/1444927367 shutdown_connections 2023-11-28T13:01:57.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.418+0000 7fdfcc618700 1 -- 172.21.15.153:0/1444927367 wait complete. 2023-11-28T13:01:57.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 Processor -- start 2023-11-28T13:01:57.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 -- start start 2023-11-28T13:01:57.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfc4071750 0x7fdfc414a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.423 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfc40cf360 0x7fdfc40c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4154e70 0x7fdfc4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40c47d0 con 0x7fdfc4071750 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40c44f0 con 0x7fdfc40cf360 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.419+0000 7fdfcc618700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc41f3e00 con 0x7fdfc4154e70 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfc4071750 0x7fdfc414a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfc4071750 0x7fdfc414a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41154/0 (socket says 172.21.15.153:41154) 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 -- 172.21.15.153:0/2532410174 learned_addr learned my addr 172.21.15.153:0/2532410174 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfcabb5700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4154e70 0x7fdfc4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfc9bb3700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfc40cf360 0x7fdfc40c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 -- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4154e70 msgr2=0x7fdfc4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4154e70 0x7fdfc4159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 -- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfc40cf360 msgr2=0x7fdfc40c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfc40cf360 0x7fdfc40c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 -- 172.21.15.153:0/2532410174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc0009550 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfca3b4700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfc4071750 0x7fdfc414a3b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000e9b0 tx=0x7fdfc000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfbaffd700 1 -- 172.21.15.153:0/2532410174 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000ee50 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfbaffd700 1 -- 172.21.15.153:0/2532410174 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000c850 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc009a70 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfbaffd700 1 -- 172.21.15.153:0/2532410174 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000cbc0 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.420+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc40af520 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.421+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc415b9a0 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.422+0000 7fdfbaffd700 1 -- 172.21.15.153:0/2532410174 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdfc0019030 con 0x7fdfc4071750 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.422+0000 7fdfbaffd700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdfa80682f0 0x7fdfa806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.425 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.422+0000 7fdfbaffd700 1 -- 172.21.15.153:0/2532410174 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfc0085b30 con 0x7fdfc4071750 2023-11-28T13:01:57.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.422+0000 7fdfc9bb3700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdfa80682f0 0x7fdfa806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.426 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.422+0000 7fdfc9bb3700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdfa80682f0 0x7fdfa806a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdfc415dff0 tx=0x7fdfb4007ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.427 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.426+0000 7fdfbaffd700 1 -- 172.21.15.153:0/2532410174 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdfc0052390 con 0x7fdfc4071750 2023-11-28T13:01:57.523 INFO:tasks.ceph.mds.b.smithi177.stderr:2023-11-28T13:01:57.522+0000 7fb3262b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:01:57.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.609+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfc415adb0 con 0x7fdfc4071750 2023-11-28T13:01:57.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.610+0000 7fdfbaffd700 1 -- 172.21.15.153:0/2532410174 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18040 (secure 0 0 0) 0x7fdfc00559b0 con 0x7fdfc4071750 2023-11-28T13:01:57.611 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:01:57.611 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":50,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:56.931399+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:54.715348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:55.897576+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:57.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdfa80682f0 msgr2=0x7fdfa806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdfa80682f0 0x7fdfa806a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdfc415dff0 tx=0x7fdfb4007ed0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfc4071750 msgr2=0x7fdfc414a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfc4071750 0x7fdfc414a3b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000e9b0 tx=0x7fdfc000ecc0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 shutdown_connections 2023-11-28T13:01:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdfa80682f0 0x7fdfa806a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdfc4071750 0x7fdfc414a3b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdfc4154e70 0x7fdfc4159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 --2- 172.21.15.153:0/2532410174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfc40cf360 0x7fdfc40c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 >> 172.21.15.153:0/2532410174 conn(0x7fdfc40d0000 msgr2=0x7fdfc4153440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 shutdown_connections 2023-11-28T13:01:57.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.612+0000 7fdfcc618700 1 -- 172.21.15.153:0/2532410174 wait complete. 2023-11-28T13:01:57.626 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:01:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.748+0000 7f494fb92700 1 Processor -- start 2023-11-28T13:01:57.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.749+0000 7f494fb92700 1 -- start start 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.749+0000 7f494fb92700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49480d01b0 0x7f49480d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.749+0000 7f494fb92700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b20b0 0x7f49480d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.749+0000 7f494fb92700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f49480b29b0 0x7f49480b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.749+0000 7f494fb92700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494815cec0 con 0x7f49480b29b0 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.749+0000 7f494fb92700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480b0230 con 0x7f49480d01b0 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.749+0000 7f494fb92700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480b4570 con 0x7f49480b20b0 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f49480b29b0 0x7f49480b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f49480b29b0 0x7f49480b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41166/0 (socket says 172.21.15.153:41166) 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 -- 172.21.15.153:0/518549342 learned_addr learned my addr 172.21.15.153:0/518549342 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:57.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494d92e700 1 --2- 172.21.15.153:0/518549342 >> v2:172.21.15.177:3300/0 conn(0x7f49480d01b0 0x7f49480d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494d12d700 1 --2- 172.21.15.153:0/518549342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b20b0 0x7f49480d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 -- 172.21.15.153:0/518549342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b20b0 msgr2=0x7f49480d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 --2- 172.21.15.153:0/518549342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b20b0 0x7f49480d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 -- 172.21.15.153:0/518549342 >> v2:172.21.15.177:3300/0 conn(0x7f49480d01b0 msgr2=0x7f49480d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 --2- 172.21.15.153:0/518549342 >> v2:172.21.15.177:3300/0 conn(0x7f49480d01b0 0x7f49480d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 -- 172.21.15.153:0/518549342 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480b4850 con 0x7f49480b29b0 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494d12d700 1 --2- 172.21.15.153:0/518549342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b20b0 0x7f49480d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494e12f700 1 --2- 172.21.15.153:0/518549342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b29b0 0x7f49480b2e30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f49480b5bb0 tx=0x7f494000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b3cb2244ee9d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494c92c700 1 -- 172.21.15.153:0/518549342 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494000e070 con 0x7f49480b29b0 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494c92c700 1 -- 172.21.15.153:0/518549342 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494000b9c0 con 0x7f49480b29b0 2023-11-28T13:01:57.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.750+0000 7f494c92c700 1 -- 172.21.15.153:0/518549342 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494000bd70 con 0x7f49480b29b0 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 -- 172.21.15.153:0/518549342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b29b0 msgr2=0x7f49480b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 --2- 172.21.15.153:0/518549342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b29b0 0x7f49480b2e30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f49480b5bb0 tx=0x7f494000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 -- 172.21.15.153:0/518549342 shutdown_connections 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 --2- 172.21.15.153:0/518549342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b29b0 0x7f49480b2e30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 --2- 172.21.15.153:0/518549342 >> v2:172.21.15.177:3300/0 conn(0x7f49480d01b0 0x7f49480d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 --2- 172.21.15.153:0/518549342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b20b0 0x7f49480d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 -- 172.21.15.153:0/518549342 >> 172.21.15.153:0/518549342 conn(0x7f494809c3f0 msgr2=0x7f49480d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 -- 172.21.15.153:0/518549342 shutdown_connections 2023-11-28T13:01:57.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.751+0000 7f494fb92700 1 -- 172.21.15.153:0/518549342 wait complete. 2023-11-28T13:01:57.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.752+0000 7f494fb92700 1 Processor -- start 2023-11-28T13:01:57.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.752+0000 7f494fb92700 1 -- start start 2023-11-28T13:01:57.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494fb92700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b20b0 0x7f49480cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494fb92700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b29b0 0x7f49480a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494fb92700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49480d01b0 0x7f49481f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494fb92700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480cd4b0 con 0x7f49480b20b0 2023-11-28T13:01:57.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494fb92700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480cc720 con 0x7f49480d01b0 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494fb92700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480cb0a0 con 0x7f49480b29b0 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b20b0 0x7f49480cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b20b0 0x7f49480cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41168/0 (socket says 172.21.15.153:41168) 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 -- 172.21.15.153:0/1746040124 learned_addr learned my addr 172.21.15.153:0/1746040124 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d12d700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b29b0 0x7f49480a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494e12f700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49480d01b0 0x7f49481f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 -- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b29b0 msgr2=0x7f49480a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b29b0 0x7f49480a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 -- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49480d01b0 msgr2=0x7f49481f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49480d01b0 0x7f49481f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 -- 172.21.15.153:0/1746040124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009b30 con 0x7f49480b20b0 2023-11-28T13:01:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d12d700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b29b0 0x7f49480a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494e12f700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49480d01b0 0x7f49481f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494d92e700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b20b0 0x7f49480cc1e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f493800d800 tx=0x7f493800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f493e7fc700 1 -- 172.21.15.153:0/1746040124 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380125f0 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f493e7fc700 1 -- 172.21.15.153:0/1746040124 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4938012750 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49480afec0 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.753+0000 7f493e7fc700 1 -- 172.21.15.153:0/1746040124 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938012b00 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.754+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49480afbe0 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.754+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49480b2600 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.755+0000 7f493e7fc700 1 -- 172.21.15.153:0/1746040124 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4938012c60 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.755+0000 7f493e7fc700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f492c0682f0 0x7f492c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.756+0000 7f493e7fc700 1 -- 172.21.15.153:0/1746040124 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4938086180 con 0x7f49480b20b0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.756+0000 7f494d12d700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f492c0682f0 0x7f492c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:57.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.756+0000 7f494d12d700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f492c0682f0 0x7f492c06a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f49480c9cd0 tx=0x7f4944008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:57.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.760+0000 7f493e7fc700 1 -- 172.21.15.153:0/1746040124 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f49380529e0 con 0x7f49480b20b0 2023-11-28T13:01:57.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:57.990+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f49480c6980 con 0x7f49480b20b0 2023-11-28T13:01:58.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.943+0000 7f493e7fc700 1 -- 172.21.15.153:0/1746040124 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4938056000 con 0x7f49480b20b0 2023-11-28T13:01:58.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f492c0682f0 msgr2=0x7f492c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:58.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f492c0682f0 0x7f492c06a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f49480c9cd0 tx=0x7f4944008040 comp rx=0 tx=0).stop 2023-11-28T13:01:58.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b20b0 msgr2=0x7f49480cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b20b0 0x7f49480cc1e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f493800d800 tx=0x7f493800dbc0 comp rx=0 tx=0).stop 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 shutdown_connections 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f492c0682f0 0x7f492c06a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49480b20b0 0x7f49480cc1e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49480d01b0 0x7f49481f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 --2- 172.21.15.153:0/1746040124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49480b29b0 0x7f49480a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 >> 172.21.15.153:0/1746040124 conn(0x7f494809c3f0 msgr2=0x7f49480b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 shutdown_connections 2023-11-28T13:01:58.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:58.946+0000 7f494fb92700 1 -- 172.21.15.153:0/1746040124 wait complete. 2023-11-28T13:01:58.962 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:01:58.962 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:01:59.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.091+0000 7f12a9900700 1 Processor -- start 2023-11-28T13:01:59.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.091+0000 7f12a9900700 1 -- start start 2023-11-28T13:01:59.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.091+0000 7f12a9900700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f12a415ca50 0x7f12a406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:59.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.091+0000 7f12a9900700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f12a40cce10 0x7f12a40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:59.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.091+0000 7f12a9900700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40cf5e0 con 0x7f12a406b8c0 2023-11-28T13:01:59.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.091+0000 7f12a9900700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a406bd60 con 0x7f12a40cce10 2023-11-28T13:01:59.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.091+0000 7f12a9900700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40cd470 con 0x7f12a415ca50 2023-11-28T13:01:59.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a37fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f12a40cce10 0x7f12a40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:59.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a27fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f12a406b8c0 0x7f12a40cc700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59340/0 (socket says 172.21.15.153:59340) 2023-11-28T13:01:59.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a27fc700 1 -- 172.21.15.153:0/2546691193 learned_addr learned my addr 172.21.15.153:0/2546691193 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:01:59.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a2ffd700 1 --2- 172.21.15.153:0/2546691193 >> v2:172.21.15.177:3300/0 conn(0x7f12a415ca50 0x7f12a406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:59.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a37fe700 1 -- 172.21.15.153:0/2546691193 >> v2:172.21.15.177:3300/0 conn(0x7f12a415ca50 msgr2=0x7f12a406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:59.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a37fe700 1 --2- 172.21.15.153:0/2546691193 >> v2:172.21.15.177:3300/0 conn(0x7f12a415ca50 0x7f12a406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a37fe700 1 -- 172.21.15.153:0/2546691193 >> v1:172.21.15.153:6789/0 conn(0x7f12a406b8c0 legacy=0x7f12a40cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a37fe700 1 -- 172.21.15.153:0/2546691193 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40cecb0 con 0x7f12a40cce10 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a1ffb700 1 -- 172.21.15.153:0/2546691193 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689678892 0 0) 0x7f12a40cf5e0 con 0x7f12a406b8c0 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a2ffd700 1 --2- 172.21.15.153:0/2546691193 >> v2:172.21.15.177:3300/0 conn(0x7f12a415ca50 0x7f12a406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a37fe700 1 --2- 172.21.15.153:0/2546691193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a40cce10 0x7f12a40cfa70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f12a4071240 tx=0x7f129c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a7069650e3f82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a1ffb700 1 -- 172.21.15.153:0/2546691193 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00e070 con 0x7f12a40cce10 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a1ffb700 1 -- 172.21.15.153:0/2546691193 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c00bb90 con 0x7f12a40cce10 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.092+0000 7f12a1ffb700 1 -- 172.21.15.153:0/2546691193 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c012430 con 0x7f12a40cce10 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 -- 172.21.15.153:0/2546691193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a40cce10 msgr2=0x7f12a40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 --2- 172.21.15.153:0/2546691193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a40cce10 0x7f12a40cfa70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f12a4071240 tx=0x7f129c00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:59.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 -- 172.21.15.153:0/2546691193 shutdown_connections 2023-11-28T13:01:59.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 --2- 172.21.15.153:0/2546691193 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a40cce10 0x7f12a40cfa70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 --2- 172.21.15.153:0/2546691193 >> v2:172.21.15.177:3300/0 conn(0x7f12a415ca50 0x7f12a406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 -- 172.21.15.153:0/2546691193 >> 172.21.15.153:0/2546691193 conn(0x7f12a409a230 msgr2=0x7f12a408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:59.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 -- 172.21.15.153:0/2546691193 shutdown_connections 2023-11-28T13:01:59.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.093+0000 7f12a9900700 1 -- 172.21.15.153:0/2546691193 wait complete. 2023-11-28T13:01:59.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.094+0000 7f12a9900700 1 Processor -- start 2023-11-28T13:01:59.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.094+0000 7f12a9900700 1 -- start start 2023-11-28T13:01:59.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a9900700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a406b8c0 0x7f12a415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a9900700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a40cce10 0x7f12a4155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a9900700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a415ca50 0x7f12a415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a9900700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40b0320 con 0x7f12a406b8c0 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a9900700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a4160070 con 0x7f12a415ca50 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a9900700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40b05c0 con 0x7f12a40cce10 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a406b8c0 0x7f12a415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a406b8c0 0x7f12a415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41182/0 (socket says 172.21.15.153:41182) 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 -- 172.21.15.153:0/324705080 learned_addr learned my addr 172.21.15.153:0/324705080 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a37fe700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a415ca50 0x7f12a415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a27fc700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a40cce10 0x7f12a4155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 -- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a40cce10 msgr2=0x7f12a4155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:59.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a40cce10 0x7f12a4155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 -- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a415ca50 msgr2=0x7f12a415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a415ca50 0x7f12a415ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 -- 172.21.15.153:0/324705080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c009a70 con 0x7f12a406b8c0 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a27fc700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a40cce10 0x7f12a4155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.095+0000 7f12a2ffd700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a406b8c0 0x7f12a415bdf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f129400e760 tx=0x7f129400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.096+0000 7f128bfff700 1 -- 172.21.15.153:0/324705080 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294013610 con 0x7f12a406b8c0 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.096+0000 7f128bfff700 1 -- 172.21.15.153:0/324705080 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294013770 con 0x7f12a406b8c0 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.096+0000 7f128bfff700 1 -- 172.21.15.153:0/324705080 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294013b20 con 0x7f12a406b8c0 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.096+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a40cf270 con 0x7f12a406b8c0 2023-11-28T13:01:59.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.096+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a40cef90 con 0x7f12a406b8c0 2023-11-28T13:01:59.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.097+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a415ce20 con 0x7f12a406b8c0 2023-11-28T13:01:59.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.097+0000 7f128bfff700 1 -- 172.21.15.153:0/324705080 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1294023070 con 0x7f12a406b8c0 2023-11-28T13:01:59.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.097+0000 7f128bfff700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12900682f0 0x7f129006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:59.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.098+0000 7f12a27fc700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12900682f0 0x7f129006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:59.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.098+0000 7f128bfff700 1 -- 172.21.15.153:0/324705080 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1294085930 con 0x7f12a406b8c0 2023-11-28T13:01:59.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.098+0000 7f12a27fc700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12900682f0 0x7f129006a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f12a41f5580 tx=0x7f1298009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:59.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.102+0000 7f128bfff700 1 -- 172.21.15.153:0/324705080 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1294052100 con 0x7f12a406b8c0 2023-11-28T13:01:59.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.330+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f12a40745c0 con 0x7f12a406b8c0 2023-11-28T13:01:59.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.945+0000 7f128bfff700 1 -- 172.21.15.153:0/324705080 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f12940210c0 con 0x7f12a406b8c0 2023-11-28T13:01:59.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.947+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12900682f0 msgr2=0x7f129006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:59.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.947+0000 7f12a9900700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12900682f0 0x7f129006a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f12a41f5580 tx=0x7f1298009040 comp rx=0 tx=0).stop 2023-11-28T13:01:59.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.947+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a406b8c0 msgr2=0x7f12a415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:59.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.947+0000 7f12a9900700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a406b8c0 0x7f12a415bdf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f129400e760 tx=0x7f129400eb20 comp rx=0 tx=0).stop 2023-11-28T13:01:59.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.947+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 shutdown_connections 2023-11-28T13:01:59.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.948+0000 7f12a9900700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12900682f0 0x7f129006a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.948+0000 7f12a9900700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12a406b8c0 0x7f12a415bdf0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.948+0000 7f12a9900700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a415ca50 0x7f12a415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.948+0000 7f12a9900700 1 --2- 172.21.15.153:0/324705080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12a40cce10 0x7f12a4155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:59.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.948+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 >> 172.21.15.153:0/324705080 conn(0x7f12a409a230 msgr2=0x7f12a4158520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:59.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.948+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 shutdown_connections 2023-11-28T13:01:59.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:01:59.948+0000 7f12a9900700 1 -- 172.21.15.153:0/324705080 wait complete. 2023-11-28T13:01:59.962 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:01:59.963 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:00.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08ea11f700 1 Processor -- start 2023-11-28T13:02:00.086 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08ea11f700 1 -- start start 2023-11-28T13:02:00.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08ea11f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e4140950 0x7f08e40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08ea11f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f08e40b2e70 0x7f08e406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08ea11f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e413ff00 con 0x7f08e40cd8c0 2023-11-28T13:02:00.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08ea11f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e41404c0 con 0x7f08e40b2e70 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08ea11f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4140dc0 con 0x7f08e4140950 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.085+0000 7f08e3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f08e40b2e70 0x7f08e406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e37fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f08e40cd8c0 0x7f08e40cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59354/0 (socket says 172.21.15.153:59354) 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e37fe700 1 -- 172.21.15.153:0/2509304563 learned_addr learned my addr 172.21.15.153:0/2509304563 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e3fff700 1 -- 172.21.15.153:0/2509304563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e4140950 msgr2=0x7f08e40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e3fff700 1 --2- 172.21.15.153:0/2509304563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e4140950 0x7f08e40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e3fff700 1 -- 172.21.15.153:0/2509304563 >> v1:172.21.15.153:6789/0 conn(0x7f08e40cd8c0 legacy=0x7f08e40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e3fff700 1 -- 172.21.15.153:0/2509304563 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e415aa80 con 0x7f08e40b2e70 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e27fc700 1 -- 172.21.15.153:0/2509304563 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605985865 0 0) 0x7f08e413ff00 con 0x7f08e40cd8c0 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e3fff700 1 --2- 172.21.15.153:0/2509304563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e406f850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009d90 tx=0x7f08dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163d489c49df6b34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e27fc700 1 -- 172.21.15.153:0/2509304563 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00e070 con 0x7f08e40b2e70 2023-11-28T13:02:00.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e27fc700 1 -- 172.21.15.153:0/2509304563 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00bbd0 con 0x7f08e40b2e70 2023-11-28T13:02:00.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.086+0000 7f08e27fc700 1 -- 172.21.15.153:0/2509304563 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc012430 con 0x7f08e40b2e70 2023-11-28T13:02:00.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 -- 172.21.15.153:0/2509304563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 msgr2=0x7f08e406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 --2- 172.21.15.153:0/2509304563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e406f850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009d90 tx=0x7f08dc00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:00.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 -- 172.21.15.153:0/2509304563 shutdown_connections 2023-11-28T13:02:00.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 --2- 172.21.15.153:0/2509304563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e406f850 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 --2- 172.21.15.153:0/2509304563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e4140950 0x7f08e40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 -- 172.21.15.153:0/2509304563 >> 172.21.15.153:0/2509304563 conn(0x7f08e406aed0 msgr2=0x7f08e406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:00.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 -- 172.21.15.153:0/2509304563 shutdown_connections 2023-11-28T13:02:00.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.087+0000 7f08ea11f700 1 -- 172.21.15.153:0/2509304563 wait complete. 2023-11-28T13:02:00.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.088+0000 7f08ea11f700 1 Processor -- start 2023-11-28T13:02:00.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.088+0000 7f08ea11f700 1 -- start start 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08ea11f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e40c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08ea11f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e40cd8c0 0x7f08e40c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08ea11f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08e4140950 0x7f08e415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08ea11f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40c39a0 con 0x7f08e40b2e70 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08ea11f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4143920 con 0x7f08e4140950 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08ea11f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4143c00 con 0x7f08e40cd8c0 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e40c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e40c0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41196/0 (socket says 172.21.15.153:41196) 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 -- 172.21.15.153:0/1549188055 learned_addr learned my addr 172.21.15.153:0/1549188055 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e3fff700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08e4140950 0x7f08e415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e2ffd700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e40cd8c0 0x7f08e40c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 -- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e40cd8c0 msgr2=0x7f08e40c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e40cd8c0 0x7f08e40c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 -- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08e4140950 msgr2=0x7f08e415cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08e4140950 0x7f08e415cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 -- 172.21.15.153:0/1549188055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc009a70 con 0x7f08e40b2e70 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e3fff700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08e4140950 0x7f08e415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08e37fe700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e40c0b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0010e0 tx=0x7f08cc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08dbfff700 1 -- 172.21.15.153:0/1549188055 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0074e0 con 0x7f08e40b2e70 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08dbfff700 1 -- 172.21.15.153:0/1549188055 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc007640 con 0x7f08e40b2e70 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.089+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e40b35f0 con 0x7f08e40b2e70 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.090+0000 7f08dbfff700 1 -- 172.21.15.153:0/1549188055 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc005a70 con 0x7f08e40b2e70 2023-11-28T13:02:00.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.090+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e40b3310 con 0x7f08e40b2e70 2023-11-28T13:02:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.090+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e4140150 con 0x7f08e40b2e70 2023-11-28T13:02:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.091+0000 7f08dbfff700 1 -- 172.21.15.153:0/1549188055 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f08cc019030 con 0x7f08e40b2e70 2023-11-28T13:02:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.091+0000 7f08dbfff700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08c40682f0 0x7f08c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.091+0000 7f08dbfff700 1 -- 172.21.15.153:0/1549188055 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08cc0850c0 con 0x7f08e40b2e70 2023-11-28T13:02:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.091+0000 7f08e2ffd700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08c40682f0 0x7f08c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.092+0000 7f08e2ffd700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08c40682f0 0x7f08c406a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f08d4006fd0 tx=0x7f08d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.095+0000 7f08dbfff700 1 -- 172.21.15.153:0/1549188055 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f08cc051920 con 0x7f08e40b2e70 2023-11-28T13:02:00.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.321+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08e4068690 con 0x7f08e40b2e70 2023-11-28T13:02:00.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.322+0000 7f08dbfff700 1 -- 172.21.15.153:0/1549188055 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f08cc017090 con 0x7f08e40b2e70 2023-11-28T13:02:00.323 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:00.323 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:00.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08c40682f0 msgr2=0x7f08c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08c40682f0 0x7f08c406a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f08d4006fd0 tx=0x7f08d4008040 comp rx=0 tx=0).stop 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 msgr2=0x7f08e40c0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e40c0b90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0010e0 tx=0x7f08cc00cae0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 shutdown_connections 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08c40682f0 0x7f08c406a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f08e40b2e70 0x7f08e40c0b90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08e4140950 0x7f08e415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 --2- 172.21.15.153:0/1549188055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08e40cd8c0 0x7f08e40c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 >> 172.21.15.153:0/1549188055 conn(0x7f08e406aed0 msgr2=0x7f08e413db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 shutdown_connections 2023-11-28T13:02:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.324+0000 7f08ea11f700 1 -- 172.21.15.153:0/1549188055 wait complete. 2023-11-28T13:02:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:00.337 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.457+0000 7f1f4c693700 1 Processor -- start 2023-11-28T13:02:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.457+0000 7f1f4c693700 1 -- start start 2023-11-28T13:02:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.457+0000 7f1f4c693700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1f4406ba90 0x7f1f440af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.457+0000 7f1f4c693700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1f440cbdf0 0x7f1f440ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4c693700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f440cc5f0 con 0x7f1f4415cef0 2023-11-28T13:02:00.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4c693700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4415d470 con 0x7f1f4406ba90 2023-11-28T13:02:00.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4c693700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4415d750 con 0x7f1f440cbdf0 2023-11-28T13:02:00.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1f4406ba90 0x7f1f440af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1f4406ba90 0x7f1f440af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41206/0 (socket says 172.21.15.153:41206) 2023-11-28T13:02:00.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 -- 172.21.15.153:0/3753462357 learned_addr learned my addr 172.21.15.153:0/3753462357 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:00.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4ac30700 1 --2- 172.21.15.153:0/3753462357 >> v2:172.21.15.177:3300/0 conn(0x7f1f440cbdf0 0x7f1f440ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 -- 172.21.15.153:0/3753462357 >> v2:172.21.15.177:3300/0 conn(0x7f1f440cbdf0 msgr2=0x7f1f440ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 --2- 172.21.15.153:0/3753462357 >> v2:172.21.15.177:3300/0 conn(0x7f1f440cbdf0 0x7f1f440ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 -- 172.21.15.153:0/3753462357 >> v1:172.21.15.177:6789/0 conn(0x7f1f4415cef0 legacy=0x7f1f4406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 -- 172.21.15.153:0/3753462357 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4415da30 con 0x7f1f4406ba90 2023-11-28T13:02:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4ac30700 1 --2- 172.21.15.153:0/3753462357 >> v2:172.21.15.177:3300/0 conn(0x7f1f440cbdf0 0x7f1f440ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:00.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f49c2e700 1 --2- 172.21.15.153:0/3753462357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f440af400 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f44073b10 tx=0x7f1f34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc0b46e6a0b74e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4942d700 1 -- 172.21.15.153:0/3753462357 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400b870 con 0x7f1f4406ba90 2023-11-28T13:02:00.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4942d700 1 -- 172.21.15.153:0/3753462357 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3400b9d0 con 0x7f1f4406ba90 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.458+0000 7f1f4942d700 1 -- 172.21.15.153:0/3753462357 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400bd60 con 0x7f1f4406ba90 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 -- 172.21.15.153:0/3753462357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 msgr2=0x7f1f440af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 --2- 172.21.15.153:0/3753462357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f440af400 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f44073b10 tx=0x7f1f34009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 -- 172.21.15.153:0/3753462357 shutdown_connections 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 --2- 172.21.15.153:0/3753462357 >> v2:172.21.15.177:3300/0 conn(0x7f1f440cbdf0 0x7f1f440ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 --2- 172.21.15.153:0/3753462357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f440af400 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 -- 172.21.15.153:0/3753462357 >> 172.21.15.153:0/3753462357 conn(0x7f1f440d0000 msgr2=0x7f1f440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:00.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 -- 172.21.15.153:0/3753462357 shutdown_connections 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.459+0000 7f1f4c693700 1 -- 172.21.15.153:0/3753462357 wait complete. 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.460+0000 7f1f4c693700 1 Processor -- start 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.460+0000 7f1f4c693700 1 -- start start 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4c693700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f4415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4c693700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f440cbdf0 0x7f1f440c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4c693700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f4415cef0 0x7f1f440af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4c693700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4415b960 con 0x7f1f4406ba90 2023-11-28T13:02:00.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4c693700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c95e0 con 0x7f1f4415cef0 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4c693700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c9020 con 0x7f1f440cbdf0 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f4415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f4415b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41210/0 (socket says 172.21.15.153:41210) 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 -- 172.21.15.153:0/1209186310 learned_addr learned my addr 172.21.15.153:0/1209186310 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4ac30700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f4415cef0 0x7f1f440af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f49c2e700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f440cbdf0 0x7f1f440c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 -- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f440cbdf0 msgr2=0x7f1f440c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f440cbdf0 0x7f1f440c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 -- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f4415cef0 msgr2=0x7f1f440af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f4415cef0 0x7f1f440af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 -- 172.21.15.153:0/1209186310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34009550 con 0x7f1f4406ba90 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f49c2e700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f440cbdf0 0x7f1f440c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4ac30700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f4415cef0 0x7f1f440af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.461+0000 7f1f4a42f700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f4415b420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c450 tx=0x7f1f4000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.462+0000 7f1f3affd700 1 -- 172.21.15.153:0/1209186310 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400052d0 con 0x7f1f4406ba90 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.462+0000 7f1f3affd700 1 -- 172.21.15.153:0/1209186310 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f40005430 con 0x7f1f4406ba90 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.462+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4415dff0 con 0x7f1f4406ba90 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.462+0000 7f1f3affd700 1 -- 172.21.15.153:0/1209186310 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400038c0 con 0x7f1f4406ba90 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.462+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4415dd10 con 0x7f1f4406ba90 2023-11-28T13:02:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.463+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f440cab90 con 0x7f1f4406ba90 2023-11-28T13:02:00.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.463+0000 7f1f3affd700 1 -- 172.21.15.153:0/1209186310 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f40019030 con 0x7f1f4406ba90 2023-11-28T13:02:00.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.463+0000 7f1f3affd700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1f280682f0 0x7f1f2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.463+0000 7f1f3affd700 1 -- 172.21.15.153:0/1209186310 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f40084fc0 con 0x7f1f4406ba90 2023-11-28T13:02:00.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.463+0000 7f1f49c2e700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1f280682f0 0x7f1f2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.464+0000 7f1f49c2e700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1f280682f0 0x7f1f2806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f34000c00 tx=0x7f1f34008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.468+0000 7f1f3affd700 1 -- 172.21.15.153:0/1209186310 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f40051820 con 0x7f1f4406ba90 2023-11-28T13:02:00.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.650+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f440ceda0 con 0x7f1f4406ba90 2023-11-28T13:02:00.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.651+0000 7f1f3affd700 1 -- 172.21.15.153:0/1209186310 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f1f40054e40 con 0x7f1f4406ba90 2023-11-28T13:02:00.652 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:00.653 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":51,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:57.932997+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:54.715348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:55.897576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:00.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1f280682f0 msgr2=0x7f1f2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1f280682f0 0x7f1f2806a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f34000c00 tx=0x7f1f34008030 comp rx=0 tx=0).stop 2023-11-28T13:02:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 msgr2=0x7f1f4415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f4415b420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c450 tx=0x7f1f4000c810 comp rx=0 tx=0).stop 2023-11-28T13:02:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 shutdown_connections 2023-11-28T13:02:00.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1f280682f0 0x7f1f2806a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f4406ba90 0x7f1f4415b420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f4415cef0 0x7f1f440af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 --2- 172.21.15.153:0/1209186310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f440cbdf0 0x7f1f440c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 >> 172.21.15.153:0/1209186310 conn(0x7f1f440d0000 msgr2=0x7f1f4406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:00.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 shutdown_connections 2023-11-28T13:02:00.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.653+0000 7f1f4c693700 1 -- 172.21.15.153:0/1209186310 wait complete. 2023-11-28T13:02:00.667 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:02:00.667 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:02:00.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.786+0000 7f423fa40700 1 Processor -- start 2023-11-28T13:02:00.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.786+0000 7f423fa40700 1 -- start start 2023-11-28T13:02:00.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423fa40700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f423806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423fa40700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4238160540 0x7f42381609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423fa40700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423815d060 con 0x7f4238071040 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423fa40700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423815d340 con 0x7f4238160540 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423fa40700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423815d620 con 0x7f423806dee0 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423dfdd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4238160540 0x7f42381609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423dfdd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4238160540 0x7f42381609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41222/0 (socket says 172.21.15.153:41222) 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423dfdd700 1 -- 172.21.15.153:0/4168101796 learned_addr learned my addr 172.21.15.153:0/4168101796 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423cfdb700 1 --2- 172.21.15.153:0/4168101796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f423806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423dfdd700 1 -- 172.21.15.153:0/4168101796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 msgr2=0x7f423806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423dfdd700 1 --2- 172.21.15.153:0/4168101796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f423806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423dfdd700 1 -- 172.21.15.153:0/4168101796 >> v1:172.21.15.177:6789/0 conn(0x7f4238071040 legacy=0x7f4238071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.787+0000 7f423dfdd700 1 -- 172.21.15.153:0/4168101796 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423815d900 con 0x7f4238160540 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423dfdd700 1 --2- 172.21.15.153:0/4168101796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42381609c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4230009e50 tx=0x7f423000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6977ea58daff4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f422ffff700 1 -- 172.21.15.153:0/4168101796 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000e070 con 0x7f4238160540 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f422ffff700 1 -- 172.21.15.153:0/4168101796 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423000b9c0 con 0x7f4238160540 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f422ffff700 1 -- 172.21.15.153:0/4168101796 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000bd70 con 0x7f4238160540 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 -- 172.21.15.153:0/4168101796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 msgr2=0x7f42381609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 --2- 172.21.15.153:0/4168101796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42381609c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4230009e50 tx=0x7f423000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 -- 172.21.15.153:0/4168101796 shutdown_connections 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 --2- 172.21.15.153:0/4168101796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42381609c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 --2- 172.21.15.153:0/4168101796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f423806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 -- 172.21.15.153:0/4168101796 >> 172.21.15.153:0/4168101796 conn(0x7f42380d0000 msgr2=0x7f42380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:00.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 -- 172.21.15.153:0/4168101796 shutdown_connections 2023-11-28T13:02:00.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.788+0000 7f423fa40700 1 -- 172.21.15.153:0/4168101796 wait complete. 2023-11-28T13:02:00.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 Processor -- start 2023-11-28T13:02:00.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 -- start start 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f42381f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4238071040 0x7f42380c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42380b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42381f3510 con 0x7f4238160540 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42380c6320 con 0x7f4238071040 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423fa40700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42380b3260 con 0x7f423806dee0 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423dfdd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42380b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.790+0000 7f423dfdd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42380b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41228/0 (socket says 172.21.15.153:41228) 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423dfdd700 1 -- 172.21.15.153:0/1043366472 learned_addr learned my addr 172.21.15.153:0/1043366472 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423d7dc700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f42381f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423cfdb700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4238071040 0x7f42380c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423dfdd700 1 -- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 msgr2=0x7f42381f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423dfdd700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f42381f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423dfdd700 1 -- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4238071040 msgr2=0x7f42380c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423dfdd700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4238071040 0x7f42380c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423dfdd700 1 -- 172.21.15.153:0/1043366472 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4230009b30 con 0x7f4238160540 2023-11-28T13:02:00.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423dfdd700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42380b4c30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f423806c7b0 tx=0x7f4230004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f422dffb700 1 -- 172.21.15.153:0/1043366472 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42300047c0 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f422dffb700 1 -- 172.21.15.153:0/1043366472 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42300099b0 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423815ccd0 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f422dffb700 1 -- 172.21.15.153:0/1043366472 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423001b670 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.791+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423806e870 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.792+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423806a270 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.793+0000 7f422dffb700 1 -- 172.21.15.153:0/1043366472 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f423002b070 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.793+0000 7f422dffb700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f421c0682f0 0x7f421c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.793+0000 7f422dffb700 1 -- 172.21.15.153:0/1043366472 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4230087460 con 0x7f4238160540 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.793+0000 7f423d7dc700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f421c0682f0 0x7f421c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:00.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.794+0000 7f423d7dc700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f421c0682f0 0x7f421c06a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4228007fd0 tx=0x7f4228009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:00.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.797+0000 7f422dffb700 1 -- 172.21.15.153:0/1043366472 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4230053bd0 con 0x7f4238160540 2023-11-28T13:02:00.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.978+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42380720a0 con 0x7f421c0682f0 2023-11-28T13:02:00.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.986+0000 7f422dffb700 1 -- 172.21.15.153:0/1043366472 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311122 (secure 0 0 0) 0x7f42380720a0 con 0x7f421c0682f0 2023-11-28T13:02:00.990 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:00.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f421c0682f0 msgr2=0x7f421c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f421c0682f0 0x7f421c06a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4228007fd0 tx=0x7f4228009040 comp rx=0 tx=0).stop 2023-11-28T13:02:00.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 msgr2=0x7f42380b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:00.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42380b4c30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f423806c7b0 tx=0x7f4230004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 shutdown_connections 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f421c0682f0 0x7f421c06a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4238160540 0x7f42380b4c30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423806dee0 0x7f42381f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 --2- 172.21.15.153:0/1043366472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4238071040 0x7f42380c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.989+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 >> 172.21.15.153:0/1043366472 conn(0x7f42380d0000 msgr2=0x7f423815e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.990+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 shutdown_connections 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:00.990+0000 7f423fa40700 1 -- 172.21.15.153:0/1043366472 wait complete. 2023-11-28T13:02:00.994 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:02:01.002 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2023-11-28T13:02:00.168045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78228,"kb_used_data":5780,"kb_used_omap":0,"kb_used_meta":72384,"kb_avail":754896492,"statfs":{"total":773094113280,"available":773014007808,"internally_reserved":0,"allocated":5918720,"data_stored":3187354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74121216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.639257"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328893+0000","last_change":"2023-11-28T13:01:56.955236+0000","last_active":"2023-11-28T13:01:59.328893+0000","last_peered":"2023-11-28T13:01:59.328893+0000","last_clean":"2023-11-28T13:01:59.328893+0000","last_became_active":"2023-11-28T13:01:56.954983+0000","last_became_peered":"2023-11-28T13:01:56.954983+0000","last_unstale":"2023-11-28T13:01:59.328893+0000","last_undegraded":"2023-11-28T13:01:59.328893+0000","last_fullsized":"2023-11-28T13:01:59.328893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:18.308328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966735+0000","last_change":"2023-11-28T13:01:56.964287+0000","last_active":"2023-11-28T13:01:56.966735+0000","last_peered":"2023-11-28T13:01:56.966735+0000","last_clean":"2023-11-28T13:01:56.966735+0000","last_became_active":"2023-11-28T13:01:56.964011+0000","last_became_peered":"2023-11-28T13:01:56.964011+0000","last_unstale":"2023-11-28T13:01:56.966735+0000","last_undegraded":"2023-11-28T13:01:56.966735+0000","last_fullsized":"2023-11-28T13:01:56.966735+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:07.713158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955981+0000","last_change":"2023-11-28T13:01:56.952523+0000","last_active":"2023-11-28T13:01:56.955981+0000","last_peered":"2023-11-28T13:01:56.955981+0000","last_clean":"2023-11-28T13:01:56.955981+0000","last_became_active":"2023-11-28T13:01:56.952277+0000","last_became_peered":"2023-11-28T13:01:56.952277+0000","last_unstale":"2023-11-28T13:01:56.955981+0000","last_undegraded":"2023-11-28T13:01:56.955981+0000","last_fullsized":"2023-11-28T13:01:56.955981+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:31.721247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966236+0000","last_change":"2023-11-28T13:01:56.962603+0000","last_active":"2023-11-28T13:01:56.966236+0000","last_peered":"2023-11-28T13:01:56.966236+0000","last_clean":"2023-11-28T13:01:56.966236+0000","last_became_active":"2023-11-28T13:01:56.962382+0000","last_became_peered":"2023-11-28T13:01:56.962382+0000","last_unstale":"2023-11-28T13:01:56.966236+0000","last_undegraded":"2023-11-28T13:01:56.966236+0000","last_fullsized":"2023-11-28T13:01:56.966236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:53.278587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961536+0000","last_change":"2023-11-28T13:01:56.957872+0000","last_active":"2023-11-28T13:01:56.961536+0000","last_peered":"2023-11-28T13:01:56.961536+0000","last_clean":"2023-11-28T13:01:56.961536+0000","last_became_active":"2023-11-28T13:01:56.957546+0000","last_became_peered":"2023-11-28T13:01:56.957546+0000","last_unstale":"2023-11-28T13:01:56.961536+0000","last_undegraded":"2023-11-28T13:01:56.961536+0000","last_fullsized":"2023-11-28T13:01:56.961536+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:07.713158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328389+0000","last_change":"2023-11-28T13:01:56.956976+0000","last_active":"2023-11-28T13:01:59.328389+0000","last_peered":"2023-11-28T13:01:59.328389+0000","last_clean":"2023-11-28T13:01:59.328389+0000","last_became_active":"2023-11-28T13:01:56.956678+0000","last_became_peered":"2023-11-28T13:01:56.956678+0000","last_unstale":"2023-11-28T13:01:59.328389+0000","last_undegraded":"2023-11-28T13:01:59.328389+0000","last_fullsized":"2023-11-28T13:01:59.328389+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:58.659349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958280+0000","last_change":"2023-11-28T13:01:56.954262+0000","last_active":"2023-11-28T13:01:56.958280+0000","last_peered":"2023-11-28T13:01:56.958280+0000","last_clean":"2023-11-28T13:01:56.958280+0000","last_became_active":"2023-11-28T13:01:56.949293+0000","last_became_peered":"2023-11-28T13:01:56.949293+0000","last_unstale":"2023-11-28T13:01:56.958280+0000","last_undegraded":"2023-11-28T13:01:56.958280+0000","last_fullsized":"2023-11-28T13:01:56.958280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:26.904660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938994+0000","last_change":"2023-11-28T13:01:56.957286+0000","last_active":"2023-11-28T13:01:57.938994+0000","last_peered":"2023-11-28T13:01:57.938994+0000","last_clean":"2023-11-28T13:01:57.938994+0000","last_became_active":"2023-11-28T13:01:56.954543+0000","last_became_peered":"2023-11-28T13:01:56.954543+0000","last_unstale":"2023-11-28T13:01:57.938994+0000","last_undegraded":"2023-11-28T13:01:57.938994+0000","last_fullsized":"2023-11-28T13:01:57.938994+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:27.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939402+0000","last_change":"2023-11-28T13:01:56.954084+0000","last_active":"2023-11-28T13:01:57.939402+0000","last_peered":"2023-11-28T13:01:57.939402+0000","last_clean":"2023-11-28T13:01:57.939402+0000","last_became_active":"2023-11-28T13:01:56.953705+0000","last_became_peered":"2023-11-28T13:01:56.953705+0000","last_unstale":"2023-11-28T13:01:57.939402+0000","last_undegraded":"2023-11-28T13:01:57.939402+0000","last_fullsized":"2023-11-28T13:01:57.939402+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:38.900791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328850+0000","last_change":"2023-11-28T13:01:56.961174+0000","last_active":"2023-11-28T13:01:59.328850+0000","last_peered":"2023-11-28T13:01:59.328850+0000","last_clean":"2023-11-28T13:01:59.328850+0000","last_became_active":"2023-11-28T13:01:56.960912+0000","last_became_peered":"2023-11-28T13:01:56.960912+0000","last_unstale":"2023-11-28T13:01:59.328850+0000","last_undegraded":"2023-11-28T13:01:59.328850+0000","last_fullsized":"2023-11-28T13:01:59.328850+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:28.462737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329138+0000","last_change":"2023-11-28T13:01:56.954116+0000","last_active":"2023-11-28T13:01:59.329138+0000","last_peered":"2023-11-28T13:01:59.329138+0000","last_clean":"2023-11-28T13:01:59.329138+0000","last_became_active":"2023-11-28T13:01:56.953826+0000","last_became_peered":"2023-11-28T13:01:56.953826+0000","last_unstale":"2023-11-28T13:01:59.329138+0000","last_undegraded":"2023-11-28T13:01:59.329138+0000","last_fullsized":"2023-11-28T13:01:59.329138+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:31.623102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939196+0000","last_change":"2023-11-28T13:01:56.954183+0000","last_active":"2023-11-28T13:01:57.939196+0000","last_peered":"2023-11-28T13:01:57.939196+0000","last_clean":"2023-11-28T13:01:57.939196+0000","last_became_active":"2023-11-28T13:01:56.953864+0000","last_became_peered":"2023-11-28T13:01:56.953864+0000","last_unstale":"2023-11-28T13:01:57.939196+0000","last_undegraded":"2023-11-28T13:01:57.939196+0000","last_fullsized":"2023-11-28T13:01:57.939196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:04:23.390277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329127+0000","last_change":"2023-11-28T13:01:56.961359+0000","last_active":"2023-11-28T13:01:59.329127+0000","last_peered":"2023-11-28T13:01:59.329127+0000","last_clean":"2023-11-28T13:01:59.329127+0000","last_became_active":"2023-11-28T13:01:56.960919+0000","last_became_peered":"2023-11-28T13:01:56.960919+0000","last_unstale":"2023-11-28T13:01:59.329127+0000","last_undegraded":"2023-11-28T13:01:59.329127+0000","last_fullsized":"2023-11-28T13:01:59.329127+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:33.988818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961305+0000","last_change":"2023-11-28T13:01:56.956495+0000","last_active":"2023-11-28T13:01:56.961305+0000","last_peered":"2023-11-28T13:01:56.961305+0000","last_clean":"2023-11-28T13:01:56.961305+0000","last_became_active":"2023-11-28T13:01:56.956157+0000","last_became_peered":"2023-11-28T13:01:56.956157+0000","last_unstale":"2023-11-28T13:01:56.961305+0000","last_undegraded":"2023-11-28T13:01:56.961305+0000","last_fullsized":"2023-11-28T13:01:56.961305+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:34:21.421755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963748+0000","last_change":"2023-11-28T13:01:56.960492+0000","last_active":"2023-11-28T13:01:56.963748+0000","last_peered":"2023-11-28T13:01:56.963748+0000","last_clean":"2023-11-28T13:01:56.963748+0000","last_became_active":"2023-11-28T13:01:56.960239+0000","last_became_peered":"2023-11-28T13:01:56.960239+0000","last_unstale":"2023-11-28T13:01:56.963748+0000","last_undegraded":"2023-11-28T13:01:56.963748+0000","last_fullsized":"2023-11-28T13:01:56.963748+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:36.453092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328082+0000","last_change":"2023-11-28T13:01:56.961369+0000","last_active":"2023-11-28T13:01:59.328082+0000","last_peered":"2023-11-28T13:01:59.328082+0000","last_clean":"2023-11-28T13:01:59.328082+0000","last_became_active":"2023-11-28T13:01:56.960911+0000","last_became_peered":"2023-11-28T13:01:56.960911+0000","last_unstale":"2023-11-28T13:01:59.328082+0000","last_undegraded":"2023-11-28T13:01:59.328082+0000","last_fullsized":"2023-11-28T13:01:59.328082+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:20.406868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961524+0000","last_change":"2023-11-28T13:01:56.956431+0000","last_active":"2023-11-28T13:01:56.961524+0000","last_peered":"2023-11-28T13:01:56.961524+0000","last_clean":"2023-11-28T13:01:56.961524+0000","last_became_active":"2023-11-28T13:01:56.955579+0000","last_became_peered":"2023-11-28T13:01:56.955579+0000","last_unstale":"2023-11-28T13:01:56.961524+0000","last_undegraded":"2023-11-28T13:01:56.961524+0000","last_fullsized":"2023-11-28T13:01:56.961524+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:17.982328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.971367+0000","last_change":"2023-11-28T13:01:56.966653+0000","last_active":"2023-11-28T13:01:56.971367+0000","last_peered":"2023-11-28T13:01:56.971367+0000","last_clean":"2023-11-28T13:01:56.971367+0000","last_became_active":"2023-11-28T13:01:56.966395+0000","last_became_peered":"2023-11-28T13:01:56.966395+0000","last_unstale":"2023-11-28T13:01:56.971367+0000","last_undegraded":"2023-11-28T13:01:56.971367+0000","last_fullsized":"2023-11-28T13:01:56.971367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:35.791553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.964176+0000","last_change":"2023-11-28T13:01:56.959599+0000","last_active":"2023-11-28T13:01:56.964176+0000","last_peered":"2023-11-28T13:01:56.964176+0000","last_clean":"2023-11-28T13:01:56.964176+0000","last_became_active":"2023-11-28T13:01:56.959337+0000","last_became_peered":"2023-11-28T13:01:56.959337+0000","last_unstale":"2023-11-28T13:01:56.964176+0000","last_undegraded":"2023-11-28T13:01:56.964176+0000","last_fullsized":"2023-11-28T13:01:56.964176+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:14.309966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939747+0000","last_change":"2023-11-28T13:01:56.960241+0000","last_active":"2023-11-28T13:01:57.939747+0000","last_peered":"2023-11-28T13:01:57.939747+0000","last_clean":"2023-11-28T13:01:57.939747+0000","last_became_active":"2023-11-28T13:01:56.959952+0000","last_became_peered":"2023-11-28T13:01:56.959952+0000","last_unstale":"2023-11-28T13:01:57.939747+0000","last_undegraded":"2023-11-28T13:01:57.939747+0000","last_fullsized":"2023-11-28T13:01:57.939747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:34.740039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939941+0000","last_change":"2023-11-28T13:01:56.953817+0000","last_active":"2023-11-28T13:01:57.939941+0000","last_peered":"2023-11-28T13:01:57.939941+0000","last_clean":"2023-11-28T13:01:57.939941+0000","last_became_active":"2023-11-28T13:01:56.953525+0000","last_became_peered":"2023-11-28T13:01:56.953525+0000","last_unstale":"2023-11-28T13:01:57.939941+0000","last_undegraded":"2023-11-28T13:01:57.939941+0000","last_fullsized":"2023-11-28T13:01:57.939941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:59.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938999+0000","last_change":"2023-11-28T13:01:56.959333+0000","last_active":"2023-11-28T13:01:57.938999+0000","last_peered":"2023-11-28T13:01:57.938999+0000","last_clean":"2023-11-28T13:01:57.938999+0000","last_became_active":"2023-11-28T13:01:56.959092+0000","last_became_peered":"2023-11-28T13:01:56.959092+0000","last_unstale":"2023-11-28T13:01:57.938999+0000","last_undegraded":"2023-11-28T13:01:57.938999+0000","last_fullsized":"2023-11-28T13:01:57.938999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:08.627426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328354+0000","last_change":"2023-11-28T13:01:56.954106+0000","last_active":"2023-11-28T13:01:59.328354+0000","last_peered":"2023-11-28T13:01:59.328354+0000","last_clean":"2023-11-28T13:01:59.328354+0000","last_became_active":"2023-11-28T13:01:56.953426+0000","last_became_peered":"2023-11-28T13:01:56.953426+0000","last_unstale":"2023-11-28T13:01:59.328354+0000","last_undegraded":"2023-11-28T13:01:59.328354+0000","last_fullsized":"2023-11-28T13:01:59.328354+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:02:49.467780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958225+0000","last_change":"2023-11-28T13:01:56.953222+0000","last_active":"2023-11-28T13:01:56.958225+0000","last_peered":"2023-11-28T13:01:56.958225+0000","last_clean":"2023-11-28T13:01:56.958225+0000","last_became_active":"2023-11-28T13:01:56.948890+0000","last_became_peered":"2023-11-28T13:01:56.948890+0000","last_unstale":"2023-11-28T13:01:56.958225+0000","last_undegraded":"2023-11-28T13:01:56.958225+0000","last_fullsized":"2023-11-28T13:01:56.958225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:33.051762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.956323+0000","last_change":"2023-11-28T13:01:56.952413+0000","last_active":"2023-11-28T13:01:56.956323+0000","last_peered":"2023-11-28T13:01:56.956323+0000","last_clean":"2023-11-28T13:01:56.956323+0000","last_became_active":"2023-11-28T13:01:56.952098+0000","last_became_peered":"2023-11-28T13:01:56.952098+0000","last_unstale":"2023-11-28T13:01:56.956323+0000","last_undegraded":"2023-11-28T13:01:56.956323+0000","last_fullsized":"2023-11-28T13:01:56.956323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:20.709046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939876+0000","last_change":"2023-11-28T13:01:56.947344+0000","last_active":"2023-11-28T13:01:57.939876+0000","last_peered":"2023-11-28T13:01:57.939876+0000","last_clean":"2023-11-28T13:01:57.939876+0000","last_became_active":"2023-11-28T13:01:56.947052+0000","last_became_peered":"2023-11-28T13:01:56.947052+0000","last_unstale":"2023-11-28T13:01:57.939876+0000","last_undegraded":"2023-11-28T13:01:57.939876+0000","last_fullsized":"2023-11-28T13:01:57.939876+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:44.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328532+0000","last_change":"2023-11-28T13:01:56.953937+0000","last_active":"2023-11-28T13:01:59.328532+0000","last_peered":"2023-11-28T13:01:59.328532+0000","last_clean":"2023-11-28T13:01:59.328532+0000","last_became_active":"2023-11-28T13:01:56.953492+0000","last_became_peered":"2023-11-28T13:01:56.953492+0000","last_unstale":"2023-11-28T13:01:59.328532+0000","last_undegraded":"2023-11-28T13:01:59.328532+0000","last_fullsized":"2023-11-28T13:01:59.328532+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:55.893471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.940036+0000","last_change":"2023-11-28T13:01:56.953154+0000","last_active":"2023-11-28T13:01:57.940036+0000","last_peered":"2023-11-28T13:01:57.940036+0000","last_clean":"2023-11-28T13:01:57.940036+0000","last_became_active":"2023-11-28T13:01:56.952854+0000","last_became_peered":"2023-11-28T13:01:56.952854+0000","last_unstale":"2023-11-28T13:01:57.940036+0000","last_undegraded":"2023-11-28T13:01:57.940036+0000","last_fullsized":"2023-11-28T13:01:57.940036+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:30.257699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958256+0000","last_change":"2023-11-28T13:01:56.953036+0000","last_active":"2023-11-28T13:01:56.958256+0000","last_peered":"2023-11-28T13:01:56.958256+0000","last_clean":"2023-11-28T13:01:56.958256+0000","last_became_active":"2023-11-28T13:01:56.952752+0000","last_became_peered":"2023-11-28T13:01:56.952752+0000","last_unstale":"2023-11-28T13:01:56.958256+0000","last_undegraded":"2023-11-28T13:01:56.958256+0000","last_fullsized":"2023-11-28T13:01:56.958256+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:28.471084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963719+0000","last_change":"2023-11-28T13:01:56.959862+0000","last_active":"2023-11-28T13:01:56.963719+0000","last_peered":"2023-11-28T13:01:56.963719+0000","last_clean":"2023-11-28T13:01:56.963719+0000","last_became_active":"2023-11-28T13:01:56.959597+0000","last_became_peered":"2023-11-28T13:01:56.959597+0000","last_unstale":"2023-11-28T13:01:56.963719+0000","last_undegraded":"2023-11-28T13:01:56.963719+0000","last_fullsized":"2023-11-28T13:01:56.963719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:08:45.732739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955898+0000","last_change":"2023-11-28T13:01:56.949530+0000","last_active":"2023-11-28T13:01:56.955898+0000","last_peered":"2023-11-28T13:01:56.955898+0000","last_clean":"2023-11-28T13:01:56.955898+0000","last_became_active":"2023-11-28T13:01:56.949009+0000","last_became_peered":"2023-11-28T13:01:56.949009+0000","last_unstale":"2023-11-28T13:01:56.955898+0000","last_undegraded":"2023-11-28T13:01:56.955898+0000","last_fullsized":"2023-11-28T13:01:56.955898+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:06:06.562735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938984+0000","last_change":"2023-11-28T13:01:56.950806+0000","last_active":"2023-11-28T13:01:57.938984+0000","last_peered":"2023-11-28T13:01:57.938984+0000","last_clean":"2023-11-28T13:01:57.938984+0000","last_became_active":"2023-11-28T13:01:56.950562+0000","last_became_peered":"2023-11-28T13:01:56.950562+0000","last_unstale":"2023-11-28T13:01:57.938984+0000","last_undegraded":"2023-11-28T13:01:57.938984+0000","last_fullsized":"2023-11-28T13:01:57.938984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:01.873986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963586+0000","last_change":"2023-11-28T13:01:56.955782+0000","last_active":"2023-11-28T13:01:56.963586+0000","last_peered":"2023-11-28T13:01:56.963586+0000","last_clean":"2023-11-28T13:01:56.963586+0000","last_became_active":"2023-11-28T13:01:56.955500+0000","last_became_peered":"2023-11-28T13:01:56.955500+0000","last_unstale":"2023-11-28T13:01:56.963586+0000","last_undegraded":"2023-11-28T13:01:56.963586+0000","last_fullsized":"2023-11-28T13:01:56.963586+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:01.873986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939696+0000","last_change":"2023-11-28T13:01:56.546817+0000","last_active":"2023-11-28T13:01:57.939696+0000","last_peered":"2023-11-28T13:01:57.939696+0000","last_clean":"2023-11-28T13:01:57.939696+0000","last_became_active":"2023-11-28T13:01:56.546488+0000","last_became_peered":"2023-11-28T13:01:56.546488+0000","last_unstale":"2023-11-28T13:01:57.939696+0000","last_undegraded":"2023-11-28T13:01:57.939696+0000","last_fullsized":"2023-11-28T13:01:57.939696+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:43.975011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938820+0000","last_change":"2023-11-28T13:01:56.957111+0000","last_active":"2023-11-28T13:01:57.938820+0000","last_peered":"2023-11-28T13:01:57.938820+0000","last_clean":"2023-11-28T13:01:57.938820+0000","last_became_active":"2023-11-28T13:01:56.956783+0000","last_became_peered":"2023-11-28T13:01:56.956783+0000","last_unstale":"2023-11-28T13:01:57.938820+0000","last_undegraded":"2023-11-28T13:01:57.938820+0000","last_fullsized":"2023-11-28T13:01:57.938820+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:17.780940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328939+0000","last_change":"2023-11-28T13:01:56.547728+0000","last_active":"2023-11-28T13:01:59.328939+0000","last_peered":"2023-11-28T13:01:59.328939+0000","last_clean":"2023-11-28T13:01:59.328939+0000","last_became_active":"2023-11-28T13:01:56.547464+0000","last_became_peered":"2023-11-28T13:01:56.547464+0000","last_unstale":"2023-11-28T13:01:59.328939+0000","last_undegraded":"2023-11-28T13:01:59.328939+0000","last_fullsized":"2023-11-28T13:01:59.328939+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:26.512099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939714+0000","last_change":"2023-11-28T13:01:56.955433+0000","last_active":"2023-11-28T13:01:57.939714+0000","last_peered":"2023-11-28T13:01:57.939714+0000","last_clean":"2023-11-28T13:01:57.939714+0000","last_became_active":"2023-11-28T13:01:56.955154+0000","last_became_peered":"2023-11-28T13:01:56.955154+0000","last_unstale":"2023-11-28T13:01:57.939714+0000","last_undegraded":"2023-11-28T13:01:57.939714+0000","last_fullsized":"2023-11-28T13:01:57.939714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:16.870674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328693+0000","last_change":"2023-11-28T13:01:56.553637+0000","last_active":"2023-11-28T13:01:59.328693+0000","last_peered":"2023-11-28T13:01:59.328693+0000","last_clean":"2023-11-28T13:01:59.328693+0000","last_became_active":"2023-11-28T13:01:56.550191+0000","last_became_peered":"2023-11-28T13:01:56.550191+0000","last_unstale":"2023-11-28T13:01:59.328693+0000","last_undegraded":"2023-11-28T13:01:59.328693+0000","last_fullsized":"2023-11-28T13:01:59.328693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:43.641338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939119+0000","last_change":"2023-11-28T13:01:56.953148+0000","last_active":"2023-11-28T13:01:57.939119+0000","last_peered":"2023-11-28T13:01:57.939119+0000","last_clean":"2023-11-28T13:01:57.939119+0000","last_became_active":"2023-11-28T13:01:56.952892+0000","last_became_peered":"2023-11-28T13:01:56.952892+0000","last_unstale":"2023-11-28T13:01:57.939119+0000","last_undegraded":"2023-11-28T13:01:57.939119+0000","last_fullsized":"2023-11-28T13:01:57.939119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:09.970320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939077+0000","last_change":"2023-11-28T13:01:56.543766+0000","last_active":"2023-11-28T13:01:57.939077+0000","last_peered":"2023-11-28T13:01:57.939077+0000","last_clean":"2023-11-28T13:01:57.939077+0000","last_became_active":"2023-11-28T13:01:56.543472+0000","last_became_peered":"2023-11-28T13:01:56.543472+0000","last_unstale":"2023-11-28T13:01:57.939077+0000","last_undegraded":"2023-11-28T13:01:57.939077+0000","last_fullsized":"2023-11-28T13:01:57.939077+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:02.634978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963213+0000","last_change":"2023-11-28T13:01:56.958663+0000","last_active":"2023-11-28T13:01:56.963213+0000","last_peered":"2023-11-28T13:01:56.963213+0000","last_clean":"2023-11-28T13:01:56.963213+0000","last_became_active":"2023-11-28T13:01:56.958394+0000","last_became_peered":"2023-11-28T13:01:56.958394+0000","last_unstale":"2023-11-28T13:01:56.963213+0000","last_undegraded":"2023-11-28T13:01:56.963213+0000","last_fullsized":"2023-11-28T13:01:56.963213+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:05.390821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.952380+0000","last_change":"2023-11-28T13:01:56.554087+0000","last_active":"2023-11-28T13:01:56.952380+0000","last_peered":"2023-11-28T13:01:56.952380+0000","last_clean":"2023-11-28T13:01:56.952380+0000","last_became_active":"2023-11-28T13:01:56.553109+0000","last_became_peered":"2023-11-28T13:01:56.553109+0000","last_unstale":"2023-11-28T13:01:56.952380+0000","last_undegraded":"2023-11-28T13:01:56.952380+0000","last_fullsized":"2023-11-28T13:01:56.952380+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:52.238893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966173+0000","last_change":"2023-11-28T13:01:56.961748+0000","last_active":"2023-11-28T13:01:56.966173+0000","last_peered":"2023-11-28T13:01:56.966173+0000","last_clean":"2023-11-28T13:01:56.966173+0000","last_became_active":"2023-11-28T13:01:56.961497+0000","last_became_peered":"2023-11-28T13:01:56.961497+0000","last_unstale":"2023-11-28T13:01:56.966173+0000","last_undegraded":"2023-11-28T13:01:56.966173+0000","last_fullsized":"2023-11-28T13:01:56.966173+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.243375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958285+0000","last_change":"2023-11-28T13:01:56.547723+0000","last_active":"2023-11-28T13:01:56.958285+0000","last_peered":"2023-11-28T13:01:56.958285+0000","last_clean":"2023-11-28T13:01:56.958285+0000","last_became_active":"2023-11-28T13:01:56.547451+0000","last_became_peered":"2023-11-28T13:01:56.547451+0000","last_unstale":"2023-11-28T13:01:56.958285+0000","last_undegraded":"2023-11-28T13:01:56.958285+0000","last_fullsized":"2023-11-28T13:01:56.958285+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:18.107890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939781+0000","last_change":"2023-11-28T13:01:56.950630+0000","last_active":"2023-11-28T13:01:57.939781+0000","last_peered":"2023-11-28T13:01:57.939781+0000","last_clean":"2023-11-28T13:01:57.939781+0000","last_became_active":"2023-11-28T13:01:56.950182+0000","last_became_peered":"2023-11-28T13:01:56.950182+0000","last_unstale":"2023-11-28T13:01:57.939781+0000","last_undegraded":"2023-11-28T13:01:57.939781+0000","last_fullsized":"2023-11-28T13:01:57.939781+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:20.762287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939530+0000","last_change":"2023-11-28T13:01:56.549973+0000","last_active":"2023-11-28T13:01:57.939530+0000","last_peered":"2023-11-28T13:01:57.939530+0000","last_clean":"2023-11-28T13:01:57.939530+0000","last_became_active":"2023-11-28T13:01:56.549717+0000","last_became_peered":"2023-11-28T13:01:56.549717+0000","last_unstale":"2023-11-28T13:01:57.939530+0000","last_undegraded":"2023-11-28T13:01:57.939530+0000","last_fullsized":"2023-11-28T13:01:57.939530+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:17.503589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966112+0000","last_change":"2023-11-28T13:01:56.961126+0000","last_active":"2023-11-28T13:01:56.966112+0000","last_peered":"2023-11-28T13:01:56.966112+0000","last_clean":"2023-11-28T13:01:56.966112+0000","last_became_active":"2023-11-28T13:01:56.960831+0000","last_became_peered":"2023-11-28T13:01:56.960831+0000","last_unstale":"2023-11-28T13:01:56.966112+0000","last_undegraded":"2023-11-28T13:01:56.966112+0000","last_fullsized":"2023-11-28T13:01:56.966112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:05.390821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.939498+0000","last_change":"2023-11-28T13:01:56.556138+0000","last_active":"2023-11-28T13:01:56.939498+0000","last_peered":"2023-11-28T13:01:56.939498+0000","last_clean":"2023-11-28T13:01:56.939498+0000","last_became_active":"2023-11-28T13:01:56.555815+0000","last_became_peered":"2023-11-28T13:01:56.555815+0000","last_unstale":"2023-11-28T13:01:56.939498+0000","last_undegraded":"2023-11-28T13:01:56.939498+0000","last_fullsized":"2023-11-28T13:01:56.939498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:02.634978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958929+0000","last_change":"2023-11-28T13:01:56.956344+0000","last_active":"2023-11-28T13:01:56.958929+0000","last_peered":"2023-11-28T13:01:56.958929+0000","last_clean":"2023-11-28T13:01:56.958929+0000","last_became_active":"2023-11-28T13:01:56.956051+0000","last_became_peered":"2023-11-28T13:01:56.956051+0000","last_unstale":"2023-11-28T13:01:56.958929+0000","last_undegraded":"2023-11-28T13:01:56.958929+0000","last_fullsized":"2023-11-28T13:01:56.958929+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.243375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.952091+0000","last_change":"2023-11-28T13:01:56.553211+0000","last_active":"2023-11-28T13:01:56.952091+0000","last_peered":"2023-11-28T13:01:56.952091+0000","last_clean":"2023-11-28T13:01:56.952091+0000","last_became_active":"2023-11-28T13:01:56.552933+0000","last_became_peered":"2023-11-28T13:01:56.552933+0000","last_unstale":"2023-11-28T13:01:56.952091+0000","last_undegraded":"2023-11-28T13:01:56.952091+0000","last_fullsized":"2023-11-28T13:01:56.952091+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:33.734479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938701+0000","last_change":"2023-11-28T13:01:56.959079+0000","last_active":"2023-11-28T13:01:57.938701+0000","last_peered":"2023-11-28T13:01:57.938701+0000","last_clean":"2023-11-28T13:01:57.938701+0000","last_became_active":"2023-11-28T13:01:56.958778+0000","last_became_peered":"2023-11-28T13:01:56.958778+0000","last_unstale":"2023-11-28T13:01:57.938701+0000","last_undegraded":"2023-11-28T13:01:57.938701+0000","last_fullsized":"2023-11-28T13:01:57.938701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:05.390821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.947102+0000","last_change":"2023-11-28T13:01:56.551775+0000","last_active":"2023-11-28T13:01:56.947102+0000","last_peered":"2023-11-28T13:01:56.947102+0000","last_clean":"2023-11-28T13:01:56.947102+0000","last_became_active":"2023-11-28T13:01:56.551487+0000","last_became_peered":"2023-11-28T13:01:56.551487+0000","last_unstale":"2023-11-28T13:01:56.947102+0000","last_undegraded":"2023-11-28T13:01:56.947102+0000","last_fullsized":"2023-11-28T13:01:56.947102+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:53.578242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961285+0000","last_change":"2023-11-28T13:01:56.955950+0000","last_active":"2023-11-28T13:01:56.961285+0000","last_peered":"2023-11-28T13:01:56.961285+0000","last_clean":"2023-11-28T13:01:56.961285+0000","last_became_active":"2023-11-28T13:01:56.955695+0000","last_became_peered":"2023-11-28T13:01:56.955695+0000","last_unstale":"2023-11-28T13:01:56.961285+0000","last_undegraded":"2023-11-28T13:01:56.961285+0000","last_fullsized":"2023-11-28T13:01:56.961285+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:19.476807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963619+0000","last_change":"2023-11-28T13:01:56.555586+0000","last_active":"2023-11-28T13:01:56.963619+0000","last_peered":"2023-11-28T13:01:56.963619+0000","last_clean":"2023-11-28T13:01:56.963619+0000","last_became_active":"2023-11-28T13:01:56.555280+0000","last_became_peered":"2023-11-28T13:01:56.555280+0000","last_unstale":"2023-11-28T13:01:56.963619+0000","last_undegraded":"2023-11-28T13:01:56.963619+0000","last_fullsized":"2023-11-28T13:01:56.963619+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:15.865114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961514+0000","last_change":"2023-11-28T13:01:56.957853+0000","last_active":"2023-11-28T13:01:56.961514+0000","last_peered":"2023-11-28T13:01:56.961514+0000","last_clean":"2023-11-28T13:01:56.961514+0000","last_became_active":"2023-11-28T13:01:56.957504+0000","last_became_peered":"2023-11-28T13:01:56.957504+0000","last_unstale":"2023-11-28T13:01:56.961514+0000","last_undegraded":"2023-11-28T13:01:56.961514+0000","last_fullsized":"2023-11-28T13:01:56.961514+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:01.873986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939226+0000","last_change":"2023-11-28T13:01:56.547924+0000","last_active":"2023-11-28T13:01:57.939226+0000","last_peered":"2023-11-28T13:01:57.939226+0000","last_clean":"2023-11-28T13:01:57.939226+0000","last_became_active":"2023-11-28T13:01:56.547584+0000","last_became_peered":"2023-11-28T13:01:56.547584+0000","last_unstale":"2023-11-28T13:01:57.939226+0000","last_undegraded":"2023-11-28T13:01:57.939226+0000","last_fullsized":"2023-11-28T13:01:57.939226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:59.635818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938967+0000","last_change":"2023-11-28T13:01:56.955534+0000","last_active":"2023-11-28T13:01:57.938967+0000","last_peered":"2023-11-28T13:01:57.938967+0000","last_clean":"2023-11-28T13:01:57.938967+0000","last_became_active":"2023-11-28T13:01:56.955324+0000","last_became_peered":"2023-11-28T13:01:56.955324+0000","last_unstale":"2023-11-28T13:01:57.938967+0000","last_undegraded":"2023-11-28T13:01:57.938967+0000","last_fullsized":"2023-11-28T13:01:57.938967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:51.796815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.957702+0000","last_change":"2023-11-28T13:01:56.551353+0000","last_active":"2023-11-28T13:01:56.957702+0000","last_peered":"2023-11-28T13:01:56.957702+0000","last_clean":"2023-11-28T13:01:56.957702+0000","last_became_active":"2023-11-28T13:01:56.551052+0000","last_became_peered":"2023-11-28T13:01:56.551052+0000","last_unstale":"2023-11-28T13:01:56.957702+0000","last_undegraded":"2023-11-28T13:01:56.957702+0000","last_fullsized":"2023-11-28T13:01:56.957702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:21.743224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955961+0000","last_change":"2023-11-28T13:01:56.952020+0000","last_active":"2023-11-28T13:01:56.955961+0000","last_peered":"2023-11-28T13:01:56.955961+0000","last_clean":"2023-11-28T13:01:56.955961+0000","last_became_active":"2023-11-28T13:01:56.948677+0000","last_became_peered":"2023-11-28T13:01:56.948677+0000","last_unstale":"2023-11-28T13:01:56.955961+0000","last_undegraded":"2023-11-28T13:01:56.955961+0000","last_fullsized":"2023-11-28T13:01:56.955961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:07.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939658+0000","last_change":"2023-11-28T13:01:56.552019+0000","last_active":"2023-11-28T13:01:57.939658+0000","last_peered":"2023-11-28T13:01:57.939658+0000","last_clean":"2023-11-28T13:01:57.939658+0000","last_became_active":"2023-11-28T13:01:56.551752+0000","last_became_peered":"2023-11-28T13:01:56.551752+0000","last_unstale":"2023-11-28T13:01:57.939658+0000","last_undegraded":"2023-11-28T13:01:57.939658+0000","last_fullsized":"2023-11-28T13:01:57.939658+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:20.944063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939687+0000","last_change":"2023-11-28T13:01:56.957281+0000","last_active":"2023-11-28T13:01:57.939687+0000","last_peered":"2023-11-28T13:01:57.939687+0000","last_clean":"2023-11-28T13:01:57.939687+0000","last_became_active":"2023-11-28T13:01:56.956996+0000","last_became_peered":"2023-11-28T13:01:56.956996+0000","last_unstale":"2023-11-28T13:01:57.939687+0000","last_undegraded":"2023-11-28T13:01:57.939687+0000","last_fullsized":"2023-11-28T13:01:57.939687+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:03.801800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.952603+0000","last_change":"2023-11-28T13:01:56.553248+0000","last_active":"2023-11-28T13:01:56.952603+0000","last_peered":"2023-11-28T13:01:56.952603+0000","last_clean":"2023-11-28T13:01:56.952603+0000","last_became_active":"2023-11-28T13:01:56.552959+0000","last_became_peered":"2023-11-28T13:01:56.552959+0000","last_unstale":"2023-11-28T13:01:56.952603+0000","last_undegraded":"2023-11-28T13:01:56.952603+0000","last_fullsized":"2023-11-28T13:01:56.952603+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:40.691336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961504+0000","last_change":"2023-11-28T13:01:56.956195+0000","last_active":"2023-11-28T13:01:56.961504+0000","last_peered":"2023-11-28T13:01:56.961504+0000","last_clean":"2023-11-28T13:01:56.961504+0000","last_became_active":"2023-11-28T13:01:56.955250+0000","last_became_peered":"2023-11-28T13:01:56.955250+0000","last_unstale":"2023-11-28T13:01:56.961504+0000","last_undegraded":"2023-11-28T13:01:56.961504+0000","last_fullsized":"2023-11-28T13:01:56.961504+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:09.136546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939177+0000","last_change":"2023-11-28T13:01:56.548096+0000","last_active":"2023-11-28T13:01:57.939177+0000","last_peered":"2023-11-28T13:01:57.939177+0000","last_clean":"2023-11-28T13:01:57.939177+0000","last_became_active":"2023-11-28T13:01:56.547764+0000","last_became_peered":"2023-11-28T13:01:56.547764+0000","last_unstale":"2023-11-28T13:01:57.939177+0000","last_undegraded":"2023-11-28T13:01:57.939177+0000","last_fullsized":"2023-11-28T13:01:57.939177+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:24.575404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939019+0000","last_change":"2023-11-28T13:01:56.957561+0000","last_active":"2023-11-28T13:01:57.939019+0000","last_peered":"2023-11-28T13:01:57.939019+0000","last_clean":"2023-11-28T13:01:57.939019+0000","last_became_active":"2023-11-28T13:01:56.957035+0000","last_became_peered":"2023-11-28T13:01:56.957035+0000","last_unstale":"2023-11-28T13:01:57.939019+0000","last_undegraded":"2023-11-28T13:01:57.939019+0000","last_fullsized":"2023-11-28T13:01:57.939019+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:11.705221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958899+0000","last_change":"2023-11-28T13:01:56.557848+0000","last_active":"2023-11-28T13:01:56.958899+0000","last_peered":"2023-11-28T13:01:56.958899+0000","last_clean":"2023-11-28T13:01:56.958899+0000","last_became_active":"2023-11-28T13:01:56.557561+0000","last_became_peered":"2023-11-28T13:01:56.557561+0000","last_unstale":"2023-11-28T13:01:56.958899+0000","last_undegraded":"2023-11-28T13:01:56.958899+0000","last_fullsized":"2023-11-28T13:01:56.958899+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:52.273027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328286+0000","last_change":"2023-11-28T13:01:56.961601+0000","last_active":"2023-11-28T13:01:59.328286+0000","last_peered":"2023-11-28T13:01:59.328286+0000","last_clean":"2023-11-28T13:01:59.328286+0000","last_became_active":"2023-11-28T13:01:56.961349+0000","last_became_peered":"2023-11-28T13:01:56.961349+0000","last_unstale":"2023-11-28T13:01:59.328286+0000","last_undegraded":"2023-11-28T13:01:59.328286+0000","last_fullsized":"2023-11-28T13:01:59.328286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:57.740216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329218+0000","last_change":"2023-11-28T13:01:56.544596+0000","last_active":"2023-11-28T13:01:59.329218+0000","last_peered":"2023-11-28T13:01:59.329218+0000","last_clean":"2023-11-28T13:01:59.329218+0000","last_became_active":"2023-11-28T13:01:56.544304+0000","last_became_peered":"2023-11-28T13:01:56.544304+0000","last_unstale":"2023-11-28T13:01:59.329218+0000","last_undegraded":"2023-11-28T13:01:59.329218+0000","last_fullsized":"2023-11-28T13:01:59.329218+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:22.175750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963227+0000","last_change":"2023-11-28T13:01:56.958793+0000","last_active":"2023-11-28T13:01:56.963227+0000","last_peered":"2023-11-28T13:01:56.963227+0000","last_clean":"2023-11-28T13:01:56.963227+0000","last_became_active":"2023-11-28T13:01:56.958502+0000","last_became_peered":"2023-11-28T13:01:56.958502+0000","last_unstale":"2023-11-28T13:01:56.963227+0000","last_undegraded":"2023-11-28T13:01:56.963227+0000","last_fullsized":"2023-11-28T13:01:56.963227+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:05.755474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939935+0000","last_change":"2023-11-28T13:01:56.554581+0000","last_active":"2023-11-28T13:01:57.939935+0000","last_peered":"2023-11-28T13:01:57.939935+0000","last_clean":"2023-11-28T13:01:57.939935+0000","last_became_active":"2023-11-28T13:01:56.554153+0000","last_became_peered":"2023-11-28T13:01:56.554153+0000","last_unstale":"2023-11-28T13:01:57.939935+0000","last_undegraded":"2023-11-28T13:01:57.939935+0000","last_fullsized":"2023-11-28T13:01:57.939935+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:38.731848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329318+0000","last_change":"2023-11-28T13:01:56.955539+0000","last_active":"2023-11-28T13:01:59.329318+0000","last_peered":"2023-11-28T13:01:59.329318+0000","last_clean":"2023-11-28T13:01:59.329318+0000","last_became_active":"2023-11-28T13:01:56.955271+0000","last_became_peered":"2023-11-28T13:01:56.955271+0000","last_unstale":"2023-11-28T13:01:59.329318+0000","last_undegraded":"2023-11-28T13:01:59.329318+0000","last_fullsized":"2023-11-28T13:01:59.329318+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:41:44.811615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.940003+0000","last_change":"2023-11-28T13:01:56.547245+0000","last_active":"2023-11-28T13:01:57.940003+0000","last_peered":"2023-11-28T13:01:57.940003+0000","last_clean":"2023-11-28T13:01:57.940003+0000","last_became_active":"2023-11-28T13:01:56.546961+0000","last_became_peered":"2023-11-28T13:01:56.546961+0000","last_unstale":"2023-11-28T13:01:57.940003+0000","last_undegraded":"2023-11-28T13:01:57.940003+0000","last_fullsized":"2023-11-28T13:01:57.940003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:43.494941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938719+0000","last_change":"2023-11-28T13:01:56.961235+0000","last_active":"2023-11-28T13:01:57.938719+0000","last_peered":"2023-11-28T13:01:57.938719+0000","last_clean":"2023-11-28T13:01:57.938719+0000","last_became_active":"2023-11-28T13:01:56.960977+0000","last_became_peered":"2023-11-28T13:01:56.960977+0000","last_unstale":"2023-11-28T13:01:57.938719+0000","last_undegraded":"2023-11-28T13:01:57.938719+0000","last_fullsized":"2023-11-28T13:01:57.938719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:07.713158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329225+0000","last_change":"2023-11-28T13:01:56.553232+0000","last_active":"2023-11-28T13:01:59.329225+0000","last_peered":"2023-11-28T13:01:59.329225+0000","last_clean":"2023-11-28T13:01:59.329225+0000","last_became_active":"2023-11-28T13:01:56.552959+0000","last_became_peered":"2023-11-28T13:01:56.552959+0000","last_unstale":"2023-11-28T13:01:59.329225+0000","last_undegraded":"2023-11-28T13:01:59.329225+0000","last_fullsized":"2023-11-28T13:01:59.329225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:04:31.186841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939851+0000","last_change":"2023-11-28T13:01:56.959192+0000","last_active":"2023-11-28T13:01:57.939851+0000","last_peered":"2023-11-28T13:01:57.939851+0000","last_clean":"2023-11-28T13:01:57.939851+0000","last_became_active":"2023-11-28T13:01:56.958904+0000","last_became_peered":"2023-11-28T13:01:56.958904+0000","last_unstale":"2023-11-28T13:01:57.939851+0000","last_undegraded":"2023-11-28T13:01:57.939851+0000","last_fullsized":"2023-11-28T13:01:57.939851+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:41.696896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939060+0000","last_change":"2023-11-28T13:01:56.548479+0000","last_active":"2023-11-28T13:01:57.939060+0000","last_peered":"2023-11-28T13:01:57.939060+0000","last_clean":"2023-11-28T13:01:57.939060+0000","last_became_active":"2023-11-28T13:01:56.548230+0000","last_became_peered":"2023-11-28T13:01:56.548230+0000","last_unstale":"2023-11-28T13:01:57.939060+0000","last_undegraded":"2023-11-28T13:01:57.939060+0000","last_fullsized":"2023-11-28T13:01:57.939060+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:27.457178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958309+0000","last_change":"2023-11-28T13:01:56.953065+0000","last_active":"2023-11-28T13:01:56.958309+0000","last_peered":"2023-11-28T13:01:56.958309+0000","last_clean":"2023-11-28T13:01:56.958309+0000","last_became_active":"2023-11-28T13:01:56.952812+0000","last_became_peered":"2023-11-28T13:01:56.952812+0000","last_unstale":"2023-11-28T13:01:56.958309+0000","last_undegraded":"2023-11-28T13:01:56.958309+0000","last_fullsized":"2023-11-28T13:01:56.958309+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:44.202342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945091+0000","last_change":"2023-11-28T13:01:56.557972+0000","last_active":"2023-11-28T13:01:56.945091+0000","last_peered":"2023-11-28T13:01:56.945091+0000","last_clean":"2023-11-28T13:01:56.945091+0000","last_became_active":"2023-11-28T13:01:56.557694+0000","last_became_peered":"2023-11-28T13:01:56.557694+0000","last_unstale":"2023-11-28T13:01:56.945091+0000","last_undegraded":"2023-11-28T13:01:56.945091+0000","last_fullsized":"2023-11-28T13:01:56.945091+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:56.755448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328988+0000","last_change":"2023-11-28T13:01:56.949449+0000","last_active":"2023-11-28T13:01:59.328988+0000","last_peered":"2023-11-28T13:01:59.328988+0000","last_clean":"2023-11-28T13:01:59.328988+0000","last_became_active":"2023-11-28T13:01:56.949158+0000","last_became_peered":"2023-11-28T13:01:56.949158+0000","last_unstale":"2023-11-28T13:01:59.328988+0000","last_undegraded":"2023-11-28T13:01:59.328988+0000","last_fullsized":"2023-11-28T13:01:59.328988+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:25.580964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329436+0000","last_change":"2023-11-28T13:01:56.550418+0000","last_active":"2023-11-28T13:01:59.329436+0000","last_peered":"2023-11-28T13:01:59.329436+0000","last_clean":"2023-11-28T13:01:59.329436+0000","last_became_active":"2023-11-28T13:01:56.550121+0000","last_became_peered":"2023-11-28T13:01:56.550121+0000","last_unstale":"2023-11-28T13:01:59.329436+0000","last_undegraded":"2023-11-28T13:01:59.329436+0000","last_fullsized":"2023-11-28T13:01:59.329436+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:25.719801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.956008+0000","last_change":"2023-11-28T13:01:56.948792+0000","last_active":"2023-11-28T13:01:56.956008+0000","last_peered":"2023-11-28T13:01:56.956008+0000","last_clean":"2023-11-28T13:01:56.956008+0000","last_became_active":"2023-11-28T13:01:56.948483+0000","last_became_peered":"2023-11-28T13:01:56.948483+0000","last_unstale":"2023-11-28T13:01:56.956008+0000","last_undegraded":"2023-11-28T13:01:56.956008+0000","last_fullsized":"2023-11-28T13:01:56.956008+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:43.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945673+0000","last_change":"2023-11-28T13:01:56.547024+0000","last_active":"2023-11-28T13:01:56.945673+0000","last_peered":"2023-11-28T13:01:56.945673+0000","last_clean":"2023-11-28T13:01:56.945673+0000","last_became_active":"2023-11-28T13:01:56.546734+0000","last_became_peered":"2023-11-28T13:01:56.546734+0000","last_unstale":"2023-11-28T13:01:56.945673+0000","last_undegraded":"2023-11-28T13:01:56.945673+0000","last_fullsized":"2023-11-28T13:01:56.945673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:21.743224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.935255+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:01:56.935255+0000","last_peered":"2023-11-28T13:01:56.935255+0000","last_clean":"2023-11-28T13:01:56.935255+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:01:56.935255+0000","last_undegraded":"2023-11-28T13:01:56.935255+0000","last_fullsized":"2023-11-28T13:01:56.935255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938770+0000","last_change":"2023-11-28T13:01:56.960019+0000","last_active":"2023-11-28T13:01:57.938770+0000","last_peered":"2023-11-28T13:01:57.938770+0000","last_clean":"2023-11-28T13:01:57.938770+0000","last_became_active":"2023-11-28T13:01:56.959748+0000","last_became_peered":"2023-11-28T13:01:56.959748+0000","last_unstale":"2023-11-28T13:01:57.938770+0000","last_undegraded":"2023-11-28T13:01:57.938770+0000","last_fullsized":"2023-11-28T13:01:57.938770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.243375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939796+0000","last_change":"2023-11-28T13:01:56.551638+0000","last_active":"2023-11-28T13:01:57.939796+0000","last_peered":"2023-11-28T13:01:57.939796+0000","last_clean":"2023-11-28T13:01:57.939796+0000","last_became_active":"2023-11-28T13:01:56.551278+0000","last_became_peered":"2023-11-28T13:01:56.551278+0000","last_unstale":"2023-11-28T13:01:57.939796+0000","last_undegraded":"2023-11-28T13:01:57.939796+0000","last_fullsized":"2023-11-28T13:01:57.939796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:36.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945115+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:01:56.945115+0000","last_peered":"2023-11-28T13:01:56.945115+0000","last_clean":"2023-11-28T13:01:56.945115+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:01:56.945115+0000","last_undegraded":"2023-11-28T13:01:56.945115+0000","last_fullsized":"2023-11-28T13:01:56.945115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939189+0000","last_change":"2023-11-28T13:01:56.959427+0000","last_active":"2023-11-28T13:01:57.939189+0000","last_peered":"2023-11-28T13:01:57.939189+0000","last_clean":"2023-11-28T13:01:57.939189+0000","last_became_active":"2023-11-28T13:01:56.959197+0000","last_became_peered":"2023-11-28T13:01:56.959197+0000","last_unstale":"2023-11-28T13:01:57.939189+0000","last_undegraded":"2023-11-28T13:01:57.939189+0000","last_fullsized":"2023-11-28T13:01:57.939189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:06.623917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.939708+0000","last_change":"2023-11-28T13:01:56.553782+0000","last_active":"2023-11-28T13:01:56.939708+0000","last_peered":"2023-11-28T13:01:56.939708+0000","last_clean":"2023-11-28T13:01:56.939708+0000","last_became_active":"2023-11-28T13:01:56.553175+0000","last_became_peered":"2023-11-28T13:01:56.553175+0000","last_unstale":"2023-11-28T13:01:56.939708+0000","last_undegraded":"2023-11-28T13:01:56.939708+0000","last_fullsized":"2023-11-28T13:01:56.939708+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:34.785993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.938103+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:01:56.938103+0000","last_peered":"2023-11-28T13:01:56.938103+0000","last_clean":"2023-11-28T13:01:56.938103+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:01:56.938103+0000","last_undegraded":"2023-11-28T13:01:56.938103+0000","last_fullsized":"2023-11-28T13:01:56.938103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939794+0000","last_change":"2023-11-28T13:01:56.955397+0000","last_active":"2023-11-28T13:01:57.939794+0000","last_peered":"2023-11-28T13:01:57.939794+0000","last_clean":"2023-11-28T13:01:57.939794+0000","last_became_active":"2023-11-28T13:01:56.955082+0000","last_became_peered":"2023-11-28T13:01:56.955082+0000","last_unstale":"2023-11-28T13:01:57.939794+0000","last_undegraded":"2023-11-28T13:01:57.939794+0000","last_fullsized":"2023-11-28T13:01:57.939794+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:38:45.297181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.939490+0000","last_change":"2023-11-28T13:01:56.553790+0000","last_active":"2023-11-28T13:01:56.939490+0000","last_peered":"2023-11-28T13:01:56.939490+0000","last_clean":"2023-11-28T13:01:56.939490+0000","last_became_active":"2023-11-28T13:01:56.553497+0000","last_became_peered":"2023-11-28T13:01:56.553497+0000","last_unstale":"2023-11-28T13:01:56.939490+0000","last_undegraded":"2023-11-28T13:01:56.939490+0000","last_fullsized":"2023-11-28T13:01:56.939490+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:53.578242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945239+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:01:56.945239+0000","last_peered":"2023-11-28T13:01:56.945239+0000","last_clean":"2023-11-28T13:01:56.945239+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:01:56.945239+0000","last_undegraded":"2023-11-28T13:01:56.945239+0000","last_fullsized":"2023-11-28T13:01:56.945239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.547187+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:01:56.547187+0000","last_peered":"2023-11-28T13:01:56.547187+0000","last_clean":"2023-11-28T13:01:56.547187+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:01:56.547187+0000","last_undegraded":"2023-11-28T13:01:56.547187+0000","last_fullsized":"2023-11-28T13:01:56.547187+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328477+0000","last_change":"2023-11-28T13:01:56.940142+0000","last_active":"2023-11-28T13:01:59.328477+0000","last_peered":"2023-11-28T13:01:59.328477+0000","last_clean":"2023-11-28T13:01:59.328477+0000","last_became_active":"2023-11-28T13:01:56.939845+0000","last_became_peered":"2023-11-28T13:01:56.939845+0000","last_unstale":"2023-11-28T13:01:59.328477+0000","last_undegraded":"2023-11-28T13:01:59.328477+0000","last_fullsized":"2023-11-28T13:01:59.328477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:00.641378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.953147+0000","last_change":"2023-11-28T13:01:56.555323+0000","last_active":"2023-11-28T13:01:56.953147+0000","last_peered":"2023-11-28T13:01:56.953147+0000","last_clean":"2023-11-28T13:01:56.953147+0000","last_became_active":"2023-11-28T13:01:56.555065+0000","last_became_peered":"2023-11-28T13:01:56.555065+0000","last_unstale":"2023-11-28T13:01:56.953147+0000","last_undegraded":"2023-11-28T13:01:56.953147+0000","last_fullsized":"2023-11-28T13:01:56.953147+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:58.072158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.547091+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:01:56.547091+0000","last_peered":"2023-11-28T13:01:56.547091+0000","last_clean":"2023-11-28T13:01:56.547091+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:01:56.547091+0000","last_undegraded":"2023-11-28T13:01:56.547091+0000","last_fullsized":"2023-11-28T13:01:56.547091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939472+0000","last_change":"2023-11-28T13:01:56.947502+0000","last_active":"2023-11-28T13:01:57.939472+0000","last_peered":"2023-11-28T13:01:57.939472+0000","last_clean":"2023-11-28T13:01:57.939472+0000","last_became_active":"2023-11-28T13:01:56.947264+0000","last_became_peered":"2023-11-28T13:01:56.947264+0000","last_unstale":"2023-11-28T13:01:57.939472+0000","last_undegraded":"2023-11-28T13:01:57.939472+0000","last_fullsized":"2023-11-28T13:01:57.939472+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:27:18.348401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938879+0000","last_change":"2023-11-28T13:01:56.551111+0000","last_active":"2023-11-28T13:01:57.938879+0000","last_peered":"2023-11-28T13:01:57.938879+0000","last_clean":"2023-11-28T13:01:57.938879+0000","last_became_active":"2023-11-28T13:01:56.550805+0000","last_became_peered":"2023-11-28T13:01:56.550805+0000","last_unstale":"2023-11-28T13:01:57.938879+0000","last_undegraded":"2023-11-28T13:01:57.938879+0000","last_fullsized":"2023-11-28T13:01:57.938879+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:18.107890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.549017+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:01:56.549017+0000","last_peered":"2023-11-28T13:01:56.549017+0000","last_clean":"2023-11-28T13:01:56.549017+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:01:56.549017+0000","last_undegraded":"2023-11-28T13:01:56.549017+0000","last_fullsized":"2023-11-28T13:01:56.549017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955854+0000","last_change":"2023-11-28T13:01:56.951933+0000","last_active":"2023-11-28T13:01:56.955854+0000","last_peered":"2023-11-28T13:01:56.955854+0000","last_clean":"2023-11-28T13:01:56.955854+0000","last_became_active":"2023-11-28T13:01:56.948556+0000","last_became_peered":"2023-11-28T13:01:56.948556+0000","last_unstale":"2023-11-28T13:01:56.955854+0000","last_undegraded":"2023-11-28T13:01:56.955854+0000","last_fullsized":"2023-11-28T13:01:56.955854+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:40.918026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939210+0000","last_change":"2023-11-28T13:01:56.548143+0000","last_active":"2023-11-28T13:01:57.939210+0000","last_peered":"2023-11-28T13:01:57.939210+0000","last_clean":"2023-11-28T13:01:57.939210+0000","last_became_active":"2023-11-28T13:01:56.547663+0000","last_became_peered":"2023-11-28T13:01:56.547663+0000","last_unstale":"2023-11-28T13:01:57.939210+0000","last_undegraded":"2023-11-28T13:01:57.939210+0000","last_fullsized":"2023-11-28T13:01:57.939210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:32.983258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.935661+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:01:56.935661+0000","last_peered":"2023-11-28T13:01:56.935661+0000","last_clean":"2023-11-28T13:01:56.935661+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:01:56.935661+0000","last_undegraded":"2023-11-28T13:01:56.935661+0000","last_fullsized":"2023-11-28T13:01:56.935661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938726+0000","last_change":"2023-11-28T13:01:56.956241+0000","last_active":"2023-11-28T13:01:57.938726+0000","last_peered":"2023-11-28T13:01:57.938726+0000","last_clean":"2023-11-28T13:01:57.938726+0000","last_became_active":"2023-11-28T13:01:56.955742+0000","last_became_peered":"2023-11-28T13:01:56.955742+0000","last_unstale":"2023-11-28T13:01:57.938726+0000","last_undegraded":"2023-11-28T13:01:57.938726+0000","last_fullsized":"2023-11-28T13:01:57.938726+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:05.755474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939085+0000","last_change":"2023-11-28T13:01:56.554201+0000","last_active":"2023-11-28T13:01:57.939085+0000","last_peered":"2023-11-28T13:01:57.939085+0000","last_clean":"2023-11-28T13:01:57.939085+0000","last_became_active":"2023-11-28T13:01:56.553924+0000","last_became_peered":"2023-11-28T13:01:56.553924+0000","last_unstale":"2023-11-28T13:01:57.939085+0000","last_undegraded":"2023-11-28T13:01:57.939085+0000","last_fullsized":"2023-11-28T13:01:57.939085+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:19:22.180181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.948262+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:01:56.948262+0000","last_peered":"2023-11-28T13:01:56.948262+0000","last_clean":"2023-11-28T13:01:56.948262+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:01:56.948262+0000","last_undegraded":"2023-11-28T13:01:56.948262+0000","last_fullsized":"2023-11-28T13:01:56.948262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9180,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362660,"statfs":{"total":96636764160,"available":96627363840,"internally_reserved":0,"allocated":610304,"data_stored":256967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.218,"5min":8.218,"15min":8.218},"last":0.34399999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.36199999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":13.384,"5min":13.384,"15min":13.384},"last":0.378},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.226,"5min":5.226,"15min":5.226},"last":0.41299999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.3209999999999997,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.623},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.1029999999999998,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.3849999999999998,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.67000000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8908,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362932,"statfs":{"total":96636764160,"available":96627642368,"internally_reserved":0,"allocated":593920,"data_stored":250292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.251}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.373}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.308},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35999999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.495},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47299999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9028,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362812,"statfs":{"total":96636764160,"available":96627519488,"internally_reserved":0,"allocated":585728,"data_stored":249310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.50800000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.66200000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.624},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.69399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14004,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94357836,"statfs":{"total":96636764160,"available":96622424064,"internally_reserved":0,"allocated":1159168,"data_stored":831215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":14.997999999999999,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":19.045999999999999,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":18.995000000000001,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":19.02,"5min":19.02,"15min":19.02},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.504},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.1539999999999999,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":18.957999999999998,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.68400000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10188,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361652,"statfs":{"total":96636764160,"available":96626331648,"internally_reserved":0,"allocated":1183744,"data_stored":839236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.503}]},{"osd":1,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.44},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.48799999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9036,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362804,"statfs":{"total":96636764160,"available":96627511296,"internally_reserved":0,"allocated":593920,"data_stored":255397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.624}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.372}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.499}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":602112,"data_stored":256033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.56699999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.443}]},{"osd":3,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.628},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.65100000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8968,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362872,"statfs":{"total":96636764160,"available":96627580928,"internally_reserved":0,"allocated":589824,"data_stored":248904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.27100000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.32000000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.52400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:02:01.008 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:02:01.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.164+0000 7f48be894700 1 Processor -- start 2023-11-28T13:02:01.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.164+0000 7f48be894700 1 -- start start 2023-11-28T13:02:01.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48be894700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48be894700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f48b8159860 0x7f48b80b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48be894700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b80ccc00 con 0x7f48b80cdc80 2023-11-28T13:02:01.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48be894700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b80ccec0 con 0x7f48b8159860 2023-11-28T13:02:01.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48be894700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80cd1a0 con 0x7f48b8159490 2023-11-28T13:02:01.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f48b8159860 0x7f48b80b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b7fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f48b8159860 0x7f48b80b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:50698/0 (socket says 172.21.15.153:50698) 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b7fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:58842/0 (socket says 172.21.15.153:58842) 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b77fe700 1 -- 172.21.15.153:0/1632694305 learned_addr learned my addr 172.21.15.153:0/1632694305 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b7fff700 1 -- 172.21.15.153:0/1632694305 >> v2:172.21.15.177:3300/0 conn(0x7f48b8159860 msgr2=0x7f48b80b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b7fff700 1 --2- 172.21.15.153:0/1632694305 >> v2:172.21.15.177:3300/0 conn(0x7f48b8159860 0x7f48b80b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b7fff700 1 -- 172.21.15.153:0/1632694305 >> v1:172.21.15.177:6789/0 conn(0x7f48b80cdc80 legacy=0x7f48b80c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b7fff700 1 -- 172.21.15.153:0/1632694305 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80cd480 con 0x7f48b8159490 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.165+0000 7f48b77fe700 1 --2- 172.21.15.153:0/1632694305 >> v2:172.21.15.177:3300/0 conn(0x7f48b8159860 0x7f48b80b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48b7fff700 1 --2- 172.21.15.153:0/1632694305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b0670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009840 tx=0x7f48ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da43fccfe208f480 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48b6ffd700 1 -- 172.21.15.153:0/1632694305 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011830 con 0x7f48b8159490 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48b6ffd700 1 -- 172.21.15.153:0/1632694305 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac011990 con 0x7f48b8159490 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48b6ffd700 1 -- 172.21.15.153:0/1632694305 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011d20 con 0x7f48b8159490 2023-11-28T13:02:01.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 -- 172.21.15.153:0/1632694305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 msgr2=0x7f48b80b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 --2- 172.21.15.153:0/1632694305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b0670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009840 tx=0x7f48ac009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:01.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 -- 172.21.15.153:0/1632694305 shutdown_connections 2023-11-28T13:02:01.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 --2- 172.21.15.153:0/1632694305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b0670 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 --2- 172.21.15.153:0/1632694305 >> v2:172.21.15.177:3300/0 conn(0x7f48b8159860 0x7f48b80b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 -- 172.21.15.153:0/1632694305 >> 172.21.15.153:0/1632694305 conn(0x7f48b80cb670 msgr2=0x7f48b80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:01.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 -- 172.21.15.153:0/1632694305 shutdown_connections 2023-11-28T13:02:01.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.166+0000 7f48be894700 1 -- 172.21.15.153:0/1632694305 wait complete. 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 Processor -- start 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 -- start start 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48b80cdc80 0x7f48b81f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b8070ee0 0x7f48b81579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81f6b00 con 0x7f48b8070ee0 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81f6260 con 0x7f48b80cdc80 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.168+0000 7f48be894700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81f6540 con 0x7f48b8159490 2023-11-28T13:02:01.172 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b8070ee0 0x7f48b81579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b8070ee0 0x7f48b81579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41242/0 (socket says 172.21.15.153:41242) 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 -- 172.21.15.153:0/4197665470 learned_addr learned my addr 172.21.15.153:0/4197665470 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48b7fff700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48b80cdc80 0x7f48b81f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48b77fe700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 -- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 msgr2=0x7f48b80b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 -- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48b80cdc80 msgr2=0x7f48b81f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48b80cdc80 0x7f48b81f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 -- 172.21.15.153:0/4197665470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8009550 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48bce31700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b8070ee0 0x7f48b81579b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48b000cc50 tx=0x7f48b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48b4ff9700 1 -- 172.21.15.153:0/4197665470 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0003c00 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48b4ff9700 1 -- 172.21.15.153:0/4197665470 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0003d60 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac0094a0 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48b4ff9700 1 -- 172.21.15.153:0/4197665470 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b000aab0 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.169+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b80c9bd0 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.170+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b80cd760 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.171+0000 7f48b4ff9700 1 -- 172.21.15.153:0/4197665470 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f48b0010050 con 0x7f48b8070ee0 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.171+0000 7f48b4ff9700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f48a40682f0 0x7f48a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.171+0000 7f48b4ff9700 1 -- 172.21.15.153:0/4197665470 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48b0085ac0 con 0x7f48b8070ee0 2023-11-28T13:02:01.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.171+0000 7f48b7fff700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f48a40682f0 0x7f48a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.171+0000 7f48b7fff700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f48a40682f0 0x7f48a406a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000c00 tx=0x7f48ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.175+0000 7f48b4ff9700 1 -- 172.21.15.153:0/4197665470 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48b0052320 con 0x7f48b8070ee0 2023-11-28T13:02:01.353 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.353+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48b80c9d60 con 0x7f48a40682f0 2023-11-28T13:02:01.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.361+0000 7f48b4ff9700 1 -- 172.21.15.153:0/4197665470 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311122 (secure 0 0 0) 0x7f48b80c9d60 con 0x7f48a40682f0 2023-11-28T13:02:01.365 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f48a40682f0 msgr2=0x7f48a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f48a40682f0 0x7f48a406a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000c00 tx=0x7f48ac008030 comp rx=0 tx=0).stop 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b8070ee0 msgr2=0x7f48b81579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b8070ee0 0x7f48b81579b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48b000cc50 tx=0x7f48b000cf60 comp rx=0 tx=0).stop 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 shutdown_connections 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f48a40682f0 0x7f48a406a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b8070ee0 0x7f48b81579b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48b80cdc80 0x7f48b81f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 --2- 172.21.15.153:0/4197665470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48b8159490 0x7f48b80b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 >> 172.21.15.153:0/4197665470 conn(0x7f48b80cb670 msgr2=0x7f48b80cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:01.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 shutdown_connections 2023-11-28T13:02:01.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.364+0000 7f48be894700 1 -- 172.21.15.153:0/4197665470 wait complete. 2023-11-28T13:02:01.369 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:02:01.377 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2023-11-28T13:02:00.168045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78228,"kb_used_data":5780,"kb_used_omap":0,"kb_used_meta":72384,"kb_avail":754896492,"statfs":{"total":773094113280,"available":773014007808,"internally_reserved":0,"allocated":5918720,"data_stored":3187354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74121216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.639257"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328893+0000","last_change":"2023-11-28T13:01:56.955236+0000","last_active":"2023-11-28T13:01:59.328893+0000","last_peered":"2023-11-28T13:01:59.328893+0000","last_clean":"2023-11-28T13:01:59.328893+0000","last_became_active":"2023-11-28T13:01:56.954983+0000","last_became_peered":"2023-11-28T13:01:56.954983+0000","last_unstale":"2023-11-28T13:01:59.328893+0000","last_undegraded":"2023-11-28T13:01:59.328893+0000","last_fullsized":"2023-11-28T13:01:59.328893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:18.308328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966735+0000","last_change":"2023-11-28T13:01:56.964287+0000","last_active":"2023-11-28T13:01:56.966735+0000","last_peered":"2023-11-28T13:01:56.966735+0000","last_clean":"2023-11-28T13:01:56.966735+0000","last_became_active":"2023-11-28T13:01:56.964011+0000","last_became_peered":"2023-11-28T13:01:56.964011+0000","last_unstale":"2023-11-28T13:01:56.966735+0000","last_undegraded":"2023-11-28T13:01:56.966735+0000","last_fullsized":"2023-11-28T13:01:56.966735+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:07.713158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955981+0000","last_change":"2023-11-28T13:01:56.952523+0000","last_active":"2023-11-28T13:01:56.955981+0000","last_peered":"2023-11-28T13:01:56.955981+0000","last_clean":"2023-11-28T13:01:56.955981+0000","last_became_active":"2023-11-28T13:01:56.952277+0000","last_became_peered":"2023-11-28T13:01:56.952277+0000","last_unstale":"2023-11-28T13:01:56.955981+0000","last_undegraded":"2023-11-28T13:01:56.955981+0000","last_fullsized":"2023-11-28T13:01:56.955981+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:31.721247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966236+0000","last_change":"2023-11-28T13:01:56.962603+0000","last_active":"2023-11-28T13:01:56.966236+0000","last_peered":"2023-11-28T13:01:56.966236+0000","last_clean":"2023-11-28T13:01:56.966236+0000","last_became_active":"2023-11-28T13:01:56.962382+0000","last_became_peered":"2023-11-28T13:01:56.962382+0000","last_unstale":"2023-11-28T13:01:56.966236+0000","last_undegraded":"2023-11-28T13:01:56.966236+0000","last_fullsized":"2023-11-28T13:01:56.966236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:53.278587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961536+0000","last_change":"2023-11-28T13:01:56.957872+0000","last_active":"2023-11-28T13:01:56.961536+0000","last_peered":"2023-11-28T13:01:56.961536+0000","last_clean":"2023-11-28T13:01:56.961536+0000","last_became_active":"2023-11-28T13:01:56.957546+0000","last_became_peered":"2023-11-28T13:01:56.957546+0000","last_unstale":"2023-11-28T13:01:56.961536+0000","last_undegraded":"2023-11-28T13:01:56.961536+0000","last_fullsized":"2023-11-28T13:01:56.961536+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:07.713158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328389+0000","last_change":"2023-11-28T13:01:56.956976+0000","last_active":"2023-11-28T13:01:59.328389+0000","last_peered":"2023-11-28T13:01:59.328389+0000","last_clean":"2023-11-28T13:01:59.328389+0000","last_became_active":"2023-11-28T13:01:56.956678+0000","last_became_peered":"2023-11-28T13:01:56.956678+0000","last_unstale":"2023-11-28T13:01:59.328389+0000","last_undegraded":"2023-11-28T13:01:59.328389+0000","last_fullsized":"2023-11-28T13:01:59.328389+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:58.659349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958280+0000","last_change":"2023-11-28T13:01:56.954262+0000","last_active":"2023-11-28T13:01:56.958280+0000","last_peered":"2023-11-28T13:01:56.958280+0000","last_clean":"2023-11-28T13:01:56.958280+0000","last_became_active":"2023-11-28T13:01:56.949293+0000","last_became_peered":"2023-11-28T13:01:56.949293+0000","last_unstale":"2023-11-28T13:01:56.958280+0000","last_undegraded":"2023-11-28T13:01:56.958280+0000","last_fullsized":"2023-11-28T13:01:56.958280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:26.904660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938994+0000","last_change":"2023-11-28T13:01:56.957286+0000","last_active":"2023-11-28T13:01:57.938994+0000","last_peered":"2023-11-28T13:01:57.938994+0000","last_clean":"2023-11-28T13:01:57.938994+0000","last_became_active":"2023-11-28T13:01:56.954543+0000","last_became_peered":"2023-11-28T13:01:56.954543+0000","last_unstale":"2023-11-28T13:01:57.938994+0000","last_undegraded":"2023-11-28T13:01:57.938994+0000","last_fullsized":"2023-11-28T13:01:57.938994+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:27.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939402+0000","last_change":"2023-11-28T13:01:56.954084+0000","last_active":"2023-11-28T13:01:57.939402+0000","last_peered":"2023-11-28T13:01:57.939402+0000","last_clean":"2023-11-28T13:01:57.939402+0000","last_became_active":"2023-11-28T13:01:56.953705+0000","last_became_peered":"2023-11-28T13:01:56.953705+0000","last_unstale":"2023-11-28T13:01:57.939402+0000","last_undegraded":"2023-11-28T13:01:57.939402+0000","last_fullsized":"2023-11-28T13:01:57.939402+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:38.900791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328850+0000","last_change":"2023-11-28T13:01:56.961174+0000","last_active":"2023-11-28T13:01:59.328850+0000","last_peered":"2023-11-28T13:01:59.328850+0000","last_clean":"2023-11-28T13:01:59.328850+0000","last_became_active":"2023-11-28T13:01:56.960912+0000","last_became_peered":"2023-11-28T13:01:56.960912+0000","last_unstale":"2023-11-28T13:01:59.328850+0000","last_undegraded":"2023-11-28T13:01:59.328850+0000","last_fullsized":"2023-11-28T13:01:59.328850+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:28.462737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329138+0000","last_change":"2023-11-28T13:01:56.954116+0000","last_active":"2023-11-28T13:01:59.329138+0000","last_peered":"2023-11-28T13:01:59.329138+0000","last_clean":"2023-11-28T13:01:59.329138+0000","last_became_active":"2023-11-28T13:01:56.953826+0000","last_became_peered":"2023-11-28T13:01:56.953826+0000","last_unstale":"2023-11-28T13:01:59.329138+0000","last_undegraded":"2023-11-28T13:01:59.329138+0000","last_fullsized":"2023-11-28T13:01:59.329138+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:31.623102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939196+0000","last_change":"2023-11-28T13:01:56.954183+0000","last_active":"2023-11-28T13:01:57.939196+0000","last_peered":"2023-11-28T13:01:57.939196+0000","last_clean":"2023-11-28T13:01:57.939196+0000","last_became_active":"2023-11-28T13:01:56.953864+0000","last_became_peered":"2023-11-28T13:01:56.953864+0000","last_unstale":"2023-11-28T13:01:57.939196+0000","last_undegraded":"2023-11-28T13:01:57.939196+0000","last_fullsized":"2023-11-28T13:01:57.939196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:04:23.390277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329127+0000","last_change":"2023-11-28T13:01:56.961359+0000","last_active":"2023-11-28T13:01:59.329127+0000","last_peered":"2023-11-28T13:01:59.329127+0000","last_clean":"2023-11-28T13:01:59.329127+0000","last_became_active":"2023-11-28T13:01:56.960919+0000","last_became_peered":"2023-11-28T13:01:56.960919+0000","last_unstale":"2023-11-28T13:01:59.329127+0000","last_undegraded":"2023-11-28T13:01:59.329127+0000","last_fullsized":"2023-11-28T13:01:59.329127+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:33.988818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961305+0000","last_change":"2023-11-28T13:01:56.956495+0000","last_active":"2023-11-28T13:01:56.961305+0000","last_peered":"2023-11-28T13:01:56.961305+0000","last_clean":"2023-11-28T13:01:56.961305+0000","last_became_active":"2023-11-28T13:01:56.956157+0000","last_became_peered":"2023-11-28T13:01:56.956157+0000","last_unstale":"2023-11-28T13:01:56.961305+0000","last_undegraded":"2023-11-28T13:01:56.961305+0000","last_fullsized":"2023-11-28T13:01:56.961305+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:34:21.421755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963748+0000","last_change":"2023-11-28T13:01:56.960492+0000","last_active":"2023-11-28T13:01:56.963748+0000","last_peered":"2023-11-28T13:01:56.963748+0000","last_clean":"2023-11-28T13:01:56.963748+0000","last_became_active":"2023-11-28T13:01:56.960239+0000","last_became_peered":"2023-11-28T13:01:56.960239+0000","last_unstale":"2023-11-28T13:01:56.963748+0000","last_undegraded":"2023-11-28T13:01:56.963748+0000","last_fullsized":"2023-11-28T13:01:56.963748+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:27:36.453092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328082+0000","last_change":"2023-11-28T13:01:56.961369+0000","last_active":"2023-11-28T13:01:59.328082+0000","last_peered":"2023-11-28T13:01:59.328082+0000","last_clean":"2023-11-28T13:01:59.328082+0000","last_became_active":"2023-11-28T13:01:56.960911+0000","last_became_peered":"2023-11-28T13:01:56.960911+0000","last_unstale":"2023-11-28T13:01:59.328082+0000","last_undegraded":"2023-11-28T13:01:59.328082+0000","last_fullsized":"2023-11-28T13:01:59.328082+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:20.406868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961524+0000","last_change":"2023-11-28T13:01:56.956431+0000","last_active":"2023-11-28T13:01:56.961524+0000","last_peered":"2023-11-28T13:01:56.961524+0000","last_clean":"2023-11-28T13:01:56.961524+0000","last_became_active":"2023-11-28T13:01:56.955579+0000","last_became_peered":"2023-11-28T13:01:56.955579+0000","last_unstale":"2023-11-28T13:01:56.961524+0000","last_undegraded":"2023-11-28T13:01:56.961524+0000","last_fullsized":"2023-11-28T13:01:56.961524+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:17.982328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.971367+0000","last_change":"2023-11-28T13:01:56.966653+0000","last_active":"2023-11-28T13:01:56.971367+0000","last_peered":"2023-11-28T13:01:56.971367+0000","last_clean":"2023-11-28T13:01:56.971367+0000","last_became_active":"2023-11-28T13:01:56.966395+0000","last_became_peered":"2023-11-28T13:01:56.966395+0000","last_unstale":"2023-11-28T13:01:56.971367+0000","last_undegraded":"2023-11-28T13:01:56.971367+0000","last_fullsized":"2023-11-28T13:01:56.971367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:35.791553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.964176+0000","last_change":"2023-11-28T13:01:56.959599+0000","last_active":"2023-11-28T13:01:56.964176+0000","last_peered":"2023-11-28T13:01:56.964176+0000","last_clean":"2023-11-28T13:01:56.964176+0000","last_became_active":"2023-11-28T13:01:56.959337+0000","last_became_peered":"2023-11-28T13:01:56.959337+0000","last_unstale":"2023-11-28T13:01:56.964176+0000","last_undegraded":"2023-11-28T13:01:56.964176+0000","last_fullsized":"2023-11-28T13:01:56.964176+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:14.309966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939747+0000","last_change":"2023-11-28T13:01:56.960241+0000","last_active":"2023-11-28T13:01:57.939747+0000","last_peered":"2023-11-28T13:01:57.939747+0000","last_clean":"2023-11-28T13:01:57.939747+0000","last_became_active":"2023-11-28T13:01:56.959952+0000","last_became_peered":"2023-11-28T13:01:56.959952+0000","last_unstale":"2023-11-28T13:01:57.939747+0000","last_undegraded":"2023-11-28T13:01:57.939747+0000","last_fullsized":"2023-11-28T13:01:57.939747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:34.740039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939941+0000","last_change":"2023-11-28T13:01:56.953817+0000","last_active":"2023-11-28T13:01:57.939941+0000","last_peered":"2023-11-28T13:01:57.939941+0000","last_clean":"2023-11-28T13:01:57.939941+0000","last_became_active":"2023-11-28T13:01:56.953525+0000","last_became_peered":"2023-11-28T13:01:56.953525+0000","last_unstale":"2023-11-28T13:01:57.939941+0000","last_undegraded":"2023-11-28T13:01:57.939941+0000","last_fullsized":"2023-11-28T13:01:57.939941+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:59.077718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938999+0000","last_change":"2023-11-28T13:01:56.959333+0000","last_active":"2023-11-28T13:01:57.938999+0000","last_peered":"2023-11-28T13:01:57.938999+0000","last_clean":"2023-11-28T13:01:57.938999+0000","last_became_active":"2023-11-28T13:01:56.959092+0000","last_became_peered":"2023-11-28T13:01:56.959092+0000","last_unstale":"2023-11-28T13:01:57.938999+0000","last_undegraded":"2023-11-28T13:01:57.938999+0000","last_fullsized":"2023-11-28T13:01:57.938999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:08.627426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328354+0000","last_change":"2023-11-28T13:01:56.954106+0000","last_active":"2023-11-28T13:01:59.328354+0000","last_peered":"2023-11-28T13:01:59.328354+0000","last_clean":"2023-11-28T13:01:59.328354+0000","last_became_active":"2023-11-28T13:01:56.953426+0000","last_became_peered":"2023-11-28T13:01:56.953426+0000","last_unstale":"2023-11-28T13:01:59.328354+0000","last_undegraded":"2023-11-28T13:01:59.328354+0000","last_fullsized":"2023-11-28T13:01:59.328354+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:02:49.467780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958225+0000","last_change":"2023-11-28T13:01:56.953222+0000","last_active":"2023-11-28T13:01:56.958225+0000","last_peered":"2023-11-28T13:01:56.958225+0000","last_clean":"2023-11-28T13:01:56.958225+0000","last_became_active":"2023-11-28T13:01:56.948890+0000","last_became_peered":"2023-11-28T13:01:56.948890+0000","last_unstale":"2023-11-28T13:01:56.958225+0000","last_undegraded":"2023-11-28T13:01:56.958225+0000","last_fullsized":"2023-11-28T13:01:56.958225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:33.051762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.956323+0000","last_change":"2023-11-28T13:01:56.952413+0000","last_active":"2023-11-28T13:01:56.956323+0000","last_peered":"2023-11-28T13:01:56.956323+0000","last_clean":"2023-11-28T13:01:56.956323+0000","last_became_active":"2023-11-28T13:01:56.952098+0000","last_became_peered":"2023-11-28T13:01:56.952098+0000","last_unstale":"2023-11-28T13:01:56.956323+0000","last_undegraded":"2023-11-28T13:01:56.956323+0000","last_fullsized":"2023-11-28T13:01:56.956323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:20.709046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939876+0000","last_change":"2023-11-28T13:01:56.947344+0000","last_active":"2023-11-28T13:01:57.939876+0000","last_peered":"2023-11-28T13:01:57.939876+0000","last_clean":"2023-11-28T13:01:57.939876+0000","last_became_active":"2023-11-28T13:01:56.947052+0000","last_became_peered":"2023-11-28T13:01:56.947052+0000","last_unstale":"2023-11-28T13:01:57.939876+0000","last_undegraded":"2023-11-28T13:01:57.939876+0000","last_fullsized":"2023-11-28T13:01:57.939876+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:44.377544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328532+0000","last_change":"2023-11-28T13:01:56.953937+0000","last_active":"2023-11-28T13:01:59.328532+0000","last_peered":"2023-11-28T13:01:59.328532+0000","last_clean":"2023-11-28T13:01:59.328532+0000","last_became_active":"2023-11-28T13:01:56.953492+0000","last_became_peered":"2023-11-28T13:01:56.953492+0000","last_unstale":"2023-11-28T13:01:59.328532+0000","last_undegraded":"2023-11-28T13:01:59.328532+0000","last_fullsized":"2023-11-28T13:01:59.328532+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:55.893471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.940036+0000","last_change":"2023-11-28T13:01:56.953154+0000","last_active":"2023-11-28T13:01:57.940036+0000","last_peered":"2023-11-28T13:01:57.940036+0000","last_clean":"2023-11-28T13:01:57.940036+0000","last_became_active":"2023-11-28T13:01:56.952854+0000","last_became_peered":"2023-11-28T13:01:56.952854+0000","last_unstale":"2023-11-28T13:01:57.940036+0000","last_undegraded":"2023-11-28T13:01:57.940036+0000","last_fullsized":"2023-11-28T13:01:57.940036+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:09:30.257699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958256+0000","last_change":"2023-11-28T13:01:56.953036+0000","last_active":"2023-11-28T13:01:56.958256+0000","last_peered":"2023-11-28T13:01:56.958256+0000","last_clean":"2023-11-28T13:01:56.958256+0000","last_became_active":"2023-11-28T13:01:56.952752+0000","last_became_peered":"2023-11-28T13:01:56.952752+0000","last_unstale":"2023-11-28T13:01:56.958256+0000","last_undegraded":"2023-11-28T13:01:56.958256+0000","last_fullsized":"2023-11-28T13:01:56.958256+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:28.471084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963719+0000","last_change":"2023-11-28T13:01:56.959862+0000","last_active":"2023-11-28T13:01:56.963719+0000","last_peered":"2023-11-28T13:01:56.963719+0000","last_clean":"2023-11-28T13:01:56.963719+0000","last_became_active":"2023-11-28T13:01:56.959597+0000","last_became_peered":"2023-11-28T13:01:56.959597+0000","last_unstale":"2023-11-28T13:01:56.963719+0000","last_undegraded":"2023-11-28T13:01:56.963719+0000","last_fullsized":"2023-11-28T13:01:56.963719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:08:45.732739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955898+0000","last_change":"2023-11-28T13:01:56.949530+0000","last_active":"2023-11-28T13:01:56.955898+0000","last_peered":"2023-11-28T13:01:56.955898+0000","last_clean":"2023-11-28T13:01:56.955898+0000","last_became_active":"2023-11-28T13:01:56.949009+0000","last_became_peered":"2023-11-28T13:01:56.949009+0000","last_unstale":"2023-11-28T13:01:56.955898+0000","last_undegraded":"2023-11-28T13:01:56.955898+0000","last_fullsized":"2023-11-28T13:01:56.955898+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:06:06.562735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938984+0000","last_change":"2023-11-28T13:01:56.950806+0000","last_active":"2023-11-28T13:01:57.938984+0000","last_peered":"2023-11-28T13:01:57.938984+0000","last_clean":"2023-11-28T13:01:57.938984+0000","last_became_active":"2023-11-28T13:01:56.950562+0000","last_became_peered":"2023-11-28T13:01:56.950562+0000","last_unstale":"2023-11-28T13:01:57.938984+0000","last_undegraded":"2023-11-28T13:01:57.938984+0000","last_fullsized":"2023-11-28T13:01:57.938984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:01.873986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963586+0000","last_change":"2023-11-28T13:01:56.955782+0000","last_active":"2023-11-28T13:01:56.963586+0000","last_peered":"2023-11-28T13:01:56.963586+0000","last_clean":"2023-11-28T13:01:56.963586+0000","last_became_active":"2023-11-28T13:01:56.955500+0000","last_became_peered":"2023-11-28T13:01:56.955500+0000","last_unstale":"2023-11-28T13:01:56.963586+0000","last_undegraded":"2023-11-28T13:01:56.963586+0000","last_fullsized":"2023-11-28T13:01:56.963586+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:01.873986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939696+0000","last_change":"2023-11-28T13:01:56.546817+0000","last_active":"2023-11-28T13:01:57.939696+0000","last_peered":"2023-11-28T13:01:57.939696+0000","last_clean":"2023-11-28T13:01:57.939696+0000","last_became_active":"2023-11-28T13:01:56.546488+0000","last_became_peered":"2023-11-28T13:01:56.546488+0000","last_unstale":"2023-11-28T13:01:57.939696+0000","last_undegraded":"2023-11-28T13:01:57.939696+0000","last_fullsized":"2023-11-28T13:01:57.939696+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:43.975011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938820+0000","last_change":"2023-11-28T13:01:56.957111+0000","last_active":"2023-11-28T13:01:57.938820+0000","last_peered":"2023-11-28T13:01:57.938820+0000","last_clean":"2023-11-28T13:01:57.938820+0000","last_became_active":"2023-11-28T13:01:56.956783+0000","last_became_peered":"2023-11-28T13:01:56.956783+0000","last_unstale":"2023-11-28T13:01:57.938820+0000","last_undegraded":"2023-11-28T13:01:57.938820+0000","last_fullsized":"2023-11-28T13:01:57.938820+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:17.780940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328939+0000","last_change":"2023-11-28T13:01:56.547728+0000","last_active":"2023-11-28T13:01:59.328939+0000","last_peered":"2023-11-28T13:01:59.328939+0000","last_clean":"2023-11-28T13:01:59.328939+0000","last_became_active":"2023-11-28T13:01:56.547464+0000","last_became_peered":"2023-11-28T13:01:56.547464+0000","last_unstale":"2023-11-28T13:01:59.328939+0000","last_undegraded":"2023-11-28T13:01:59.328939+0000","last_fullsized":"2023-11-28T13:01:59.328939+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:26.512099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939714+0000","last_change":"2023-11-28T13:01:56.955433+0000","last_active":"2023-11-28T13:01:57.939714+0000","last_peered":"2023-11-28T13:01:57.939714+0000","last_clean":"2023-11-28T13:01:57.939714+0000","last_became_active":"2023-11-28T13:01:56.955154+0000","last_became_peered":"2023-11-28T13:01:56.955154+0000","last_unstale":"2023-11-28T13:01:57.939714+0000","last_undegraded":"2023-11-28T13:01:57.939714+0000","last_fullsized":"2023-11-28T13:01:57.939714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:16.870674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328693+0000","last_change":"2023-11-28T13:01:56.553637+0000","last_active":"2023-11-28T13:01:59.328693+0000","last_peered":"2023-11-28T13:01:59.328693+0000","last_clean":"2023-11-28T13:01:59.328693+0000","last_became_active":"2023-11-28T13:01:56.550191+0000","last_became_peered":"2023-11-28T13:01:56.550191+0000","last_unstale":"2023-11-28T13:01:59.328693+0000","last_undegraded":"2023-11-28T13:01:59.328693+0000","last_fullsized":"2023-11-28T13:01:59.328693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:32:43.641338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939119+0000","last_change":"2023-11-28T13:01:56.953148+0000","last_active":"2023-11-28T13:01:57.939119+0000","last_peered":"2023-11-28T13:01:57.939119+0000","last_clean":"2023-11-28T13:01:57.939119+0000","last_became_active":"2023-11-28T13:01:56.952892+0000","last_became_peered":"2023-11-28T13:01:56.952892+0000","last_unstale":"2023-11-28T13:01:57.939119+0000","last_undegraded":"2023-11-28T13:01:57.939119+0000","last_fullsized":"2023-11-28T13:01:57.939119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:09.970320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939077+0000","last_change":"2023-11-28T13:01:56.543766+0000","last_active":"2023-11-28T13:01:57.939077+0000","last_peered":"2023-11-28T13:01:57.939077+0000","last_clean":"2023-11-28T13:01:57.939077+0000","last_became_active":"2023-11-28T13:01:56.543472+0000","last_became_peered":"2023-11-28T13:01:56.543472+0000","last_unstale":"2023-11-28T13:01:57.939077+0000","last_undegraded":"2023-11-28T13:01:57.939077+0000","last_fullsized":"2023-11-28T13:01:57.939077+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:02.634978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963213+0000","last_change":"2023-11-28T13:01:56.958663+0000","last_active":"2023-11-28T13:01:56.963213+0000","last_peered":"2023-11-28T13:01:56.963213+0000","last_clean":"2023-11-28T13:01:56.963213+0000","last_became_active":"2023-11-28T13:01:56.958394+0000","last_became_peered":"2023-11-28T13:01:56.958394+0000","last_unstale":"2023-11-28T13:01:56.963213+0000","last_undegraded":"2023-11-28T13:01:56.963213+0000","last_fullsized":"2023-11-28T13:01:56.963213+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:05.390821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.952380+0000","last_change":"2023-11-28T13:01:56.554087+0000","last_active":"2023-11-28T13:01:56.952380+0000","last_peered":"2023-11-28T13:01:56.952380+0000","last_clean":"2023-11-28T13:01:56.952380+0000","last_became_active":"2023-11-28T13:01:56.553109+0000","last_became_peered":"2023-11-28T13:01:56.553109+0000","last_unstale":"2023-11-28T13:01:56.952380+0000","last_undegraded":"2023-11-28T13:01:56.952380+0000","last_fullsized":"2023-11-28T13:01:56.952380+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:52.238893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966173+0000","last_change":"2023-11-28T13:01:56.961748+0000","last_active":"2023-11-28T13:01:56.966173+0000","last_peered":"2023-11-28T13:01:56.966173+0000","last_clean":"2023-11-28T13:01:56.966173+0000","last_became_active":"2023-11-28T13:01:56.961497+0000","last_became_peered":"2023-11-28T13:01:56.961497+0000","last_unstale":"2023-11-28T13:01:56.966173+0000","last_undegraded":"2023-11-28T13:01:56.966173+0000","last_fullsized":"2023-11-28T13:01:56.966173+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.243375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958285+0000","last_change":"2023-11-28T13:01:56.547723+0000","last_active":"2023-11-28T13:01:56.958285+0000","last_peered":"2023-11-28T13:01:56.958285+0000","last_clean":"2023-11-28T13:01:56.958285+0000","last_became_active":"2023-11-28T13:01:56.547451+0000","last_became_peered":"2023-11-28T13:01:56.547451+0000","last_unstale":"2023-11-28T13:01:56.958285+0000","last_undegraded":"2023-11-28T13:01:56.958285+0000","last_fullsized":"2023-11-28T13:01:56.958285+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:18.107890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939781+0000","last_change":"2023-11-28T13:01:56.950630+0000","last_active":"2023-11-28T13:01:57.939781+0000","last_peered":"2023-11-28T13:01:57.939781+0000","last_clean":"2023-11-28T13:01:57.939781+0000","last_became_active":"2023-11-28T13:01:56.950182+0000","last_became_peered":"2023-11-28T13:01:56.950182+0000","last_unstale":"2023-11-28T13:01:57.939781+0000","last_undegraded":"2023-11-28T13:01:57.939781+0000","last_fullsized":"2023-11-28T13:01:57.939781+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:20.762287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939530+0000","last_change":"2023-11-28T13:01:56.549973+0000","last_active":"2023-11-28T13:01:57.939530+0000","last_peered":"2023-11-28T13:01:57.939530+0000","last_clean":"2023-11-28T13:01:57.939530+0000","last_became_active":"2023-11-28T13:01:56.549717+0000","last_became_peered":"2023-11-28T13:01:56.549717+0000","last_unstale":"2023-11-28T13:01:57.939530+0000","last_undegraded":"2023-11-28T13:01:57.939530+0000","last_fullsized":"2023-11-28T13:01:57.939530+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:17.503589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.966112+0000","last_change":"2023-11-28T13:01:56.961126+0000","last_active":"2023-11-28T13:01:56.966112+0000","last_peered":"2023-11-28T13:01:56.966112+0000","last_clean":"2023-11-28T13:01:56.966112+0000","last_became_active":"2023-11-28T13:01:56.960831+0000","last_became_peered":"2023-11-28T13:01:56.960831+0000","last_unstale":"2023-11-28T13:01:56.966112+0000","last_undegraded":"2023-11-28T13:01:56.966112+0000","last_fullsized":"2023-11-28T13:01:56.966112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:05.390821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.939498+0000","last_change":"2023-11-28T13:01:56.556138+0000","last_active":"2023-11-28T13:01:56.939498+0000","last_peered":"2023-11-28T13:01:56.939498+0000","last_clean":"2023-11-28T13:01:56.939498+0000","last_became_active":"2023-11-28T13:01:56.555815+0000","last_became_peered":"2023-11-28T13:01:56.555815+0000","last_unstale":"2023-11-28T13:01:56.939498+0000","last_undegraded":"2023-11-28T13:01:56.939498+0000","last_fullsized":"2023-11-28T13:01:56.939498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:02.634978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958929+0000","last_change":"2023-11-28T13:01:56.956344+0000","last_active":"2023-11-28T13:01:56.958929+0000","last_peered":"2023-11-28T13:01:56.958929+0000","last_clean":"2023-11-28T13:01:56.958929+0000","last_became_active":"2023-11-28T13:01:56.956051+0000","last_became_peered":"2023-11-28T13:01:56.956051+0000","last_unstale":"2023-11-28T13:01:56.958929+0000","last_undegraded":"2023-11-28T13:01:56.958929+0000","last_fullsized":"2023-11-28T13:01:56.958929+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.243375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.952091+0000","last_change":"2023-11-28T13:01:56.553211+0000","last_active":"2023-11-28T13:01:56.952091+0000","last_peered":"2023-11-28T13:01:56.952091+0000","last_clean":"2023-11-28T13:01:56.952091+0000","last_became_active":"2023-11-28T13:01:56.552933+0000","last_became_peered":"2023-11-28T13:01:56.552933+0000","last_unstale":"2023-11-28T13:01:56.952091+0000","last_undegraded":"2023-11-28T13:01:56.952091+0000","last_fullsized":"2023-11-28T13:01:56.952091+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:33.734479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938701+0000","last_change":"2023-11-28T13:01:56.959079+0000","last_active":"2023-11-28T13:01:57.938701+0000","last_peered":"2023-11-28T13:01:57.938701+0000","last_clean":"2023-11-28T13:01:57.938701+0000","last_became_active":"2023-11-28T13:01:56.958778+0000","last_became_peered":"2023-11-28T13:01:56.958778+0000","last_unstale":"2023-11-28T13:01:57.938701+0000","last_undegraded":"2023-11-28T13:01:57.938701+0000","last_fullsized":"2023-11-28T13:01:57.938701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:05.390821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.947102+0000","last_change":"2023-11-28T13:01:56.551775+0000","last_active":"2023-11-28T13:01:56.947102+0000","last_peered":"2023-11-28T13:01:56.947102+0000","last_clean":"2023-11-28T13:01:56.947102+0000","last_became_active":"2023-11-28T13:01:56.551487+0000","last_became_peered":"2023-11-28T13:01:56.551487+0000","last_unstale":"2023-11-28T13:01:56.947102+0000","last_undegraded":"2023-11-28T13:01:56.947102+0000","last_fullsized":"2023-11-28T13:01:56.947102+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:53.578242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961285+0000","last_change":"2023-11-28T13:01:56.955950+0000","last_active":"2023-11-28T13:01:56.961285+0000","last_peered":"2023-11-28T13:01:56.961285+0000","last_clean":"2023-11-28T13:01:56.961285+0000","last_became_active":"2023-11-28T13:01:56.955695+0000","last_became_peered":"2023-11-28T13:01:56.955695+0000","last_unstale":"2023-11-28T13:01:56.961285+0000","last_undegraded":"2023-11-28T13:01:56.961285+0000","last_fullsized":"2023-11-28T13:01:56.961285+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:19.476807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963619+0000","last_change":"2023-11-28T13:01:56.555586+0000","last_active":"2023-11-28T13:01:56.963619+0000","last_peered":"2023-11-28T13:01:56.963619+0000","last_clean":"2023-11-28T13:01:56.963619+0000","last_became_active":"2023-11-28T13:01:56.555280+0000","last_became_peered":"2023-11-28T13:01:56.555280+0000","last_unstale":"2023-11-28T13:01:56.963619+0000","last_undegraded":"2023-11-28T13:01:56.963619+0000","last_fullsized":"2023-11-28T13:01:56.963619+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:15.865114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961514+0000","last_change":"2023-11-28T13:01:56.957853+0000","last_active":"2023-11-28T13:01:56.961514+0000","last_peered":"2023-11-28T13:01:56.961514+0000","last_clean":"2023-11-28T13:01:56.961514+0000","last_became_active":"2023-11-28T13:01:56.957504+0000","last_became_peered":"2023-11-28T13:01:56.957504+0000","last_unstale":"2023-11-28T13:01:56.961514+0000","last_undegraded":"2023-11-28T13:01:56.961514+0000","last_fullsized":"2023-11-28T13:01:56.961514+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:01.873986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939226+0000","last_change":"2023-11-28T13:01:56.547924+0000","last_active":"2023-11-28T13:01:57.939226+0000","last_peered":"2023-11-28T13:01:57.939226+0000","last_clean":"2023-11-28T13:01:57.939226+0000","last_became_active":"2023-11-28T13:01:56.547584+0000","last_became_peered":"2023-11-28T13:01:56.547584+0000","last_unstale":"2023-11-28T13:01:57.939226+0000","last_undegraded":"2023-11-28T13:01:57.939226+0000","last_fullsized":"2023-11-28T13:01:57.939226+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:59.635818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938967+0000","last_change":"2023-11-28T13:01:56.955534+0000","last_active":"2023-11-28T13:01:57.938967+0000","last_peered":"2023-11-28T13:01:57.938967+0000","last_clean":"2023-11-28T13:01:57.938967+0000","last_became_active":"2023-11-28T13:01:56.955324+0000","last_became_peered":"2023-11-28T13:01:56.955324+0000","last_unstale":"2023-11-28T13:01:57.938967+0000","last_undegraded":"2023-11-28T13:01:57.938967+0000","last_fullsized":"2023-11-28T13:01:57.938967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:51.796815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.957702+0000","last_change":"2023-11-28T13:01:56.551353+0000","last_active":"2023-11-28T13:01:56.957702+0000","last_peered":"2023-11-28T13:01:56.957702+0000","last_clean":"2023-11-28T13:01:56.957702+0000","last_became_active":"2023-11-28T13:01:56.551052+0000","last_became_peered":"2023-11-28T13:01:56.551052+0000","last_unstale":"2023-11-28T13:01:56.957702+0000","last_undegraded":"2023-11-28T13:01:56.957702+0000","last_fullsized":"2023-11-28T13:01:56.957702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:21.743224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955961+0000","last_change":"2023-11-28T13:01:56.952020+0000","last_active":"2023-11-28T13:01:56.955961+0000","last_peered":"2023-11-28T13:01:56.955961+0000","last_clean":"2023-11-28T13:01:56.955961+0000","last_became_active":"2023-11-28T13:01:56.948677+0000","last_became_peered":"2023-11-28T13:01:56.948677+0000","last_unstale":"2023-11-28T13:01:56.955961+0000","last_undegraded":"2023-11-28T13:01:56.955961+0000","last_fullsized":"2023-11-28T13:01:56.955961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:07.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939658+0000","last_change":"2023-11-28T13:01:56.552019+0000","last_active":"2023-11-28T13:01:57.939658+0000","last_peered":"2023-11-28T13:01:57.939658+0000","last_clean":"2023-11-28T13:01:57.939658+0000","last_became_active":"2023-11-28T13:01:56.551752+0000","last_became_peered":"2023-11-28T13:01:56.551752+0000","last_unstale":"2023-11-28T13:01:57.939658+0000","last_undegraded":"2023-11-28T13:01:57.939658+0000","last_fullsized":"2023-11-28T13:01:57.939658+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:20.944063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939687+0000","last_change":"2023-11-28T13:01:56.957281+0000","last_active":"2023-11-28T13:01:57.939687+0000","last_peered":"2023-11-28T13:01:57.939687+0000","last_clean":"2023-11-28T13:01:57.939687+0000","last_became_active":"2023-11-28T13:01:56.956996+0000","last_became_peered":"2023-11-28T13:01:56.956996+0000","last_unstale":"2023-11-28T13:01:57.939687+0000","last_undegraded":"2023-11-28T13:01:57.939687+0000","last_fullsized":"2023-11-28T13:01:57.939687+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:03.801800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.952603+0000","last_change":"2023-11-28T13:01:56.553248+0000","last_active":"2023-11-28T13:01:56.952603+0000","last_peered":"2023-11-28T13:01:56.952603+0000","last_clean":"2023-11-28T13:01:56.952603+0000","last_became_active":"2023-11-28T13:01:56.552959+0000","last_became_peered":"2023-11-28T13:01:56.552959+0000","last_unstale":"2023-11-28T13:01:56.952603+0000","last_undegraded":"2023-11-28T13:01:56.952603+0000","last_fullsized":"2023-11-28T13:01:56.952603+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:40.691336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.961504+0000","last_change":"2023-11-28T13:01:56.956195+0000","last_active":"2023-11-28T13:01:56.961504+0000","last_peered":"2023-11-28T13:01:56.961504+0000","last_clean":"2023-11-28T13:01:56.961504+0000","last_became_active":"2023-11-28T13:01:56.955250+0000","last_became_peered":"2023-11-28T13:01:56.955250+0000","last_unstale":"2023-11-28T13:01:56.961504+0000","last_undegraded":"2023-11-28T13:01:56.961504+0000","last_fullsized":"2023-11-28T13:01:56.961504+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:09.136546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939177+0000","last_change":"2023-11-28T13:01:56.548096+0000","last_active":"2023-11-28T13:01:57.939177+0000","last_peered":"2023-11-28T13:01:57.939177+0000","last_clean":"2023-11-28T13:01:57.939177+0000","last_became_active":"2023-11-28T13:01:56.547764+0000","last_became_peered":"2023-11-28T13:01:56.547764+0000","last_unstale":"2023-11-28T13:01:57.939177+0000","last_undegraded":"2023-11-28T13:01:57.939177+0000","last_fullsized":"2023-11-28T13:01:57.939177+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:24.575404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939019+0000","last_change":"2023-11-28T13:01:56.957561+0000","last_active":"2023-11-28T13:01:57.939019+0000","last_peered":"2023-11-28T13:01:57.939019+0000","last_clean":"2023-11-28T13:01:57.939019+0000","last_became_active":"2023-11-28T13:01:56.957035+0000","last_became_peered":"2023-11-28T13:01:56.957035+0000","last_unstale":"2023-11-28T13:01:57.939019+0000","last_undegraded":"2023-11-28T13:01:57.939019+0000","last_fullsized":"2023-11-28T13:01:57.939019+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:11.705221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958899+0000","last_change":"2023-11-28T13:01:56.557848+0000","last_active":"2023-11-28T13:01:56.958899+0000","last_peered":"2023-11-28T13:01:56.958899+0000","last_clean":"2023-11-28T13:01:56.958899+0000","last_became_active":"2023-11-28T13:01:56.557561+0000","last_became_peered":"2023-11-28T13:01:56.557561+0000","last_unstale":"2023-11-28T13:01:56.958899+0000","last_undegraded":"2023-11-28T13:01:56.958899+0000","last_fullsized":"2023-11-28T13:01:56.958899+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:52.273027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328286+0000","last_change":"2023-11-28T13:01:56.961601+0000","last_active":"2023-11-28T13:01:59.328286+0000","last_peered":"2023-11-28T13:01:59.328286+0000","last_clean":"2023-11-28T13:01:59.328286+0000","last_became_active":"2023-11-28T13:01:56.961349+0000","last_became_peered":"2023-11-28T13:01:56.961349+0000","last_unstale":"2023-11-28T13:01:59.328286+0000","last_undegraded":"2023-11-28T13:01:59.328286+0000","last_fullsized":"2023-11-28T13:01:59.328286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:27:57.740216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329218+0000","last_change":"2023-11-28T13:01:56.544596+0000","last_active":"2023-11-28T13:01:59.329218+0000","last_peered":"2023-11-28T13:01:59.329218+0000","last_clean":"2023-11-28T13:01:59.329218+0000","last_became_active":"2023-11-28T13:01:56.544304+0000","last_became_peered":"2023-11-28T13:01:56.544304+0000","last_unstale":"2023-11-28T13:01:59.329218+0000","last_undegraded":"2023-11-28T13:01:59.329218+0000","last_fullsized":"2023-11-28T13:01:59.329218+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:22.175750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.963227+0000","last_change":"2023-11-28T13:01:56.958793+0000","last_active":"2023-11-28T13:01:56.963227+0000","last_peered":"2023-11-28T13:01:56.963227+0000","last_clean":"2023-11-28T13:01:56.963227+0000","last_became_active":"2023-11-28T13:01:56.958502+0000","last_became_peered":"2023-11-28T13:01:56.958502+0000","last_unstale":"2023-11-28T13:01:56.963227+0000","last_undegraded":"2023-11-28T13:01:56.963227+0000","last_fullsized":"2023-11-28T13:01:56.963227+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:05.755474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939935+0000","last_change":"2023-11-28T13:01:56.554581+0000","last_active":"2023-11-28T13:01:57.939935+0000","last_peered":"2023-11-28T13:01:57.939935+0000","last_clean":"2023-11-28T13:01:57.939935+0000","last_became_active":"2023-11-28T13:01:56.554153+0000","last_became_peered":"2023-11-28T13:01:56.554153+0000","last_unstale":"2023-11-28T13:01:57.939935+0000","last_undegraded":"2023-11-28T13:01:57.939935+0000","last_fullsized":"2023-11-28T13:01:57.939935+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:11:38.731848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329318+0000","last_change":"2023-11-28T13:01:56.955539+0000","last_active":"2023-11-28T13:01:59.329318+0000","last_peered":"2023-11-28T13:01:59.329318+0000","last_clean":"2023-11-28T13:01:59.329318+0000","last_became_active":"2023-11-28T13:01:56.955271+0000","last_became_peered":"2023-11-28T13:01:56.955271+0000","last_unstale":"2023-11-28T13:01:59.329318+0000","last_undegraded":"2023-11-28T13:01:59.329318+0000","last_fullsized":"2023-11-28T13:01:59.329318+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:41:44.811615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.940003+0000","last_change":"2023-11-28T13:01:56.547245+0000","last_active":"2023-11-28T13:01:57.940003+0000","last_peered":"2023-11-28T13:01:57.940003+0000","last_clean":"2023-11-28T13:01:57.940003+0000","last_became_active":"2023-11-28T13:01:56.546961+0000","last_became_peered":"2023-11-28T13:01:56.546961+0000","last_unstale":"2023-11-28T13:01:57.940003+0000","last_undegraded":"2023-11-28T13:01:57.940003+0000","last_fullsized":"2023-11-28T13:01:57.940003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:43.494941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938719+0000","last_change":"2023-11-28T13:01:56.961235+0000","last_active":"2023-11-28T13:01:57.938719+0000","last_peered":"2023-11-28T13:01:57.938719+0000","last_clean":"2023-11-28T13:01:57.938719+0000","last_became_active":"2023-11-28T13:01:56.960977+0000","last_became_peered":"2023-11-28T13:01:56.960977+0000","last_unstale":"2023-11-28T13:01:57.938719+0000","last_undegraded":"2023-11-28T13:01:57.938719+0000","last_fullsized":"2023-11-28T13:01:57.938719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:07.713158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329225+0000","last_change":"2023-11-28T13:01:56.553232+0000","last_active":"2023-11-28T13:01:59.329225+0000","last_peered":"2023-11-28T13:01:59.329225+0000","last_clean":"2023-11-28T13:01:59.329225+0000","last_became_active":"2023-11-28T13:01:56.552959+0000","last_became_peered":"2023-11-28T13:01:56.552959+0000","last_unstale":"2023-11-28T13:01:59.329225+0000","last_undegraded":"2023-11-28T13:01:59.329225+0000","last_fullsized":"2023-11-28T13:01:59.329225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:04:31.186841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939851+0000","last_change":"2023-11-28T13:01:56.959192+0000","last_active":"2023-11-28T13:01:57.939851+0000","last_peered":"2023-11-28T13:01:57.939851+0000","last_clean":"2023-11-28T13:01:57.939851+0000","last_became_active":"2023-11-28T13:01:56.958904+0000","last_became_peered":"2023-11-28T13:01:56.958904+0000","last_unstale":"2023-11-28T13:01:57.939851+0000","last_undegraded":"2023-11-28T13:01:57.939851+0000","last_fullsized":"2023-11-28T13:01:57.939851+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:00:41.696896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939060+0000","last_change":"2023-11-28T13:01:56.548479+0000","last_active":"2023-11-28T13:01:57.939060+0000","last_peered":"2023-11-28T13:01:57.939060+0000","last_clean":"2023-11-28T13:01:57.939060+0000","last_became_active":"2023-11-28T13:01:56.548230+0000","last_became_peered":"2023-11-28T13:01:56.548230+0000","last_unstale":"2023-11-28T13:01:57.939060+0000","last_undegraded":"2023-11-28T13:01:57.939060+0000","last_fullsized":"2023-11-28T13:01:57.939060+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:27.457178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.958309+0000","last_change":"2023-11-28T13:01:56.953065+0000","last_active":"2023-11-28T13:01:56.958309+0000","last_peered":"2023-11-28T13:01:56.958309+0000","last_clean":"2023-11-28T13:01:56.958309+0000","last_became_active":"2023-11-28T13:01:56.952812+0000","last_became_peered":"2023-11-28T13:01:56.952812+0000","last_unstale":"2023-11-28T13:01:56.958309+0000","last_undegraded":"2023-11-28T13:01:56.958309+0000","last_fullsized":"2023-11-28T13:01:56.958309+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:44.202342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945091+0000","last_change":"2023-11-28T13:01:56.557972+0000","last_active":"2023-11-28T13:01:56.945091+0000","last_peered":"2023-11-28T13:01:56.945091+0000","last_clean":"2023-11-28T13:01:56.945091+0000","last_became_active":"2023-11-28T13:01:56.557694+0000","last_became_peered":"2023-11-28T13:01:56.557694+0000","last_unstale":"2023-11-28T13:01:56.945091+0000","last_undegraded":"2023-11-28T13:01:56.945091+0000","last_fullsized":"2023-11-28T13:01:56.945091+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:56.755448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328988+0000","last_change":"2023-11-28T13:01:56.949449+0000","last_active":"2023-11-28T13:01:59.328988+0000","last_peered":"2023-11-28T13:01:59.328988+0000","last_clean":"2023-11-28T13:01:59.328988+0000","last_became_active":"2023-11-28T13:01:56.949158+0000","last_became_peered":"2023-11-28T13:01:56.949158+0000","last_unstale":"2023-11-28T13:01:59.328988+0000","last_undegraded":"2023-11-28T13:01:59.328988+0000","last_fullsized":"2023-11-28T13:01:59.328988+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:25.580964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.329436+0000","last_change":"2023-11-28T13:01:56.550418+0000","last_active":"2023-11-28T13:01:59.329436+0000","last_peered":"2023-11-28T13:01:59.329436+0000","last_clean":"2023-11-28T13:01:59.329436+0000","last_became_active":"2023-11-28T13:01:56.550121+0000","last_became_peered":"2023-11-28T13:01:56.550121+0000","last_unstale":"2023-11-28T13:01:59.329436+0000","last_undegraded":"2023-11-28T13:01:59.329436+0000","last_fullsized":"2023-11-28T13:01:59.329436+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:25.719801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.956008+0000","last_change":"2023-11-28T13:01:56.948792+0000","last_active":"2023-11-28T13:01:56.956008+0000","last_peered":"2023-11-28T13:01:56.956008+0000","last_clean":"2023-11-28T13:01:56.956008+0000","last_became_active":"2023-11-28T13:01:56.948483+0000","last_became_peered":"2023-11-28T13:01:56.948483+0000","last_unstale":"2023-11-28T13:01:56.956008+0000","last_undegraded":"2023-11-28T13:01:56.956008+0000","last_fullsized":"2023-11-28T13:01:56.956008+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:10:43.663763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945673+0000","last_change":"2023-11-28T13:01:56.547024+0000","last_active":"2023-11-28T13:01:56.945673+0000","last_peered":"2023-11-28T13:01:56.945673+0000","last_clean":"2023-11-28T13:01:56.945673+0000","last_became_active":"2023-11-28T13:01:56.546734+0000","last_became_peered":"2023-11-28T13:01:56.546734+0000","last_unstale":"2023-11-28T13:01:56.945673+0000","last_undegraded":"2023-11-28T13:01:56.945673+0000","last_fullsized":"2023-11-28T13:01:56.945673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:21.743224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.935255+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:01:56.935255+0000","last_peered":"2023-11-28T13:01:56.935255+0000","last_clean":"2023-11-28T13:01:56.935255+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:01:56.935255+0000","last_undegraded":"2023-11-28T13:01:56.935255+0000","last_fullsized":"2023-11-28T13:01:56.935255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938770+0000","last_change":"2023-11-28T13:01:56.960019+0000","last_active":"2023-11-28T13:01:57.938770+0000","last_peered":"2023-11-28T13:01:57.938770+0000","last_clean":"2023-11-28T13:01:57.938770+0000","last_became_active":"2023-11-28T13:01:56.959748+0000","last_became_peered":"2023-11-28T13:01:56.959748+0000","last_unstale":"2023-11-28T13:01:57.938770+0000","last_undegraded":"2023-11-28T13:01:57.938770+0000","last_fullsized":"2023-11-28T13:01:57.938770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.243375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939796+0000","last_change":"2023-11-28T13:01:56.551638+0000","last_active":"2023-11-28T13:01:57.939796+0000","last_peered":"2023-11-28T13:01:57.939796+0000","last_clean":"2023-11-28T13:01:57.939796+0000","last_became_active":"2023-11-28T13:01:56.551278+0000","last_became_peered":"2023-11-28T13:01:56.551278+0000","last_unstale":"2023-11-28T13:01:57.939796+0000","last_undegraded":"2023-11-28T13:01:57.939796+0000","last_fullsized":"2023-11-28T13:01:57.939796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:05:36.670635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945115+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:01:56.945115+0000","last_peered":"2023-11-28T13:01:56.945115+0000","last_clean":"2023-11-28T13:01:56.945115+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:01:56.945115+0000","last_undegraded":"2023-11-28T13:01:56.945115+0000","last_fullsized":"2023-11-28T13:01:56.945115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939189+0000","last_change":"2023-11-28T13:01:56.959427+0000","last_active":"2023-11-28T13:01:57.939189+0000","last_peered":"2023-11-28T13:01:57.939189+0000","last_clean":"2023-11-28T13:01:57.939189+0000","last_became_active":"2023-11-28T13:01:56.959197+0000","last_became_peered":"2023-11-28T13:01:56.959197+0000","last_unstale":"2023-11-28T13:01:57.939189+0000","last_undegraded":"2023-11-28T13:01:57.939189+0000","last_fullsized":"2023-11-28T13:01:57.939189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:06.623917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.939708+0000","last_change":"2023-11-28T13:01:56.553782+0000","last_active":"2023-11-28T13:01:56.939708+0000","last_peered":"2023-11-28T13:01:56.939708+0000","last_clean":"2023-11-28T13:01:56.939708+0000","last_became_active":"2023-11-28T13:01:56.553175+0000","last_became_peered":"2023-11-28T13:01:56.553175+0000","last_unstale":"2023-11-28T13:01:56.939708+0000","last_undegraded":"2023-11-28T13:01:56.939708+0000","last_fullsized":"2023-11-28T13:01:56.939708+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:13:34.785993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.938103+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:01:56.938103+0000","last_peered":"2023-11-28T13:01:56.938103+0000","last_clean":"2023-11-28T13:01:56.938103+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:01:56.938103+0000","last_undegraded":"2023-11-28T13:01:56.938103+0000","last_fullsized":"2023-11-28T13:01:56.938103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939794+0000","last_change":"2023-11-28T13:01:56.955397+0000","last_active":"2023-11-28T13:01:57.939794+0000","last_peered":"2023-11-28T13:01:57.939794+0000","last_clean":"2023-11-28T13:01:57.939794+0000","last_became_active":"2023-11-28T13:01:56.955082+0000","last_became_peered":"2023-11-28T13:01:56.955082+0000","last_unstale":"2023-11-28T13:01:57.939794+0000","last_undegraded":"2023-11-28T13:01:57.939794+0000","last_fullsized":"2023-11-28T13:01:57.939794+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:38:45.297181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.939490+0000","last_change":"2023-11-28T13:01:56.553790+0000","last_active":"2023-11-28T13:01:56.939490+0000","last_peered":"2023-11-28T13:01:56.939490+0000","last_clean":"2023-11-28T13:01:56.939490+0000","last_became_active":"2023-11-28T13:01:56.553497+0000","last_became_peered":"2023-11-28T13:01:56.553497+0000","last_unstale":"2023-11-28T13:01:56.939490+0000","last_undegraded":"2023-11-28T13:01:56.939490+0000","last_fullsized":"2023-11-28T13:01:56.939490+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:53.578242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.945239+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:01:56.945239+0000","last_peered":"2023-11-28T13:01:56.945239+0000","last_clean":"2023-11-28T13:01:56.945239+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:01:56.945239+0000","last_undegraded":"2023-11-28T13:01:56.945239+0000","last_fullsized":"2023-11-28T13:01:56.945239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.547187+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:01:56.547187+0000","last_peered":"2023-11-28T13:01:56.547187+0000","last_clean":"2023-11-28T13:01:56.547187+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:01:56.547187+0000","last_undegraded":"2023-11-28T13:01:56.547187+0000","last_fullsized":"2023-11-28T13:01:56.547187+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:59.328477+0000","last_change":"2023-11-28T13:01:56.940142+0000","last_active":"2023-11-28T13:01:59.328477+0000","last_peered":"2023-11-28T13:01:59.328477+0000","last_clean":"2023-11-28T13:01:59.328477+0000","last_became_active":"2023-11-28T13:01:56.939845+0000","last_became_peered":"2023-11-28T13:01:56.939845+0000","last_unstale":"2023-11-28T13:01:59.328477+0000","last_undegraded":"2023-11-28T13:01:59.328477+0000","last_fullsized":"2023-11-28T13:01:59.328477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:00.641378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.953147+0000","last_change":"2023-11-28T13:01:56.555323+0000","last_active":"2023-11-28T13:01:56.953147+0000","last_peered":"2023-11-28T13:01:56.953147+0000","last_clean":"2023-11-28T13:01:56.953147+0000","last_became_active":"2023-11-28T13:01:56.555065+0000","last_became_peered":"2023-11-28T13:01:56.555065+0000","last_unstale":"2023-11-28T13:01:56.953147+0000","last_undegraded":"2023-11-28T13:01:56.953147+0000","last_fullsized":"2023-11-28T13:01:56.953147+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:33:58.072158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.547091+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:01:56.547091+0000","last_peered":"2023-11-28T13:01:56.547091+0000","last_clean":"2023-11-28T13:01:56.547091+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:01:56.547091+0000","last_undegraded":"2023-11-28T13:01:56.547091+0000","last_fullsized":"2023-11-28T13:01:56.547091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939472+0000","last_change":"2023-11-28T13:01:56.947502+0000","last_active":"2023-11-28T13:01:57.939472+0000","last_peered":"2023-11-28T13:01:57.939472+0000","last_clean":"2023-11-28T13:01:57.939472+0000","last_became_active":"2023-11-28T13:01:56.947264+0000","last_became_peered":"2023-11-28T13:01:56.947264+0000","last_unstale":"2023-11-28T13:01:57.939472+0000","last_undegraded":"2023-11-28T13:01:57.939472+0000","last_fullsized":"2023-11-28T13:01:57.939472+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:27:18.348401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938879+0000","last_change":"2023-11-28T13:01:56.551111+0000","last_active":"2023-11-28T13:01:57.938879+0000","last_peered":"2023-11-28T13:01:57.938879+0000","last_clean":"2023-11-28T13:01:57.938879+0000","last_became_active":"2023-11-28T13:01:56.550805+0000","last_became_peered":"2023-11-28T13:01:56.550805+0000","last_unstale":"2023-11-28T13:01:57.938879+0000","last_undegraded":"2023-11-28T13:01:57.938879+0000","last_fullsized":"2023-11-28T13:01:57.938879+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:41:18.107890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.549017+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:01:56.549017+0000","last_peered":"2023-11-28T13:01:56.549017+0000","last_clean":"2023-11-28T13:01:56.549017+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:01:56.549017+0000","last_undegraded":"2023-11-28T13:01:56.549017+0000","last_fullsized":"2023-11-28T13:01:56.549017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.955854+0000","last_change":"2023-11-28T13:01:56.951933+0000","last_active":"2023-11-28T13:01:56.955854+0000","last_peered":"2023-11-28T13:01:56.955854+0000","last_clean":"2023-11-28T13:01:56.955854+0000","last_became_active":"2023-11-28T13:01:56.948556+0000","last_became_peered":"2023-11-28T13:01:56.948556+0000","last_unstale":"2023-11-28T13:01:56.955854+0000","last_undegraded":"2023-11-28T13:01:56.955854+0000","last_fullsized":"2023-11-28T13:01:56.955854+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:40.918026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939210+0000","last_change":"2023-11-28T13:01:56.548143+0000","last_active":"2023-11-28T13:01:57.939210+0000","last_peered":"2023-11-28T13:01:57.939210+0000","last_clean":"2023-11-28T13:01:57.939210+0000","last_became_active":"2023-11-28T13:01:56.547663+0000","last_became_peered":"2023-11-28T13:01:56.547663+0000","last_unstale":"2023-11-28T13:01:57.939210+0000","last_undegraded":"2023-11-28T13:01:57.939210+0000","last_fullsized":"2023-11-28T13:01:57.939210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:58:32.983258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.935661+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:01:56.935661+0000","last_peered":"2023-11-28T13:01:56.935661+0000","last_clean":"2023-11-28T13:01:56.935661+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:01:56.935661+0000","last_undegraded":"2023-11-28T13:01:56.935661+0000","last_fullsized":"2023-11-28T13:01:56.935661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.938726+0000","last_change":"2023-11-28T13:01:56.956241+0000","last_active":"2023-11-28T13:01:57.938726+0000","last_peered":"2023-11-28T13:01:57.938726+0000","last_clean":"2023-11-28T13:01:57.938726+0000","last_became_active":"2023-11-28T13:01:56.955742+0000","last_became_peered":"2023-11-28T13:01:56.955742+0000","last_unstale":"2023-11-28T13:01:57.938726+0000","last_undegraded":"2023-11-28T13:01:57.938726+0000","last_fullsized":"2023-11-28T13:01:57.938726+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:56.528749+0000","last_clean_scrub_stamp":"2023-11-28T13:01:56.528749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:05.755474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T13:01:57.939085+0000","last_change":"2023-11-28T13:01:56.554201+0000","last_active":"2023-11-28T13:01:57.939085+0000","last_peered":"2023-11-28T13:01:57.939085+0000","last_clean":"2023-11-28T13:01:57.939085+0000","last_became_active":"2023-11-28T13:01:56.553924+0000","last_became_peered":"2023-11-28T13:01:56.553924+0000","last_unstale":"2023-11-28T13:01:57.939085+0000","last_undegraded":"2023-11-28T13:01:57.939085+0000","last_fullsized":"2023-11-28T13:01:57.939085+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:55.523189+0000","last_clean_scrub_stamp":"2023-11-28T13:01:55.523189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:19:22.180181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-11-28T13:01:56.948262+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:01:56.948262+0000","last_peered":"2023-11-28T13:01:56.948262+0000","last_clean":"2023-11-28T13:01:56.948262+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:01:56.948262+0000","last_undegraded":"2023-11-28T13:01:56.948262+0000","last_fullsized":"2023-11-28T13:01:56.948262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9180,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362660,"statfs":{"total":96636764160,"available":96627363840,"internally_reserved":0,"allocated":610304,"data_stored":256967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":8.218,"5min":8.218,"15min":8.218},"last":0.34399999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.36199999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":13.384,"5min":13.384,"15min":13.384},"last":0.378},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.226,"5min":5.226,"15min":5.226},"last":0.41299999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.3609999999999998,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.1579999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.3209999999999997,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.623},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":8.1029999999999998,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.3849999999999998,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.67000000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8908,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362932,"statfs":{"total":96636764160,"available":96627642368,"internally_reserved":0,"allocated":593920,"data_stored":250292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.318,"15min":0.318},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.251}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.373}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.308},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35999999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.495},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47299999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9028,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362812,"statfs":{"total":96636764160,"available":96627519488,"internally_reserved":0,"allocated":585728,"data_stored":249310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.50800000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.66200000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.624},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.69399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14004,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94357836,"statfs":{"total":96636764160,"available":96622424064,"internally_reserved":0,"allocated":1159168,"data_stored":831215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":14.997999999999999,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":19.045999999999999,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":18.995000000000001,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":19.02,"5min":19.02,"15min":19.02},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.2359999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.504},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.1150000000000002,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.1539999999999999,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":18.957999999999998,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.68400000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10188,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361652,"statfs":{"total":96636764160,"available":96626331648,"internally_reserved":0,"allocated":1183744,"data_stored":839236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.503}]},{"osd":1,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.44},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.48799999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9036,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362804,"statfs":{"total":96636764160,"available":96627511296,"internally_reserved":0,"allocated":593920,"data_stored":255397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.624}]},{"osd":3,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.372}]},{"osd":6,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.499}]},{"osd":7,"last update":"Tue Nov 28 13:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48399999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":602112,"data_stored":256033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.56699999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.443}]},{"osd":3,"last update":"Tue Nov 28 13:01:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.499}]},{"osd":5,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.628},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.65100000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8968,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362872,"statfs":{"total":96636764160,"available":96627580928,"internally_reserved":0,"allocated":589824,"data_stored":248904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.27100000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.32000000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.52400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:02:01.382 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:02:01.382 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:02:01.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.544+0000 7febc5c60700 1 Processor -- start 2023-11-28T13:02:01.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.544+0000 7febc5c60700 1 -- start start 2023-11-28T13:02:01.547 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.544+0000 7febc5c60700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7febc0161340 0x7febc00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.547 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.544+0000 7febc5c60700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc00d1950 con 0x7febc015c9b0 2023-11-28T13:02:01.547 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febc5c60700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc00b5520 con 0x7febc00d2320 2023-11-28T13:02:01.547 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febc5c60700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc00b5800 con 0x7febc0161340 2023-11-28T13:02:01.547 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febb7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7febc0161340 0x7febc00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.547 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febbf7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7febc015c9b0 0x7febc00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59410/0 (socket says 172.21.15.153:59410) 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febbf7fe700 1 -- 172.21.15.153:0/95025586 learned_addr learned my addr 172.21.15.153:0/95025586 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febb7fff700 1 -- 172.21.15.153:0/95025586 >> v1:172.21.15.177:6789/0 conn(0x7febc00d2320 legacy=0x7febc00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febb7fff700 1 -- 172.21.15.153:0/95025586 >> v1:172.21.15.153:6789/0 conn(0x7febc015c9b0 legacy=0x7febc00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febb7fff700 1 -- 172.21.15.153:0/95025586 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc00d1300 con 0x7febc0161340 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febbeffd700 1 -- 172.21.15.153:0/95025586 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345567647 0 0) 0x7febc00d1950 con 0x7febc015c9b0 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febb7fff700 1 --2- 172.21.15.153:0/95025586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0161340 0x7febc00d1de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feba8009840 tx=0x7feba8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1df4a6a8dadff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febbeffd700 1 -- 172.21.15.153:0/95025586 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8011830 con 0x7febc0161340 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febbeffd700 1 -- 172.21.15.153:0/95025586 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba8011990 con 0x7febc0161340 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.545+0000 7febbeffd700 1 -- 172.21.15.153:0/95025586 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8011d20 con 0x7febc0161340 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.546+0000 7febc5c60700 1 -- 172.21.15.153:0/95025586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0161340 msgr2=0x7febc00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.546+0000 7febc5c60700 1 --2- 172.21.15.153:0/95025586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0161340 0x7febc00d1de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feba8009840 tx=0x7feba8009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:01.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.546+0000 7febc5c60700 1 -- 172.21.15.153:0/95025586 shutdown_connections 2023-11-28T13:02:01.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.546+0000 7febc5c60700 1 --2- 172.21.15.153:0/95025586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc0161340 0x7febc00d1de0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.546+0000 7febc5c60700 1 -- 172.21.15.153:0/95025586 >> 172.21.15.153:0/95025586 conn(0x7febc0072600 msgr2=0x7febc00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:01.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.546+0000 7febc5c60700 1 -- 172.21.15.153:0/95025586 shutdown_connections 2023-11-28T13:02:01.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.546+0000 7febc5c60700 1 -- 172.21.15.153:0/95025586 wait complete. 2023-11-28T13:02:01.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.547+0000 7febc5c60700 1 Processor -- start 2023-11-28T13:02:01.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.548+0000 7febc5c60700 1 -- start start 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.548+0000 7febc5c60700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febc00d2320 0x7febc015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.548+0000 7febc5c60700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc015c9b0 0x7febc01e8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.548+0000 7febc5c60700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febc0161340 0x7febc00b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.548+0000 7febc5c60700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc015f430 con 0x7febc015c9b0 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.548+0000 7febc5c60700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01e6ff0 con 0x7febc0161340 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.548+0000 7febc5c60700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc01e72d0 con 0x7febc00d2320 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc015c9b0 0x7febc01e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc015c9b0 0x7febc01e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41254/0 (socket says 172.21.15.153:41254) 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 -- 172.21.15.153:0/3658093771 learned_addr learned my addr 172.21.15.153:0/3658093771 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febbffff700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febc0161340 0x7febc00b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febbf7fe700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febc00d2320 0x7febc015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 -- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febc00d2320 msgr2=0x7febc015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febc00d2320 0x7febc015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 -- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febc0161340 msgr2=0x7febc00b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febc0161340 0x7febc00b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 -- 172.21.15.153:0/3658093771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80094a0 con 0x7febc015c9b0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febb7fff700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc015c9b0 0x7febc01e8090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feba8014040 tx=0x7feba8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febbcff9700 1 -- 172.21.15.153:0/3658093771 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba801a980 con 0x7febc015c9b0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febbcff9700 1 -- 172.21.15.153:0/3658093771 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba801ab00 con 0x7febc015c9b0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc015cd80 con 0x7febc015c9b0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febbcff9700 1 -- 172.21.15.153:0/3658093771 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8024420 con 0x7febc015c9b0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.549+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc00d15e0 con 0x7febc015c9b0 2023-11-28T13:02:01.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.550+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc015cf10 con 0x7febc015c9b0 2023-11-28T13:02:01.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.550+0000 7febbcff9700 1 -- 172.21.15.153:0/3658093771 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feba802b030 con 0x7febc015c9b0 2023-11-28T13:02:01.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.551+0000 7febbcff9700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feba00682f0 0x7feba006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.551+0000 7febbcff9700 1 -- 172.21.15.153:0/3658093771 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feba8086fc0 con 0x7febc015c9b0 2023-11-28T13:02:01.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.551+0000 7febbf7fe700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feba00682f0 0x7feba006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.551+0000 7febbf7fe700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feba00682f0 0x7feba006a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f60 tx=0x7febb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.555+0000 7febbcff9700 1 -- 172.21.15.153:0/3658093771 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feba8053820 con 0x7febc015c9b0 2023-11-28T13:02:01.814 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.813+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7febc00745c0 con 0x7febc015c9b0 2023-11-28T13:02:01.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.817+0000 7febbcff9700 1 -- 172.21.15.153:0/3658093771 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7feba80150e0 con 0x7febc015c9b0 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.819+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feba00682f0 msgr2=0x7feba006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.819+0000 7febc5c60700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feba00682f0 0x7feba006a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f60 tx=0x7febb0009040 comp rx=0 tx=0).stop 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.819+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc015c9b0 msgr2=0x7febc01e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.819+0000 7febc5c60700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc015c9b0 0x7febc01e8090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feba8014040 tx=0x7feba8004750 comp rx=0 tx=0).stop 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 shutdown_connections 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febc0161340 0x7febc00b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febc00d2320 0x7febc015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7feba00682f0 0x7feba006a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 --2- 172.21.15.153:0/3658093771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febc015c9b0 0x7febc01e8090 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 >> 172.21.15.153:0/3658093771 conn(0x7febc0072600 msgr2=0x7febc0160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 shutdown_connections 2023-11-28T13:02:01.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.820+0000 7febc5c60700 1 -- 172.21.15.153:0/3658093771 wait complete. 2023-11-28T13:02:01.822 INFO:teuthology.orchestra.run.smithi153.stderr:updated caps for client.0 2023-11-28T13:02:01.832 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:01.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.952+0000 7f2972242700 1 Processor -- start 2023-11-28T13:02:01.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.952+0000 7f2972242700 1 -- start start 2023-11-28T13:02:01.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.952+0000 7f2972242700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f296c0cd8c0 0x7f296c140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.952+0000 7f2972242700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cdc90 0x7f296c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.952+0000 7f2972242700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c13ff00 con 0x7f296c0b2e70 2023-11-28T13:02:01.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.952+0000 7f2972242700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c1404c0 con 0x7f296c0cd8c0 2023-11-28T13:02:01.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.952+0000 7f2972242700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c15aa80 con 0x7f296c0cdc90 2023-11-28T13:02:01.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f296c0cd8c0 0x7f296c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f296c0b2e70 0x7f296c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59428/0 (socket says 172.21.15.153:59428) 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296bfff700 1 -- 172.21.15.153:0/3657355285 learned_addr learned my addr 172.21.15.153:0/3657355285 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296affd700 1 --2- 172.21.15.153:0/3657355285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cdc90 0x7f296c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296b7fe700 1 -- 172.21.15.153:0/3657355285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cdc90 msgr2=0x7f296c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296b7fe700 1 --2- 172.21.15.153:0/3657355285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cdc90 0x7f296c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296b7fe700 1 -- 172.21.15.153:0/3657355285 >> v1:172.21.15.153:6789/0 conn(0x7f296c0b2e70 legacy=0x7f296c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296b7fe700 1 -- 172.21.15.153:0/3657355285 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0b3310 con 0x7f296c0cd8c0 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296a7fc700 1 -- 172.21.15.153:0/3657355285 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105923660 0 0) 0x7f296c13ff00 con 0x7f296c0b2e70 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296b7fe700 1 --2- 172.21.15.153:0/3657355285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cd8c0 0x7f296c140950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f295c009820 tx=0x7f295c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d9809f69b3e457 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296a7fc700 1 -- 172.21.15.153:0/3657355285 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c005e30 con 0x7f296c0cd8c0 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296a7fc700 1 -- 172.21.15.153:0/3657355285 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c006030 con 0x7f296c0cd8c0 2023-11-28T13:02:01.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.953+0000 7f296a7fc700 1 -- 172.21.15.153:0/3657355285 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c0063c0 con 0x7f296c0cd8c0 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 -- 172.21.15.153:0/3657355285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cd8c0 msgr2=0x7f296c140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 --2- 172.21.15.153:0/3657355285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cd8c0 0x7f296c140950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f295c009820 tx=0x7f295c009b30 comp rx=0 tx=0).stop 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 -- 172.21.15.153:0/3657355285 shutdown_connections 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 --2- 172.21.15.153:0/3657355285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cd8c0 0x7f296c140950 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 --2- 172.21.15.153:0/3657355285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cdc90 0x7f296c0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 -- 172.21.15.153:0/3657355285 >> 172.21.15.153:0/3657355285 conn(0x7f296c06aed0 msgr2=0x7f296c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 -- 172.21.15.153:0/3657355285 shutdown_connections 2023-11-28T13:02:01.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.954+0000 7f2972242700 1 -- 172.21.15.153:0/3657355285 wait complete. 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.955+0000 7f2972242700 1 Processor -- start 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.955+0000 7f2972242700 1 -- start start 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f2972242700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0b2e70 0x7f296c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f2972242700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c0cd8c0 0x7f296c143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f2972242700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cdc90 0x7f296c0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f2972242700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0b27b0 con 0x7f296c0cdc90 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f2972242700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0c8020 con 0x7f296c0cd8c0 2023-11-28T13:02:01.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f2972242700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0c8300 con 0x7f296c0b2e70 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cdc90 0x7f296c0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cdc90 0x7f296c0b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41270/0 (socket says 172.21.15.153:41270) 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 -- 172.21.15.153:0/3037502708 learned_addr learned my addr 172.21.15.153:0/3037502708 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296affd700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c0cd8c0 0x7f296c143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296b7fe700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0b2e70 0x7f296c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 -- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0b2e70 msgr2=0x7f296c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0b2e70 0x7f296c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 -- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c0cd8c0 msgr2=0x7f296c143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c0cd8c0 0x7f296c143ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296bfff700 1 -- 172.21.15.153:0/3037502708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c009430 con 0x7f296c0cdc90 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.956+0000 7f296affd700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c0cd8c0 0x7f296c143ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.957+0000 7f296bfff700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cdc90 0x7f296c0b1f80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f296400cbe0 tx=0x7f296400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.957+0000 7f2953fff700 1 -- 172.21.15.153:0/3037502708 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296400ab50 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.957+0000 7f2953fff700 1 -- 172.21.15.153:0/3037502708 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296400acd0 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.957+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c140150 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.957+0000 7f2953fff700 1 -- 172.21.15.153:0/3037502708 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964007310 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.957+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c0b35f0 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.958+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c0ce100 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.958+0000 7f2953fff700 1 -- 172.21.15.153:0/3037502708 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f296401a030 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.958+0000 7f2953fff700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29580682f0 0x7f295806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.959+0000 7f2953fff700 1 -- 172.21.15.153:0/3037502708 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2964085920 con 0x7f296c0cdc90 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.959+0000 7f296b7fe700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29580682f0 0x7f295806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:01.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.959+0000 7f296b7fe700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29580682f0 0x7f295806a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f295c009b70 tx=0x7f295c008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:01.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:01.963+0000 7f2953fff700 1 -- 172.21.15.153:0/3037502708 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2964052180 con 0x7f296c0cdc90 2023-11-28T13:02:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.185+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f296c068690 con 0x7f296c0cdc90 2023-11-28T13:02:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.186+0000 7f2953fff700 1 -- 172.21.15.153:0/3037502708 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f2964018090 con 0x7f296c0cdc90 2023-11-28T13:02:02.186 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:02.187 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:02.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.187+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29580682f0 msgr2=0x7f295806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.187+0000 7f2972242700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29580682f0 0x7f295806a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f295c009b70 tx=0x7f295c008070 comp rx=0 tx=0).stop 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.187+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cdc90 msgr2=0x7f296c0b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.187+0000 7f2972242700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cdc90 0x7f296c0b1f80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f296400cbe0 tx=0x7f296400cef0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 shutdown_connections 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f29580682f0 0x7f295806a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f296c0cdc90 0x7f296c0b1f80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0b2e70 0x7f296c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 --2- 172.21.15.153:0/3037502708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c0cd8c0 0x7f296c143ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 >> 172.21.15.153:0/3037502708 conn(0x7f296c06aed0 msgr2=0x7f296c13db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 shutdown_connections 2023-11-28T13:02:02.189 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.188+0000 7f2972242700 1 -- 172.21.15.153:0/3037502708 wait complete. 2023-11-28T13:02:02.190 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:02.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:01:56.931710+0000', 'modified': '2023-11-28T13:01:59.942702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5547': {'gid': 5547, 'name': 'b', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/3478129537', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 3478129537}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 3478129537}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:02:02.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:02:02.603 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T13:02:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f216d7700 1 Processor -- start 2023-11-28T13:02:02.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f216d7700 1 -- start start 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f216d7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4f1c159860 0x7f4f1c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f216d7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f1c0b0d80 0x7f4f1c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f216d7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0ccc00 con 0x7f4f1c159490 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f216d7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0ccec0 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f216d7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0cd1a0 con 0x7f4f1c159860 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f1affd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4f1c159490 0x7f4f1c0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59438/0 (socket says 172.21.15.153:59438) 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f1b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f1c0b0d80 0x7f4f1c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.727+0000 7f4f1affd700 1 -- 172.21.15.153:0/3743321276 learned_addr learned my addr 172.21.15.153:0/3743321276 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/3743321276 >> v2:172.21.15.177:3300/0 conn(0x7f4f1c159860 0x7f4f1c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3743321276 >> v2:172.21.15.177:3300/0 conn(0x7f4f1c159860 msgr2=0x7f4f1c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3743321276 >> v2:172.21.15.177:3300/0 conn(0x7f4f1c159860 0x7f4f1c0c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3743321276 >> v1:172.21.15.153:6789/0 conn(0x7f4f1c159490 legacy=0x7f4f1c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3743321276 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0cd480 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f19ffb700 1 -- 172.21.15.153:0/3743321276 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246873481 0 0) 0x7f4f1c0ccc00 con 0x7f4f1c159490 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3743321276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9a00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0cb5d0 tx=0x7f4f1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272e12b366e3ede0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f19ffb700 1 -- 172.21.15.153:0/3743321276 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400e070 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f19ffb700 1 -- 172.21.15.153:0/3743321276 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1400bbd0 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.728+0000 7f4f19ffb700 1 -- 172.21.15.153:0/3743321276 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14012430 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 -- 172.21.15.153:0/3743321276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 msgr2=0x7f4f1c0c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 --2- 172.21.15.153:0/3743321276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9a00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0cb5d0 tx=0x7f4f1400b330 comp rx=0 tx=0).stop 2023-11-28T13:02:02.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 -- 172.21.15.153:0/3743321276 shutdown_connections 2023-11-28T13:02:02.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 --2- 172.21.15.153:0/3743321276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9a00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 --2- 172.21.15.153:0/3743321276 >> v2:172.21.15.177:3300/0 conn(0x7f4f1c159860 0x7f4f1c0c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 -- 172.21.15.153:0/3743321276 >> 172.21.15.153:0/3743321276 conn(0x7f4f1c0cb670 msgr2=0x7f4f1c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 -- 172.21.15.153:0/3743321276 shutdown_connections 2023-11-28T13:02:02.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.729+0000 7f4f216d7700 1 -- 172.21.15.153:0/3743321276 wait complete. 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.730+0000 7f4f216d7700 1 Processor -- start 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.730+0000 7f4f216d7700 1 -- start start 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f216d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f216d7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c159490 0x7f4f1c0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f216d7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0c3c90 0x7f4f1c15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f216d7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0c4d40 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f216d7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0c4530 con 0x7f4f1c159490 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f216d7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c159e70 con 0x7f4f1c0c3c90 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f1affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f1affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41280/0 (socket says 172.21.15.153:41280) 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.731+0000 7f4f1affd700 1 -- 172.21.15.153:0/1593041489 learned_addr learned my addr 172.21.15.153:0/1593041489 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f1affd700 1 -- 172.21.15.153:0/1593041489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0c3c90 msgr2=0x7f4f1c15bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f1affd700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0c3c90 0x7f4f1c15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f1affd700 1 -- 172.21.15.153:0/1593041489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c159490 msgr2=0x7f4f1c0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f1affd700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c159490 0x7f4f1c0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f1affd700 1 -- 172.21.15.153:0/1593041489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f14009a70 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f1affd700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9950 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4f100011e0 tx=0x7f4f1000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f037fe700 1 -- 172.21.15.153:0/1593041489 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10004c80 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f037fe700 1 -- 172.21.15.153:0/1593041489 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f10004e00 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f037fe700 1 -- 172.21.15.153:0/1593041489 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10007310 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c0c9f40 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.732+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c0cd760 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.734+0000 7f4f037fe700 1 -- 172.21.15.153:0/1593041489 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f1001a020 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.734+0000 7f4f037fe700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f080682f0 0x7f4f0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.734+0000 7f4f037fe700 1 -- 172.21.15.153:0/1593041489 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4f100859d0 con 0x7f4f1c0b0d80 2023-11-28T13:02:02.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.734+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f080682f0 0x7f4f0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.735+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f080682f0 0x7f4f0806a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c15cf70 tx=0x7f4f0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.736+0000 7f4f2d860b80 1 Processor -- start 2023-11-28T13:02:02.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.736+0000 7f4f2d860b80 1 -- start start 2023-11-28T13:02:02.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c14be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c15120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c0e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8ad4bc0 con 0x55b0d8c14810 2023-11-28T13:02:02.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8b43350 con 0x55b0d8be91c0 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8b3c880 con 0x55b0d8c0dce0 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f1affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c14be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f1affd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c14be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41288/0 (socket says 172.21.15.153:41288) 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f1affd700 1 -- 172.21.15.153:0/1996918648 learned_addr learned my addr 172.21.15.153:0/1996918648 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c15120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.737+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c0e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f1affd700 1 -- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 msgr2=0x55b0d8c0e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f1affd700 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c0e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f1affd700 1 -- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 msgr2=0x55b0d8c15120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f1affd700 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c15120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f1affd700 1 -- 172.21.15.153:0/1996918648 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0d89be5e0 con 0x55b0d8c14810 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:02.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f1affd700 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c14be0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4f10083fa0 tx=0x7f4f10055890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b62f98d507d6dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f00ff9700 1 -- 172.21.15.153:0/1996918648 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f100552f0 con 0x55b0d8c14810 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f00ff9700 1 -- 172.21.15.153:0/1996918648 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1006d070 con 0x55b0d8c14810 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f00ff9700 1 -- 172.21.15.153:0/1996918648 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1005da00 con 0x55b0d8c14810 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f2d860b80 1 -- 172.21.15.153:0/1996918648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 msgr2=0x55b0d8c14be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.738+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c14be0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4f10083fa0 tx=0x7f4f10055890 comp rx=0 tx=0).stop 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 -- 172.21.15.153:0/1996918648 shutdown_connections 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c14be0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c0e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/1996918648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 -- 172.21.15.153:0/1996918648 >> 172.21.15.153:0/1996918648 conn(0x55b0d8be7d60 msgr2=0x55b0d8be86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 -- 172.21.15.153:0/1996918648 shutdown_connections 2023-11-28T13:02:02.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 -- 172.21.15.153:0/1996918648 wait complete. 2023-11-28T13:02:02.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 -- start start 2023-11-28T13:02:02.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.739+0000 7f4f2d860b80 1 Processor -- start 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8be91c0 0x55b0d8c3b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8c0dce0 0x55b0d8c05920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d89c08d0 con 0x55b0d8c14810 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c037d0 con 0x55b0d8c0dce0 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c03c30 con 0x55b0d8be91c0 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 10 client.0 _ll_register_callbacks cb 0x55b0d89e2240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c31eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41298/0 (socket says 172.21.15.153:41298) 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3953164428 learned_addr learned my addr 172.21.15.153:0/3953164428 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:02.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8c0dce0 0x55b0d8c05920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1affd700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8be91c0 0x55b0d8c3b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8be91c0 msgr2=0x55b0d8c3b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8be91c0 0x55b0d8c3b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8c0dce0 msgr2=0x55b0d8c05920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8c0dce0 0x55b0d8c05920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3953164428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c0cd760 con 0x55b0d8c14810 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1affd700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8be91c0 0x55b0d8c3b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c31eb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c067360 tx=0x7f4f14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebb7f35b44e8a82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4ef9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3953164428 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14012430 con 0x55b0d8c14810 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3953164428 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4f14003e20 con 0x55b0d8c14810 2023-11-28T13:02:02.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.741+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0d8b0a140 con 0x55b0d8c14810 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3953164428 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f140098f0 con 0x55b0d8c14810 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 10 client.5688 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55b0d896ad90 con 0x55b0d8c14810 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3953164428 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0d8b0a140 con 0x55b0d8c14810 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3953164428 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4f14012c80 con 0x55b0d8c14810 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 10 client.5688 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 10 client.5688 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 20 client.5688 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8be9ed0 0x55b0d8c08e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 4 client.5688 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0d89c0f10 con 0x55b0d8be9ed0 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f1affd700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8be9ed0 0x55b0d8c08e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4ef9ffb700 10 client.5688 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.742+0000 7f4f1affd700 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8be9ed0 0x55b0d8c08e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5771a51a6e8bb82 server_cookie=261d23056d589a7c in_seq=0 out_seq=0 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3953164428 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55b0d89c0f10 con 0x55b0d8be9ed0 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4ef9ffb700 10 client.5688 handle_command_reply: tid=0 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 client.5688 shutdown 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 msgr2=0x55b0d8c31eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c31eb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c067360 tx=0x7f4f14004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 shutdown_connections 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c31eb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8be9ed0 0x55b0d8c08e30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8be91c0 0x55b0d8c3b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4ef9ffb700 0 client.5688 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:02.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3953164428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8c0dce0 0x55b0d8c05920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 >> 172.21.15.153:0/3953164428 conn(0x55b0d8be7d60 msgr2=0x55b0d8c14be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 shutdown_connections 2023-11-28T13:02:02.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.743+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3953164428 wait complete. 2023-11-28T13:02:02.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.744+0000 7f4f2d860b80 20 client.5688 trim_cache size 0 max 16384 2023-11-28T13:02:02.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 Processor -- start 2023-11-28T13:02:02.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 -- start start 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c3bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c0dce0 0x55b0d8be8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c14810 0x55b0d8be8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c3c350 con 0x55b0d8c0dce0 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c3c170 con 0x55b0d8be91c0 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8beac90 con 0x55b0d8c14810 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.761+0000 7f4f1a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c0dce0 0x55b0d8be8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c0dce0 0x55b0d8be8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41304/0 (socket says 172.21.15.153:41304) 2023-11-28T13:02:02.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 -- 172.21.15.153:0/567025144 learned_addr learned my addr 172.21.15.153:0/567025144 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1affd700 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c3bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c14810 0x55b0d8be8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 -- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c14810 msgr2=0x55b0d8be8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c14810 0x55b0d8be8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 -- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 msgr2=0x55b0d8c3bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c3bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 -- 172.21.15.153:0/567025144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0d8b03140 con 0x55b0d8c0dce0 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c0dce0 0x55b0d8be8630 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c158c60 tx=0x7f4f0c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e8f8dae4b26184 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f017fa700 1 -- 172.21.15.153:0/567025144 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c022520 con 0x55b0d8c0dce0 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f017fa700 1 -- 172.21.15.153:0/567025144 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c022680 con 0x55b0d8c0dce0 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.762+0000 7f4f017fa700 1 -- 172.21.15.153:0/567025144 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0227f0 con 0x55b0d8c0dce0 2023-11-28T13:02:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 -- 172.21.15.153:0/567025144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c0dce0 msgr2=0x55b0d8be8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c0dce0 0x55b0d8be8630 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c158c60 tx=0x7f4f0c013000 comp rx=0 tx=0).stop 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 -- 172.21.15.153:0/567025144 shutdown_connections 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c14810 0x55b0d8be8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c3bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/567025144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c0dce0 0x55b0d8be8630 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 -- 172.21.15.153:0/567025144 >> 172.21.15.153:0/567025144 conn(0x55b0d8be7d60 msgr2=0x55b0d8c32b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 -- 172.21.15.153:0/567025144 shutdown_connections 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 -- 172.21.15.153:0/567025144 wait complete. 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 -- start start 2023-11-28T13:02:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.763+0000 7f4f2d860b80 1 Processor -- start 2023-11-28T13:02:02.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c02e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c37d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f2d860b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c38770 con 0x55b0d8c14810 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c0cfa0 con 0x55b0d8be91c0 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f2d860b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0d8c26200 con 0x55b0d8c0dce0 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f2d860b80 10 client.0 _ll_register_callbacks cb 0x55b0d89e2240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c30050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41312/0 (socket says 172.21.15.153:41312) 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3687685349 learned_addr learned my addr 172.21.15.153:0/3687685349 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1affd700 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c02e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1a7fc700 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c37d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 msgr2=0x55b0d8c37d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c37d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 msgr2=0x55b0d8c02e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c02e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.764+0000 7f4f1b7fe700 1 -- 172.21.15.153:0/3687685349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0c01c040 con 0x55b0d8c14810 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f1b7fe700 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c30050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55b0d8beaa80 tx=0x7f4f14012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782daf991e11d0be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4ef9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3687685349 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140042c0 con 0x55b0d8c14810 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3687685349 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4f14004420 con 0x55b0d8c14810 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0d8afa160 con 0x55b0d8c14810 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3687685349 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f140045e0 con 0x55b0d8c14810 2023-11-28T13:02:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 10 client.5694 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55b0d8991200 con 0x55b0d8c14810 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3687685349 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0d8afa160 con 0x55b0d8c14810 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3687685349 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4f14004740 con 0x55b0d8c14810 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 10 client.5694 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 10 client.5694 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 20 client.5694 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8c30590 0x55b0d8c98c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 4 client.5694 mds_command: new command op to 5547 tid=0[{"prefix": "status"}] 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b0d8ae4170 con 0x55b0d8c30590 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.765+0000 7f4f1affd700 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8c30590 0x55b0d8c98c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4ef9ffb700 10 client.5694 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:02.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f1affd700 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8c30590 0x55b0d8c98c20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887e9e48bf9b3202 server_cookie=f8ad19dff64aebf7 in_seq=0 out_seq=0 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4ef9ffb700 1 -- 172.21.15.153:0/3687685349 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b0d8ae4170 con 0x55b0d8c30590 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4ef9ffb700 10 client.5694 handle_command_reply: tid=0 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 client.5694 shutdown 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 msgr2=0x55b0d8c30050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c30050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55b0d8beaa80 tx=0x7f4f14012000 comp rx=0 tx=0).stop 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 shutdown_connections 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55b0d8c14810 0x55b0d8c30050 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55b0d8c30590 0x55b0d8c98c20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b0d8be91c0 0x55b0d8c02e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 --2- 172.21.15.153:0/3687685349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b0d8c0dce0 0x55b0d8c37d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4ef9ffb700 0 client.5694 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 >> 172.21.15.153:0/3687685349 conn(0x55b0d8be7d60 msgr2=0x55b0d8be8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 shutdown_connections 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 1 -- 172.21.15.153:0/3687685349 wait complete. 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.766+0000 7f4f2d860b80 20 client.5694 trim_cache size 0 max 16384 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stdout: "cluster_fsid": "47bfaa00-a151-4537-aa4c-32b0866eb248", 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stdout: "whoami": 0, 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 5547, 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stdout: "want_state": "up:active", 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stdout: "state": "up:active", 2023-11-28T13:02:02.774 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs", 2023-11-28T13:02:02.775 INFO:teuthology.orchestra.run.smithi153.stdout: "rank_uptime": 5.8333486389999996, 2023-11-28T13:02:02.775 INFO:teuthology.orchestra.run.smithi153.stdout: "mdsmap_epoch": 23, 2023-11-28T13:02:02.775 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch": 51, 2023-11-28T13:02:02.775 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch_barrier": 51, 2023-11-28T13:02:02.775 INFO:teuthology.orchestra.run.smithi153.stdout: "uptime": 11.172203122999999 2023-11-28T13:02:02.775 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f080682f0 msgr2=0x7f4f0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f080682f0 0x7f4f0806a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c15cf70 tx=0x7f4f0c008040 comp rx=0 tx=0).stop 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 msgr2=0x7f4f1c0c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9950 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4f100011e0 tx=0x7f4f1000cae0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 shutdown_connections 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f080682f0 0x7f4f0806a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f1c0b0d80 0x7f4f1c0c9950 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f1c0c3c90 0x7f4f1c15bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 --2- 172.21.15.153:0/1593041489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f1c159490 0x7f4f1c0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 >> 172.21.15.153:0/1593041489 conn(0x7f4f1c0cb670 msgr2=0x7f4f1c14fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 shutdown_connections 2023-11-28T13:02:02.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.767+0000 7f4f216d7700 1 -- 172.21.15.153:0/1593041489 wait complete. 2023-11-28T13:02:02.780 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:02:02.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc9867d2700 1 Processor -- start 2023-11-28T13:02:02.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc9867d2700 1 -- start start 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc9867d2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc98006c990 0x7fc98006ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc9867d2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9800ca240 0x7fc9800ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc9867d2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98006ae60 con 0x7fc98006e9f0 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc9867d2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98006d350 con 0x7fc98006c990 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc9867d2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800cabc0 con 0x7fc9800ca240 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc97f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc98006c990 0x7fc98006ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.900+0000 7fc97ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc98006e9f0 0x7fc98006b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59480/0 (socket says 172.21.15.153:59480) 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97ffff700 1 -- 172.21.15.153:0/139652598 learned_addr learned my addr 172.21.15.153:0/139652598 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc984d6f700 1 --2- 172.21.15.153:0/139652598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9800ca240 0x7fc9800ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97effd700 1 -- 172.21.15.153:0/139652598 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277823227 0 0) 0x7fc98006ae60 con 0x7fc98006e9f0 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97effd700 1 -- 172.21.15.153:0/139652598 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9680035b0 con 0x7fc98006e9f0 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97f7fe700 1 -- 172.21.15.153:0/139652598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9800ca240 msgr2=0x7fc9800ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97f7fe700 1 --2- 172.21.15.153:0/139652598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9800ca240 0x7fc9800ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97f7fe700 1 -- 172.21.15.153:0/139652598 >> v1:172.21.15.153:6789/0 conn(0x7fc98006e9f0 legacy=0x7fc98006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97f7fe700 1 -- 172.21.15.153:0/139652598 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98006a270 con 0x7fc98006c990 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97f7fe700 1 --2- 172.21.15.153:0/139652598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc98006c990 0x7fc98006ce10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc974009840 tx=0x7fc974009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a8b2450a00343c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97effd700 1 -- 172.21.15.153:0/139652598 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974011830 con 0x7fc98006c990 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97effd700 1 -- 172.21.15.153:0/139652598 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc974011990 con 0x7fc98006c990 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.901+0000 7fc97effd700 1 -- 172.21.15.153:0/139652598 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974011d20 con 0x7fc98006c990 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 -- 172.21.15.153:0/139652598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc98006c990 msgr2=0x7fc98006ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 --2- 172.21.15.153:0/139652598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc98006c990 0x7fc98006ce10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc974009840 tx=0x7fc974009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:02.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 -- 172.21.15.153:0/139652598 shutdown_connections 2023-11-28T13:02:02.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 --2- 172.21.15.153:0/139652598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9800ca240 0x7fc9800ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 --2- 172.21.15.153:0/139652598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc98006c990 0x7fc98006ce10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 -- 172.21.15.153:0/139652598 >> 172.21.15.153:0/139652598 conn(0x7fc9800b12b0 msgr2=0x7fc9800b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:02.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 -- 172.21.15.153:0/139652598 shutdown_connections 2023-11-28T13:02:02.905 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.902+0000 7fc9867d2700 1 -- 172.21.15.153:0/139652598 wait complete. 2023-11-28T13:02:02.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.903+0000 7fc9867d2700 1 Processor -- start 2023-11-28T13:02:02.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.903+0000 7fc9867d2700 1 -- start start 2023-11-28T13:02:02.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc9867d2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98006c990 0x7fc9801f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc9867d2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc98006e9f0 0x7fc98014fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc9867d2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9800ca240 0x7fc9801f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc9867d2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801f3a90 con 0x7fc9800ca240 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc9867d2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801f34d0 con 0x7fc98006c990 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc9867d2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801f37b0 con 0x7fc98006e9f0 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9800ca240 0x7fc9801f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9800ca240 0x7fc9801f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41326/0 (socket says 172.21.15.153:41326) 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 -- 172.21.15.153:0/3331945717 learned_addr learned my addr 172.21.15.153:0/3331945717 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc97ffff700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98006c990 0x7fc9801f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc97f7fe700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc98006e9f0 0x7fc98014fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 -- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc98006e9f0 msgr2=0x7fc98014fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc98006e9f0 0x7fc98014fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 -- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98006c990 msgr2=0x7fc9801f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98006c990 0x7fc9801f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 -- 172.21.15.153:0/3331945717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740094a0 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc97f7fe700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc98006e9f0 0x7fc98014fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.904+0000 7fc984d6f700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9800ca240 0x7fc9801f2aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc97800bde0 tx=0x7fc97800e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.905+0000 7fc97cff9700 1 -- 172.21.15.153:0/3331945717 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978010040 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.905+0000 7fc97cff9700 1 -- 172.21.15.153:0/3331945717 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978003a20 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.905+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98006a810 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.905+0000 7fc97cff9700 1 -- 172.21.15.153:0/3331945717 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780092b0 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.905+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98006a530 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.905+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98006aaf0 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.906+0000 7fc97cff9700 1 -- 172.21.15.153:0/3331945717 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc97801c030 con 0x7fc9800ca240 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.906+0000 7fc97cff9700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc96c0682f0 0x7fc96c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:02.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.907+0000 7fc97cff9700 1 -- 172.21.15.153:0/3331945717 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc978085cc0 con 0x7fc9800ca240 2023-11-28T13:02:02.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.907+0000 7fc97ffff700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc96c0682f0 0x7fc96c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:02.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.907+0000 7fc97ffff700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc96c0682f0 0x7fc96c06a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc970007fd0 tx=0x7fc970007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:02.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:02.911+0000 7fc97cff9700 1 -- 172.21.15.153:0/3331945717 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc978052520 con 0x7fc9800ca240 2023-11-28T13:02:03.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.136+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9800cd5b0 con 0x7fc9800ca240 2023-11-28T13:02:03.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.136+0000 7fc97cff9700 1 -- 172.21.15.153:0/3331945717 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc978055b40 con 0x7fc9800ca240 2023-11-28T13:02:03.137 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:02:03.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc96c0682f0 msgr2=0x7fc96c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc96c0682f0 0x7fc96c06a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc970007fd0 tx=0x7fc970007dc0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9800ca240 msgr2=0x7fc9801f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9800ca240 0x7fc9801f2aa0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc97800bde0 tx=0x7fc97800e3a0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 shutdown_connections 2023-11-28T13:02:03.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc96c0682f0 0x7fc96c06a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc9800ca240 0x7fc9801f2aa0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98006c990 0x7fc9801f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 --2- 172.21.15.153:0/3331945717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc98006e9f0 0x7fc98014fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 >> 172.21.15.153:0/3331945717 conn(0x7fc9800b12b0 msgr2=0x7fc980065f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:03.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 shutdown_connections 2023-11-28T13:02:03.140 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:03.138+0000 7fc9867d2700 1 -- 172.21.15.153:0/3331945717 wait complete. 2023-11-28T13:02:03.151 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T13:02:03.151 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:02:03.151 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-11-28T13:02:03.151 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:02:03.151 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:03.152 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T13:02:03.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:02:03.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:02:03.152 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:02:03.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.152 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-11-28T13:02:03.169 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:03.170 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42257sec preferred_lft 42257sec 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ad7d:b13e:f07:e265/64 scope link noprefixroute 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 4e:45:7e:fd:cf:4e brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:03.171 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::6cbb:9eff:fe64:da81/64 scope link 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether a2:10:9d:7d:8b:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::a010:9dff:fe7d:8b89/64 scope link 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 4e:45:7e:fd:cf:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::4c45:7eff:fefd:cf4e/64 scope link 2023-11-28T13:02:03.172 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:03.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.173 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:02:03.235 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-11-28T13:02:03.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.236 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-11-28T13:02:03.295 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-11-28T13:02:03.295 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:02:03.295 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:02:03.295 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:02:03.295 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:02:03.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.296 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:02:03.296 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:02:03.296 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:02:03.296 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:02:03.296 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:02:03.296 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:02:03.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.400 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:02:03.417 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:02:03.417 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:02:03.417 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:02:03.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.418 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-11-28T13:02:03.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:02:03.485 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:03.541 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:02:03.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:02:03.542 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:03.598 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:02:03.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.598 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:02:03.682 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:02:03.682 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:02:03.682 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:02:03.691 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:02:03.691 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:02:03.691 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:02:03.691 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:02:03.691 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T13:02:03.722 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=47bfaa00-a151-4537-aa4c-32b0866eb248" will pass to kernel 2023-11-28T13:02:03.725 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:02:03.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.725 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:03.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.760 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:02:03.852 INFO:teuthology.orchestra.run.smithi153.stdout:client5706 2023-11-28T13:02:03.853 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:02:03.853 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:03.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.869 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T13:02:03.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.933 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T13:02:03.992 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:02:03.992 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:02:03.992 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:02:03.992 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:02:03.992 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T13:02:03.992 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:02:04.097 INFO:teuthology.orchestra.run.smithi153.stdout:1099511627777 2023-11-28T13:02:04.286 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:04.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.447+0000 7f6e961c1700 1 Processor -- start 2023-11-28T13:02:04.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.447+0000 7f6e961c1700 1 -- start start 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.447+0000 7f6e961c1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6e90071040 0x7f6e90071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e961c1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6e9006dee0 0x7f6e9006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e961c1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e90071950 con 0x7f6e9006cdc0 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e961c1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e90161550 con 0x7f6e90071040 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e961c1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9015d060 con 0x7f6e9006dee0 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6e90071040 0x7f6e90071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6e90071040 0x7f6e90071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41334/0 (socket says 172.21.15.153:41334) 2023-11-28T13:02:04.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 -- 172.21.15.153:0/596746034 learned_addr learned my addr 172.21.15.153:0/596746034 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8effd700 1 --2- 172.21.15.153:0/596746034 >> v2:172.21.15.177:3300/0 conn(0x7f6e9006dee0 0x7f6e9006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 -- 172.21.15.153:0/596746034 >> v2:172.21.15.177:3300/0 conn(0x7f6e9006dee0 msgr2=0x7f6e9006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 --2- 172.21.15.153:0/596746034 >> v2:172.21.15.177:3300/0 conn(0x7f6e9006dee0 0x7f6e9006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 -- 172.21.15.153:0/596746034 >> v1:172.21.15.177:6789/0 conn(0x7f6e9006cdc0 legacy=0x7f6e9006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 -- 172.21.15.153:0/596746034 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9015d340 con 0x7f6e90071040 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8effd700 1 --2- 172.21.15.153:0/596746034 >> v2:172.21.15.177:3300/0 conn(0x7f6e9006dee0 0x7f6e9006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8f7fe700 1 --2- 172.21.15.153:0/596746034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e90071040 0x7f6e90071410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6e9006fb00 tx=0x7f6e80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec69db8a820bab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8e7fc700 1 -- 172.21.15.153:0/596746034 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80004230 con 0x7f6e90071040 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8e7fc700 1 -- 172.21.15.153:0/596746034 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80004390 con 0x7f6e90071040 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.448+0000 7f6e8e7fc700 1 -- 172.21.15.153:0/596746034 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e800036a0 con 0x7f6e90071040 2023-11-28T13:02:04.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 -- 172.21.15.153:0/596746034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e90071040 msgr2=0x7f6e90071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 --2- 172.21.15.153:0/596746034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e90071040 0x7f6e90071410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6e9006fb00 tx=0x7f6e80009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 -- 172.21.15.153:0/596746034 shutdown_connections 2023-11-28T13:02:04.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 --2- 172.21.15.153:0/596746034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e90071040 0x7f6e90071410 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 --2- 172.21.15.153:0/596746034 >> v2:172.21.15.177:3300/0 conn(0x7f6e9006dee0 0x7f6e9006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 -- 172.21.15.153:0/596746034 >> 172.21.15.153:0/596746034 conn(0x7f6e900d0000 msgr2=0x7f6e900d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:04.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 -- 172.21.15.153:0/596746034 shutdown_connections 2023-11-28T13:02:04.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.449+0000 7f6e961c1700 1 -- 172.21.15.153:0/596746034 wait complete. 2023-11-28T13:02:04.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.450+0000 7f6e961c1700 1 Processor -- start 2023-11-28T13:02:04.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.451+0000 7f6e961c1700 1 -- start start 2023-11-28T13:02:04.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.451+0000 7f6e961c1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9006cdc0 0x7f6e900ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.451+0000 7f6e961c1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e9006dee0 0x7f6e90154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.451+0000 7f6e961c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e90071040 0x7f6e90153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e961c1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900b3530 con 0x7f6e9006dee0 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e961c1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900acdf0 con 0x7f6e9006cdc0 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e961c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900ad0b0 con 0x7f6e90071040 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e9006dee0 0x7f6e90154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e9006dee0 0x7f6e90154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41342/0 (socket says 172.21.15.153:41342) 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 -- 172.21.15.153:0/3818156374 learned_addr learned my addr 172.21.15.153:0/3818156374 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8f7fe700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9006cdc0 0x7f6e900ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8ffff700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e90071040 0x7f6e90153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 -- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e90071040 msgr2=0x7f6e90153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e90071040 0x7f6e90153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 -- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9006cdc0 msgr2=0x7f6e900ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9006cdc0 0x7f6e900ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 -- 172.21.15.153:0/3818156374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e80009550 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e8effd700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e9006dee0 0x7f6e90154700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c7f0 tx=0x7f6e8400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e77fff700 1 -- 172.21.15.153:0/3818156374 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84011800 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e77fff700 1 -- 172.21.15.153:0/3818156374 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e84011960 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9015d900 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e77fff700 1 -- 172.21.15.153:0/3818156374 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84011cf0 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.452+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9015d620 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.453+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9006e870 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.454+0000 7f6e77fff700 1 -- 172.21.15.153:0/3818156374 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e84021070 con 0x7f6e9006dee0 2023-11-28T13:02:04.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.454+0000 7f6e77fff700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6e7c0682f0 0x7f6e7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.454+0000 7f6e77fff700 1 -- 172.21.15.153:0/3818156374 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e84085990 con 0x7f6e9006dee0 2023-11-28T13:02:04.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.454+0000 7f6e8f7fe700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6e7c0682f0 0x7f6e7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.455+0000 7f6e8f7fe700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6e7c0682f0 0x7f6e7c06a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009130 tx=0x7f6e80008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:04.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.458+0000 7f6e77fff700 1 -- 172.21.15.153:0/3818156374 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e840521f0 con 0x7f6e9006dee0 2023-11-28T13:02:04.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.686+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e901e6430 con 0x7f6e9006dee0 2023-11-28T13:02:04.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.686+0000 7f6e77fff700 1 -- 172.21.15.153:0/3818156374 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f6e84055810 con 0x7f6e9006dee0 2023-11-28T13:02:04.687 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:04.687 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:04.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6e7c0682f0 msgr2=0x7f6e7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6e7c0682f0 0x7f6e7c06a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009130 tx=0x7f6e80008030 comp rx=0 tx=0).stop 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e9006dee0 msgr2=0x7f6e90154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e9006dee0 0x7f6e90154700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c7f0 tx=0x7f6e8400cb00 comp rx=0 tx=0).stop 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 shutdown_connections 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e90071040 0x7f6e90153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9006cdc0 0x7f6e900ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6e7c0682f0 0x7f6e7c06a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 --2- 172.21.15.153:0/3818156374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6e9006dee0 0x7f6e90154700 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 >> 172.21.15.153:0/3818156374 conn(0x7f6e900d0000 msgr2=0x7f6e9006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.688+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 shutdown_connections 2023-11-28T13:02:04.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.689+0000 7f6e961c1700 1 -- 172.21.15.153:0/3818156374 wait complete. 2023-11-28T13:02:04.691 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:04.702 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-11-28T13:02:04.859 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2023-11-28T13:02:04.860 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:02:04.860 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:02:04.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.903+0000 7f5867a6e080 1 Processor -- start 2023-11-28T13:02:04.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.903+0000 7f5867a6e080 1 -- start start 2023-11-28T13:02:04.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f5867a6e080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71ddd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f5867a6e080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 0x557b71f1a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f5867a6e080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71f1aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f5867a6e080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b71d58000 con 0x557b71e8cde0 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f5867a6e080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b71ceafe0 con 0x557b71db27e0 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f5867a6e080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b71de05a0 con 0x557b71e88450 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71f1aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71f1aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41358/0 (socket says 172.21.15.153:41358) 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 -- 172.21.15.153:0/1186029736 learned_addr learned my addr 172.21.15.153:0/1186029736 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585db71700 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71ddd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585d370700 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 0x557b71f1a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 -- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 msgr2=0x557b71f1a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 0x557b71f1a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 -- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 msgr2=0x557b71ddd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71ddd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 -- 172.21.15.153:0/1186029736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585800a1c0 con 0x557b71e8cde0 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f585e372700 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71f1aa50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5858013040 tx=0x7f585800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a761ee017f331c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f584ffff700 1 -- 172.21.15.153:0/1186029736 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585800e9e0 con 0x557b71e8cde0 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f584ffff700 1 -- 172.21.15.153:0/1186029736 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585800eb40 con 0x557b71e8cde0 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.904+0000 7f584ffff700 1 -- 172.21.15.153:0/1186029736 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585800eeb0 con 0x557b71e8cde0 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 -- 172.21.15.153:0/1186029736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 msgr2=0x557b71f1aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71f1aa50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5858013040 tx=0x7f585800e000 comp rx=0 tx=0).stop 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 -- 172.21.15.153:0/1186029736 shutdown_connections 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71f1aa50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71ddd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 --2- 172.21.15.153:0/1186029736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 0x557b71f1a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 -- 172.21.15.153:0/1186029736 >> 172.21.15.153:0/1186029736 conn(0x557b71ddcb60 msgr2=0x557b71df13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 -- 172.21.15.153:0/1186029736 shutdown_connections 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 -- 172.21.15.153:0/1186029736 wait complete. 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.905+0000 7f5867a6e080 1 Processor -- start 2023-11-28T13:02:04.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f5867a6e080 1 -- start start 2023-11-28T13:02:04.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f5867a6e080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71e7e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f5867a6e080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 0x557b71e86b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f5867a6e080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71e87040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f5867a6e080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b71dddaa0 con 0x557b71e8cde0 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f5867a6e080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b71e84010 con 0x557b71db27e0 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f5867a6e080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b71e847d0 con 0x557b71e88450 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f585e372700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71e87040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f585e372700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71e87040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41364/0 (socket says 172.21.15.153:41364) 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f585e372700 1 -- 172.21.15.153:0/3299295597 learned_addr learned my addr 172.21.15.153:0/3299295597 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f585db71700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71e7e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.906+0000 7f585d370700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 0x557b71e86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f585e372700 1 -- 172.21.15.153:0/3299295597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 msgr2=0x557b71e86b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f585e372700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557b71e88450 0x557b71e86b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f585e372700 1 -- 172.21.15.153:0/3299295597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 msgr2=0x557b71e7e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f585e372700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71e7e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f585e372700 1 -- 172.21.15.153:0/3299295597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b71d03120 con 0x557b71e8cde0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f585db71700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b71db27e0 0x557b71e7e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f585e372700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71e87040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f585800a190 tx=0x7f5858003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f584effd700 1 -- 172.21.15.153:0/3299295597 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858024660 con 0x557b71e8cde0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f584effd700 1 -- 172.21.15.153:0/3299295597 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5858009030 con 0x557b71e8cde0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557b71cf6830 con 0x557b71e8cde0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f584effd700 1 -- 172.21.15.153:0/3299295597 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58580249c0 con 0x557b71e8cde0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.907+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b71d03360 con 0x557b71e8cde0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.908+0000 7f584effd700 1 -- 172.21.15.153:0/3299295597 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5858035030 con 0x557b71e8cde0 2023-11-28T13:02:04.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f584effd700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5840066c10 0x7f58400690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f584effd700 1 -- 172.21.15.153:0/3299295597 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f585802b070 con 0x557b71e8cde0 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f585db71700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5840066c10 0x7f58400690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f5867a6e080 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x557b71db21f0 0x557b71f13030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 --> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v8 -- 0x557b71f13570 con 0x557b71db21f0 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f585d370700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x557b71db21f0 0x557b71f13030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f585db71700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5840066c10 0x7f58400690c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x557b71deead0 tx=0x7f58540027a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.909+0000 7f585d370700 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x557b71db21f0 0x557b71f13030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f585d370700 1 -- 172.21.15.153:0/3299295597 <== osd.1 v2:172.21.15.153:6816/103564 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f585000b040 con 0x557b71db21f0 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x557b71db21f0 msgr2=0x557b71f13030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x557b71db21f0 0x557b71f13030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5840066c10 msgr2=0x7f58400690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5840066c10 0x7f58400690c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x557b71deead0 tx=0x7f58540027a0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 msgr2=0x557b71e87040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 --2- 172.21.15.153:0/3299295597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x557b71e8cde0 0x557b71e87040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f585800a190 tx=0x7f5858003330 comp rx=0 tx=0).stop 2023-11-28T13:02:04.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f585e372700 1 -- 172.21.15.153:0/3299295597 reap_dead start 2023-11-28T13:02:04.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 shutdown_connections 2023-11-28T13:02:04.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 >> 172.21.15.153:0/3299295597 conn(0x557b71ddcb60 msgr2=0x557b71df13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:04.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 shutdown_connections 2023-11-28T13:02:04.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:04.910+0000 7f5867a6e080 1 -- 172.21.15.153:0/3299295597 wait complete. 2023-11-28T13:02:04.918 INFO:teuthology.orchestra.run.smithi153.stdout:Yalphaparent_a 2023-11-28T13:02:04.918 DEBUG:teuthology.orchestra.run.smithi153:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:02:05.042 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:02:05.042 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 1099511627777, 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: "ancestors": [ 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: "dirino": 1099511627776, 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: "dname": "alpha", 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: "version": 2 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: "dirino": 1, 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: "dname": "parent_a", 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: "version": 4 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:05.043 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2023-11-28T13:02:05.044 INFO:teuthology.orchestra.run.smithi153.stdout: "pool": 8, 2023-11-28T13:02:05.044 INFO:teuthology.orchestra.run.smithi153.stdout: "old_pools": [] 2023-11-28T13:02:05.044 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:02:05.044 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:05.045 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T13:02:05.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.089+0000 7f915bef4080 1 Processor -- start 2023-11-28T13:02:05.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.089+0000 7f915bef4080 1 -- start start 2023-11-28T13:02:05.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.089+0000 7f915bef4080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5630269eb930 0x5630269e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.089+0000 7f915bef4080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.089+0000 7f915bef4080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563026a7b630 0x563026a77970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f915bef4080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630268ddfe0 con 0x563026a7b260 2023-11-28T13:02:05.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f915bef4080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563026a78700 con 0x5630269eb930 2023-11-28T13:02:05.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f915bef4080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630269ec260 con 0x563026a7b630 2023-11-28T13:02:05.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563026a7b630 0x563026a77970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91517f6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f9151ff7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5630269eb930 0x5630269e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563026a7b630 0x563026a77970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:58990/0 (socket says 172.21.15.153:58990) 2023-11-28T13:02:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 -- 172.21.15.153:0/2718333851 learned_addr learned my addr 172.21.15.153:0/2718333851 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 -- 172.21.15.153:0/2718333851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5630269eb930 msgr2=0x5630269e99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 --2- 172.21.15.153:0/2718333851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5630269eb930 0x5630269e99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 -- 172.21.15.153:0/2718333851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 msgr2=0x5630269e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 --2- 172.21.15.153:0/2718333851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 -- 172.21.15.153:0/2718333851 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148005d20 con 0x563026a7b630 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f91527f8700 1 --2- 172.21.15.153:0/2718333851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563026a7b630 0x563026a77970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f914c013040 tx=0x7f914c00e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cacade2aee5799ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f9143fff700 1 -- 172.21.15.153:0/2718333851 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c005790 con 0x563026a7b630 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f9143fff700 1 -- 172.21.15.153:0/2718333851 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c00e9e0 con 0x563026a7b630 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.090+0000 7f9143fff700 1 -- 172.21.15.153:0/2718333851 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00ed70 con 0x563026a7b630 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 -- 172.21.15.153:0/2718333851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563026a7b630 msgr2=0x563026a77970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 --2- 172.21.15.153:0/2718333851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563026a7b630 0x563026a77970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f914c013040 tx=0x7f914c00e000 comp rx=0 tx=0).stop 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 -- 172.21.15.153:0/2718333851 shutdown_connections 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 --2- 172.21.15.153:0/2718333851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563026a7b630 0x563026a77970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 --2- 172.21.15.153:0/2718333851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5630269eb930 0x5630269e99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 --2- 172.21.15.153:0/2718333851 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 -- 172.21.15.153:0/2718333851 >> 172.21.15.153:0/2718333851 conn(0x5630269ef9e0 msgr2=0x563026a7ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 -- 172.21.15.153:0/2718333851 shutdown_connections 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 -- 172.21.15.153:0/2718333851 wait complete. 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 Processor -- start 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.091+0000 7f915bef4080 1 -- start start 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f915bef4080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5630269eb930 0x563026a6b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f915bef4080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f915bef4080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563026a7b630 0x5630269d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f915bef4080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x563026a7a760 con 0x563026a7b260 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f915bef4080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563026a72140 con 0x563026a7b630 2023-11-28T13:02:05.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f915bef4080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630269cf400 con 0x5630269eb930 2023-11-28T13:02:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41384/0 (socket says 172.21.15.153:41384) 2023-11-28T13:02:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 -- 172.21.15.153:0/1733684603 learned_addr learned my addr 172.21.15.153:0/1733684603 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f9151ff7700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5630269eb930 0x563026a6b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91527f8700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563026a7b630 0x5630269d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 -- 172.21.15.153:0/1733684603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5630269eb930 msgr2=0x563026a6b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5630269eb930 0x563026a6b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 -- 172.21.15.153:0/1733684603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563026a7b630 msgr2=0x5630269d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563026a7b630 0x5630269d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.092+0000 7f91517f6700 1 -- 172.21.15.153:0/1733684603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630269104f0 con 0x563026a7b260 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.093+0000 7f91517f6700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269d04a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9148008760 tx=0x7f9148008a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.093+0000 7f9142ffd700 1 -- 172.21.15.153:0/1733684603 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800f040 con 0x563026a7b260 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.093+0000 7f9142ffd700 1 -- 172.21.15.153:0/1733684603 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9148015940 con 0x563026a7b260 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.093+0000 7f9142ffd700 1 -- 172.21.15.153:0/1733684603 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148015cb0 con 0x563026a7b260 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.093+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5630268e9830 con 0x563026a7b260 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.093+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c00a1c0 con 0x563026a7b260 2023-11-28T13:02:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.094+0000 7f9142ffd700 1 -- 172.21.15.153:0/1733684603 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9148025070 con 0x563026a7b260 2023-11-28T13:02:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.094+0000 7f9142ffd700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9134066c10 0x7f91340690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.095+0000 7f9142ffd700 1 -- 172.21.15.153:0/1733684603 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9148085560 con 0x563026a7b260 2023-11-28T13:02:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.095+0000 7f9151ff7700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9134066c10 0x7f91340690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.095+0000 7f915bef4080 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x5630269f0350 0x563026b1e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.095+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 --> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) v8 -- 0x563026a6b8f0 con 0x5630269f0350 2023-11-28T13:02:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.095+0000 7f91527f8700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x5630269f0350 0x563026b1e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.095+0000 7f9151ff7700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9134066c10 0x7f91340690c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9144000c00 tx=0x7f9144001740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.095+0000 7f91527f8700 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x5630269f0350 0x563026b1e010 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f91527f8700 1 -- 172.21.15.153:0/1733684603 <== osd.1 v2:172.21.15.153:6816/103564 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v51'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f914c017040 con 0x5630269f0350 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x5630269f0350 msgr2=0x563026b1e010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x5630269f0350 0x563026b1e010 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9134066c10 msgr2=0x7f91340690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9134066c10 0x7f91340690c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9144000c00 tx=0x7f9144001740 comp rx=0 tx=0).stop 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 msgr2=0x5630269d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 --2- 172.21.15.153:0/1733684603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x563026a7b260 0x5630269d04a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9148008760 tx=0x7f9148008a70 comp rx=0 tx=0).stop 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f91527f8700 1 -- 172.21.15.153:0/1733684603 reap_dead start 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 shutdown_connections 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 >> 172.21.15.153:0/1733684603 conn(0x5630269ef9e0 msgr2=0x563026a77970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.096+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 shutdown_connections 2023-11-28T13:02:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.097+0000 7f915bef4080 1 -- 172.21.15.153:0/1733684603 wait complete. 2023-11-28T13:02:05.106 DEBUG:teuthology.orchestra.run.smithi153:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T13:02:05.108 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T13:02:05.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.234+0000 7f668808f700 1 Processor -- start 2023-11-28T13:02:05.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.234+0000 7f668808f700 1 -- start start 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668808f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f66800cded0 0x7f6680161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668808f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f668015dc30 0x7f66800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668808f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6680160d70 con 0x7f668015a760 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668808f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668015e0d0 con 0x7f668015dc30 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668808f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668015ab30 con 0x7f66800cded0 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668662c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f668015dc30 0x7f66800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668562a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f66800cded0 0x7f6680161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668662c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f668015dc30 0x7f66800b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41396/0 (socket says 172.21.15.153:41396) 2023-11-28T13:02:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f668662c700 1 -- 172.21.15.153:0/2353088146 learned_addr learned my addr 172.21.15.153:0/2353088146 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.235+0000 7f6685e2b700 1 --1- 172.21.15.153:0/2353088146 >> v1:172.21.15.177:6789/0 conn(0x7f668015a760 0x7f66800cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:36170/0 (socket says 172.21.15.153:36170) 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f668562a700 1 -- 172.21.15.153:0/2353088146 >> v2:172.21.15.153:3300/0 conn(0x7f668015dc30 msgr2=0x7f66800b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f668562a700 1 --2- 172.21.15.153:0/2353088146 >> v2:172.21.15.153:3300/0 conn(0x7f668015dc30 0x7f66800b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f668562a700 1 -- 172.21.15.153:0/2353088146 >> v1:172.21.15.177:6789/0 conn(0x7f668015a760 legacy=0x7f66800cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f668562a700 1 -- 172.21.15.153:0/2353088146 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66800b5130 con 0x7f66800cded0 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f6684e29700 1 -- 172.21.15.153:0/2353088146 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141122028 0 0) 0x7f6680160d70 con 0x7f668015a760 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f668562a700 1 --2- 172.21.15.153:0/2353088146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 0x7f6680161200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f66800d2710 tx=0x7f6670009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927d0ea1d6a582a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f6684e29700 1 -- 172.21.15.153:0/2353088146 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670011830 con 0x7f66800cded0 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f6684e29700 1 -- 172.21.15.153:0/2353088146 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6670011990 con 0x7f66800cded0 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.236+0000 7f6684e29700 1 -- 172.21.15.153:0/2353088146 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670011d20 con 0x7f66800cded0 2023-11-28T13:02:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 -- 172.21.15.153:0/2353088146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 msgr2=0x7f6680161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 --2- 172.21.15.153:0/2353088146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 0x7f6680161200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f66800d2710 tx=0x7f6670009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 -- 172.21.15.153:0/2353088146 shutdown_connections 2023-11-28T13:02:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 --2- 172.21.15.153:0/2353088146 >> v2:172.21.15.153:3300/0 conn(0x7f668015dc30 0x7f66800b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 --2- 172.21.15.153:0/2353088146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 0x7f6680161200 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 -- 172.21.15.153:0/2353088146 >> 172.21.15.153:0/2353088146 conn(0x7f6680072620 msgr2=0x7f66800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 -- 172.21.15.153:0/2353088146 shutdown_connections 2023-11-28T13:02:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.237+0000 7f668808f700 1 -- 172.21.15.153:0/2353088146 wait complete. 2023-11-28T13:02:05.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.238+0000 7f668808f700 1 Processor -- start 2023-11-28T13:02:05.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.238+0000 7f668808f700 1 -- start start 2023-11-28T13:02:05.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668808f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 0x7f66800c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668808f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f668015a760 0x7f66801f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668808f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f668015dc30 0x7f66801f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668808f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66800b6cc0 con 0x7f668015a760 2023-11-28T13:02:05.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668808f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66801f6820 con 0x7f66800cded0 2023-11-28T13:02:05.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668808f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66800b60b0 con 0x7f668015dc30 2023-11-28T13:02:05.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f668015a760 0x7f66801f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f668015a760 0x7f66801f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41400/0 (socket says 172.21.15.153:41400) 2023-11-28T13:02:05.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 -- 172.21.15.153:0/810811565 learned_addr learned my addr 172.21.15.153:0/810811565 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f6685e2b700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 0x7f66800c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668662c700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f668015dc30 0x7f66801f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 -- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f668015dc30 msgr2=0x7f66801f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f668015dc30 0x7f66801f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 -- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 msgr2=0x7f66800c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 0x7f66800c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 -- 172.21.15.153:0/810811565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66700094a0 con 0x7f668015a760 2023-11-28T13:02:05.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668562a700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f668015a760 0x7f66801f62e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6670000c00 tx=0x7f6670004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f66767fc700 1 -- 172.21.15.153:0/810811565 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667001a910 con 0x7f668015a760 2023-11-28T13:02:05.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f66767fc700 1 -- 172.21.15.153:0/810811565 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667001aa70 con 0x7f668015a760 2023-11-28T13:02:05.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66800b5670 con 0x7f668015a760 2023-11-28T13:02:05.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.239+0000 7f66767fc700 1 -- 172.21.15.153:0/810811565 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667001ade0 con 0x7f668015a760 2023-11-28T13:02:05.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.240+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66800b5390 con 0x7f668015a760 2023-11-28T13:02:05.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.242+0000 7f66767fc700 1 -- 172.21.15.153:0/810811565 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f667002b030 con 0x7f668015a760 2023-11-28T13:02:05.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.242+0000 7f66767fc700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f66640682f0 0x7f666406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.242+0000 7f66767fc700 1 -- 172.21.15.153:0/810811565 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6670086f90 con 0x7f668015a760 2023-11-28T13:02:05.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.242+0000 7f6685e2b700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f66640682f0 0x7f666406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.243+0000 7f6685e2b700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f66640682f0 0x7f666406a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f667c000f60 tx=0x7f667c007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.244+0000 7f6694218b80 1 Processor -- start 2023-11-28T13:02:05.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.244+0000 7f6694218b80 1 -- start start 2023-11-28T13:02:05.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee674af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69c200 0x55cbee69ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f6694218b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69f1f0 0x55cbee6ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f6694218b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee55ec90 con 0x55cbee69f1f0 2023-11-28T13:02:05.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee5cd420 con 0x55cbee69c200 2023-11-28T13:02:05.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbee5c6950 con 0x55cbee69e8e0 2023-11-28T13:02:05.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69f1f0 0x55cbee6ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69f1f0 0x55cbee6ba840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41410/0 (socket says 172.21.15.153:41410) 2023-11-28T13:02:05.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 -- 172.21.15.153:0/178616735 learned_addr learned my addr 172.21.15.153:0/178616735 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f6685e2b700 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee674af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668562a700 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69c200 0x55cbee69ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 -- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 msgr2=0x55cbee674af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee674af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 -- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69c200 msgr2=0x55cbee69ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69c200 0x55cbee69ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 -- 172.21.15.153:0/178616735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbee4486b0 con 0x55cbee69f1f0 2023-11-28T13:02:05.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f668662c700 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69f1f0 0x55cbee6ba840 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6678009eb0 tx=0x7f6678009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cdf403856a23d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f66637fe700 1 -- 172.21.15.153:0/178616735 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800e070 con 0x55cbee69f1f0 2023-11-28T13:02:05.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f66637fe700 1 -- 172.21.15.153:0/178616735 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667800f040 con 0x55cbee69f1f0 2023-11-28T13:02:05.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.245+0000 7f66637fe700 1 -- 172.21.15.153:0/178616735 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678015640 con 0x55cbee69f1f0 2023-11-28T13:02:05.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 -- 172.21.15.153:0/178616735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69f1f0 msgr2=0x55cbee6ba840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69f1f0 0x55cbee6ba840 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6678009eb0 tx=0x7f6678009810 comp rx=0 tx=0).stop 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 -- 172.21.15.153:0/178616735 shutdown_connections 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69f1f0 0x55cbee6ba840 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee674af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 --2- 172.21.15.153:0/178616735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69c200 0x55cbee69ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 -- 172.21.15.153:0/178616735 >> 172.21.15.153:0/178616735 conn(0x55cbee671e30 msgr2=0x55cbee6727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 -- 172.21.15.153:0/178616735 shutdown_connections 2023-11-28T13:02:05.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.246+0000 7f6694218b80 1 -- 172.21.15.153:0/178616735 wait complete. 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.247+0000 7f6694218b80 1 -- start start 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.247+0000 7f6694218b80 1 Processor -- start 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee691740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f6694218b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee690f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f6694218b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee44a9a0 con 0x55cbee69e8e0 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee6a0cd0 con 0x55cbee69f1f0 2023-11-28T13:02:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbee691cb0 con 0x55cbee69c200 2023-11-28T13:02:05.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f6694218b80 10 client.0 _ll_register_callbacks cb 0x55cbee46c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:05.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f668562a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f668562a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6909d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41414/0 (socket says 172.21.15.153:41414) 2023-11-28T13:02:05.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.248+0000 7f668562a700 1 -- 172.21.15.153:0/3958874402 learned_addr learned my addr 172.21.15.153:0/3958874402 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f668662c700 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee690f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6685e2b700 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee691740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f668562a700 1 -- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 msgr2=0x55cbee691740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f668562a700 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee691740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f668562a700 1 -- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 msgr2=0x55cbee690f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f668562a700 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee690f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f668562a700 1 -- 172.21.15.153:0/3958874402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6678009b30 con 0x55cbee69e8e0 2023-11-28T13:02:05.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f668562a700 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6909d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6670057320 tx=0x7f6670085580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823550af6bb530b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6660ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6660ff9700 1 -- 172.21.15.153:0/3958874402 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667005f210 con 0x55cbee69e8e0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6660ff9700 1 -- 172.21.15.153:0/3958874402 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66700533e0 con 0x55cbee69e8e0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbee594210 con 0x55cbee69e8e0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6660ff9700 1 -- 172.21.15.153:0/3958874402 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66700535a0 con 0x55cbee69e8e0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 10 client.5742 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55cbee3f4e60 con 0x55cbee69e8e0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6660ff9700 1 -- 172.21.15.153:0/3958874402 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbee594210 con 0x55cbee69e8e0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6660ff9700 1 -- 172.21.15.153:0/3958874402 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f66700850d0 con 0x55cbee69e8e0 2023-11-28T13:02:05.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 10 client.5742 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 10 client.5742 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 20 client.5742 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee69faa0 0x55cbee69fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 4 client.5742 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.249+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbee44afe0 con 0x55cbee69faa0 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.250+0000 7f6685e2b700 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee69faa0 0x55cbee69fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.250+0000 7f6660ff9700 10 client.5742 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.250+0000 7f6685e2b700 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee69faa0 0x55cbee69fe70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce2d89e0e6c2903 server_cookie=f9114d46d98f7d24 in_seq=0 out_seq=0 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.250+0000 7f6660ff9700 1 -- 172.21.15.153:0/3958874402 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55cbee44afe0 con 0x55cbee69faa0 2023-11-28T13:02:05.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.250+0000 7f6660ff9700 10 client.5742 handle_command_reply: tid=0 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 client.5742 shutdown 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 msgr2=0x55cbee6909d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6909d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6670057320 tx=0x7f6670085580 comp rx=0 tx=0).stop 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 shutdown_connections 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee69faa0 0x55cbee69fe70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee690f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6660ff9700 0 client.5742 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee691740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 --2- 172.21.15.153:0/3958874402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6909d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 >> 172.21.15.153:0/3958874402 conn(0x55cbee671e30 msgr2=0x55cbee673300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 shutdown_connections 2023-11-28T13:02:05.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 1 -- 172.21.15.153:0/3958874402 wait complete. 2023-11-28T13:02:05.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.251+0000 7f6694218b80 20 client.5742 trim_cache size 0 max 16384 2023-11-28T13:02:05.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.256+0000 7f5cfec26700 1 Processor -- start 2023-11-28T13:02:05.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfec26700 1 -- start start 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfec26700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf80ce070 0x7f5cf80cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfec26700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5cf80b3560 0x7f5cf80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfec26700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5cf815ad30 0x7f5cf815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfec26700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf80d11b0 con 0x7f5cf815ad30 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfec26700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf815bd40 con 0x7f5cf80b3560 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfec26700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf815ba60 con 0x7f5cf80ce070 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfd1c3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5cf815ad30 0x7f5cf815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfd1c3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5cf815ad30 0x7f5cf815b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41424/0 (socket says 172.21.15.153:41424) 2023-11-28T13:02:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.257+0000 7f5cfd1c3700 1 -- 172.21.15.153:0/695959667 learned_addr learned my addr 172.21.15.153:0/695959667 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cf7fff700 1 --2- 172.21.15.153:0/695959667 >> v2:172.21.15.177:3300/0 conn(0x7f5cf80b3560 0x7f5cf80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cfc9c2700 1 --2- 172.21.15.153:0/695959667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf80ce070 0x7f5cf80cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cfd1c3700 1 -- 172.21.15.153:0/695959667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf80ce070 msgr2=0x7f5cf80cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cfd1c3700 1 --2- 172.21.15.153:0/695959667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf80ce070 0x7f5cf80cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cfd1c3700 1 -- 172.21.15.153:0/695959667 >> v2:172.21.15.177:3300/0 conn(0x7f5cf80b3560 msgr2=0x7f5cf80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cfd1c3700 1 --2- 172.21.15.153:0/695959667 >> v2:172.21.15.177:3300/0 conn(0x7f5cf80b3560 0x7f5cf80d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cfd1c3700 1 -- 172.21.15.153:0/695959667 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80b2a80 con 0x7f5cf815ad30 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cfd1c3700 1 --2- 172.21.15.153:0/695959667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf815ad30 0x7f5cf815b1b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8066ed0 tx=0x7f5cf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104b5a62c5091bfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cf77fe700 1 -- 172.21.15.153:0/695959667 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000e070 con 0x7f5cf815ad30 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cf77fe700 1 -- 172.21.15.153:0/695959667 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf000b910 con 0x7f5cf815ad30 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.258+0000 7f5cf77fe700 1 -- 172.21.15.153:0/695959667 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000bcc0 con 0x7f5cf815ad30 2023-11-28T13:02:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 -- 172.21.15.153:0/695959667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf815ad30 msgr2=0x7f5cf815b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 --2- 172.21.15.153:0/695959667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf815ad30 0x7f5cf815b1b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8066ed0 tx=0x7f5cf000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 -- 172.21.15.153:0/695959667 shutdown_connections 2023-11-28T13:02:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 --2- 172.21.15.153:0/695959667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf80ce070 0x7f5cf80cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 --2- 172.21.15.153:0/695959667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf815ad30 0x7f5cf815b1b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 --2- 172.21.15.153:0/695959667 >> v2:172.21.15.177:3300/0 conn(0x7f5cf80b3560 0x7f5cf80d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 -- 172.21.15.153:0/695959667 >> 172.21.15.153:0/695959667 conn(0x7f5cf806a270 msgr2=0x7f5cf806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 -- 172.21.15.153:0/695959667 shutdown_connections 2023-11-28T13:02:05.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.259+0000 7f5cfec26700 1 -- 172.21.15.153:0/695959667 wait complete. 2023-11-28T13:02:05.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.260+0000 7f5cfec26700 1 Processor -- start 2023-11-28T13:02:05.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.260+0000 7f5cfec26700 1 -- start start 2023-11-28T13:02:05.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfec26700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf80b3560 0x7f5cf815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfec26700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf80ce070 0x7f5cf80c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.282 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfec26700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf815ad30 0x7f5cf815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfec26700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80ae310 con 0x7f5cf80b3560 2023-11-28T13:02:05.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfec26700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80ae5f0 con 0x7f5cf80ce070 2023-11-28T13:02:05.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfec26700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf815a940 con 0x7f5cf815ad30 2023-11-28T13:02:05.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf80b3560 0x7f5cf815a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf80b3560 0x7f5cf815a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41426/0 (socket says 172.21.15.153:41426) 2023-11-28T13:02:05.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 -- 172.21.15.153:0/1899773779 learned_addr learned my addr 172.21.15.153:0/1899773779 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.283 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cf7fff700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf80ce070 0x7f5cf80c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfd1c3700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf815ad30 0x7f5cf815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 -- 172.21.15.153:0/1899773779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf815ad30 msgr2=0x7f5cf815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cf815ad30 0x7f5cf815cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 -- 172.21.15.153:0/1899773779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf80ce070 msgr2=0x7f5cf80c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cf80ce070 0x7f5cf80c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 -- 172.21.15.153:0/1899773779 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009b30 con 0x7f5cf80b3560 2023-11-28T13:02:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfc9c2700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5cf80b3560 0x7f5cf815a400 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00e9b0 tx=0x7f5cec00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec009710 con 0x7f5cf80b3560 2023-11-28T13:02:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec00ee50 con 0x7f5cf80b3560 2023-11-28T13:02:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00ca50 con 0x7f5cf80b3560 2023-11-28T13:02:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.261+0000 7f5cfec26700 1 -- 172.21.15.153:0/1899773779 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf80b3040 con 0x7f5cf80b3560 2023-11-28T13:02:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.262+0000 7f5cfec26700 1 -- 172.21.15.153:0/1899773779 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf80b2d60 con 0x7f5cf80b3560 2023-11-28T13:02:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.262+0000 7f5cfec26700 1 -- 172.21.15.153:0/1899773779 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5cf815b6f0 con 0x7f5cf80b3560 2023-11-28T13:02:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.263+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5cec019030 con 0x7f5cf80b3560 2023-11-28T13:02:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.263+0000 7f5cf57fa700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cdc0682f0 0x7f5cdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.263+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5cec085f20 con 0x7f5cf80b3560 2023-11-28T13:02:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.263+0000 7f5cf7fff700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cdc0682f0 0x7f5cdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.264+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7f5cec0863a0 con 0x7f5cf80b3560 2023-11-28T13:02:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.264+0000 7f5cfec26700 1 -- 172.21.15.153:0/1899773779 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f5cf80678e0 con 0x7f5cf80b3560 2023-11-28T13:02:05.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.264+0000 7f5cf7fff700 1 --2- 172.21.15.153:0/1899773779 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5cdc0682f0 0x7f5cdc06a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5cf815ded0 tx=0x7f5ce4009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.290 INFO:tasks.ceph.mds.b.smithi177.stderr:2023-11-28T13:02:05.275+0000 7fb322aac700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5706=0-4194304@1} caps={5706=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5706 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ad36727700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:02:05.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.269+0000 7f6694218b80 1 Processor -- start 2023-11-28T13:02:05.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.269+0000 7f6694218b80 1 -- start start 2023-11-28T13:02:05.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.269+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee69c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6694218b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee6d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6694218b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee6a2700 con 0x55cbee69e8e0 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee6a0010 con 0x55cbee69f1f0 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbee66f020 con 0x55cbee69c200 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6d1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41436/0 (socket says 172.21.15.153:41436) 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 -- 172.21.15.153:0/1191539552 learned_addr learned my addr 172.21.15.153:0/1191539552 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6685e2b700 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee69c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668662c700 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee6d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 -- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 msgr2=0x55cbee69c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee69c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 -- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 msgr2=0x55cbee6d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee6d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 -- 172.21.15.153:0/1191539552 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbee58d210 con 0x55cbee69e8e0 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f668562a700 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6d1d80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f66700570f0 tx=0x7f667005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a710af245474c4f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6663fff700 1 -- 172.21.15.153:0/1191539552 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667005f210 con 0x55cbee69e8e0 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6663fff700 1 -- 172.21.15.153:0/1191539552 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6670056470 con 0x55cbee69e8e0 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.270+0000 7f6663fff700 1 -- 172.21.15.153:0/1191539552 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700531e0 con 0x55cbee69e8e0 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 -- 172.21.15.153:0/1191539552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 msgr2=0x55cbee6d1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6d1d80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f66700570f0 tx=0x7f667005b000 comp rx=0 tx=0).stop 2023-11-28T13:02:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 -- 172.21.15.153:0/1191539552 shutdown_connections 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee6d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69c200 0x55cbee69c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 --2- 172.21.15.153:0/1191539552 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69e8e0 0x55cbee6d1d80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 -- 172.21.15.153:0/1191539552 >> 172.21.15.153:0/1191539552 conn(0x55cbee671e30 msgr2=0x55cbee6701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 -- 172.21.15.153:0/1191539552 shutdown_connections 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 -- 172.21.15.153:0/1191539552 wait complete. 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.271+0000 7f6694218b80 1 -- start start 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.272+0000 7f6694218b80 1 Processor -- start 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.272+0000 7f6694218b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69c200 0x55cbee6d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.272+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee6d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.272+0000 7f6694218b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee68a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6694218b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee68daa0 con 0x55cbee69c200 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbee6b2dc0 con 0x55cbee69f1f0 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6694218b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbee689b70 con 0x55cbee69e8e0 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6694218b80 10 client.0 _ll_register_callbacks cb 0x55cbee46c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69c200 0x55cbee6d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69c200 0x55cbee6d2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41440/0 (socket says 172.21.15.153:41440) 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 -- 172.21.15.153:0/1210007429 learned_addr learned my addr 172.21.15.153:0/1210007429 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f668562a700 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee6d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f668662c700 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee68a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 -- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 msgr2=0x55cbee6d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee6d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 -- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 msgr2=0x55cbee68a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee68a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 -- 172.21.15.153:0/1210007429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbee3f4e60 con 0x55cbee69c200 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6685e2b700 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69c200 0x55cbee6d2040 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f667c007fd0 tx=0x7f667c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52fc9cf54184b86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6660ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6660ff9700 1 -- 172.21.15.153:0/1210007429 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c026e60 con 0x55cbee69c200 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6660ff9700 1 -- 172.21.15.153:0/1210007429 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f667c0151f0 con 0x55cbee69c200 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbee584230 con 0x55cbee69c200 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6660ff9700 1 -- 172.21.15.153:0/1210007429 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f667c016c30 con 0x55cbee69c200 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6694218b80 10 client.5754 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.273+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55cbee41b2d0 con 0x55cbee69c200 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6660ff9700 1 -- 172.21.15.153:0/1210007429 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cbee584230 con 0x55cbee69c200 2023-11-28T13:02:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6660ff9700 1 -- 172.21.15.153:0/1210007429 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f667c024420 con 0x55cbee69c200 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6694218b80 10 client.5754 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6694218b80 10 client.5754 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6694218b80 20 client.5754 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6694218b80 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee68cc20 0x55cbee68a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6694218b80 4 client.5754 mds_command: new command op to 5547 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55cbee56e240 con 0x55cbee68cc20 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f668562a700 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee68cc20 0x55cbee68a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f6660ff9700 10 client.5754 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.274+0000 7f668562a700 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee68cc20 0x55cbee68a540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b57fe5ceea0682 server_cookie=ed7bccddffa297e8 in_seq=0 out_seq=0 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6660ff9700 1 -- 172.21.15.153:0/1210007429 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cbee56e240 con 0x55cbee68cc20 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6660ff9700 10 client.5754 handle_command_reply: tid=0 2023-11-28T13:02:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 client.5754 shutdown 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69c200 msgr2=0x55cbee6d2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69c200 0x55cbee6d2040 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f667c007fd0 tx=0x7f667c000ed0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 shutdown_connections 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55cbee69c200 0x55cbee6d2040 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x55cbee68cc20 0x55cbee68a540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbee69f1f0 0x55cbee68a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 --2- 172.21.15.153:0/1210007429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cbee69e8e0 0x55cbee6d2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6660ff9700 0 client.5754 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 >> 172.21.15.153:0/1210007429 conn(0x55cbee671e30 msgr2=0x55cbee69c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 shutdown_connections 2023-11-28T13:02:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 1 -- 172.21.15.153:0/1210007429 wait complete. 2023-11-28T13:02:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.275+0000 7f6694218b80 20 client.5754 trim_cache size 0 max 16384 2023-11-28T13:02:05.303 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:02:05.303 INFO:teuthology.orchestra.run.smithi153.stdout: "return_code": 0, 2023-11-28T13:02:05.303 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_tag": "8843cb55-d1b5-4459-8236-16c4c9e46534", 2023-11-28T13:02:05.304 INFO:teuthology.orchestra.run.smithi153.stdout: "mode": "asynchronous" 2023-11-28T13:02:05.304 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:02:05.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f66640682f0 msgr2=0x7f666406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f66640682f0 0x7f666406a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f667c000f60 tx=0x7f667c007be0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f668015a760 msgr2=0x7f66801f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f668015a760 0x7f66801f62e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6670000c00 tx=0x7f6670004750 comp rx=0 tx=0).stop 2023-11-28T13:02:05.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 shutdown_connections 2023-11-28T13:02:05.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f668015dc30 0x7f66801f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66800cded0 0x7f66800c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f66640682f0 0x7f666406a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 --2- 172.21.15.153:0/810811565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f668015a760 0x7f66801f62e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 >> 172.21.15.153:0/810811565 conn(0x7f6680072620 msgr2=0x7f668015f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 shutdown_connections 2023-11-28T13:02:05.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.276+0000 7f668808f700 1 -- 172.21.15.153:0/810811565 wait complete. 2023-11-28T13:02:05.307 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T13:02:05.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.429+0000 7fb472bdf700 1 Processor -- start 2023-11-28T13:02:05.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.429+0000 7fb472bdf700 1 -- start start 2023-11-28T13:02:05.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb472bdf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb46c0cd8c0 0x7fb46c140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb472bdf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cdc90 0x7fb46c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb472bdf700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c13ff00 con 0x7fb46c0b2e70 2023-11-28T13:02:05.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb472bdf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c1404c0 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb472bdf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c15aa80 con 0x7fb46c0cdc90 2023-11-28T13:02:05.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cdc90 0x7fb46c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb47097b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb46c0cd8c0 0x7fb46c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cdc90 0x7fb46c0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59060/0 (socket says 172.21.15.153:59060) 2023-11-28T13:02:05.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb46bfff700 1 -- 172.21.15.153:0/136465520 learned_addr learned my addr 172.21.15.153:0/136465520 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb46bfff700 1 -- 172.21.15.153:0/136465520 >> v2:172.21.15.177:3300/0 conn(0x7fb46c0cd8c0 msgr2=0x7fb46c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb46bfff700 1 --2- 172.21.15.153:0/136465520 >> v2:172.21.15.177:3300/0 conn(0x7fb46c0cd8c0 0x7fb46c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb46bfff700 1 -- 172.21.15.153:0/136465520 >> v1:172.21.15.177:6789/0 conn(0x7fb46c0b2e70 legacy=0x7fb46c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.430+0000 7fb46bfff700 1 -- 172.21.15.153:0/136465520 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0b3310 con 0x7fb46c0cdc90 2023-11-28T13:02:05.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb46bfff700 1 --2- 172.21.15.153:0/136465520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cdc90 0x7fb46c0b2930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4580098f0 tx=0x7fb458009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92620b737c35cb95 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb46b7fe700 1 -- 172.21.15.153:0/136465520 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458011830 con 0x7fb46c0cdc90 2023-11-28T13:02:05.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb46b7fe700 1 -- 172.21.15.153:0/136465520 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458011990 con 0x7fb46c0cdc90 2023-11-28T13:02:05.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb46b7fe700 1 -- 172.21.15.153:0/136465520 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458011d20 con 0x7fb46c0cdc90 2023-11-28T13:02:05.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 -- 172.21.15.153:0/136465520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cdc90 msgr2=0x7fb46c0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 --2- 172.21.15.153:0/136465520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cdc90 0x7fb46c0b2930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4580098f0 tx=0x7fb458009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:05.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 -- 172.21.15.153:0/136465520 shutdown_connections 2023-11-28T13:02:05.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 --2- 172.21.15.153:0/136465520 >> v2:172.21.15.177:3300/0 conn(0x7fb46c0cd8c0 0x7fb46c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 --2- 172.21.15.153:0/136465520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cdc90 0x7fb46c0b2930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 -- 172.21.15.153:0/136465520 >> 172.21.15.153:0/136465520 conn(0x7fb46c06aed0 msgr2=0x7fb46c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 -- 172.21.15.153:0/136465520 shutdown_connections 2023-11-28T13:02:05.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.431+0000 7fb472bdf700 1 -- 172.21.15.153:0/136465520 wait complete. 2023-11-28T13:02:05.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.432+0000 7fb472bdf700 1 Processor -- start 2023-11-28T13:02:05.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.433+0000 7fb472bdf700 1 -- start start 2023-11-28T13:02:05.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.433+0000 7fb472bdf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb46c0b2e70 0x7fb46c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.433+0000 7fb472bdf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cd8c0 0x7fb46c1f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.433+0000 7fb472bdf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb46c0cdc90 0x7fb46c13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.433+0000 7fb472bdf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1f2360 con 0x7fb46c0cdc90 2023-11-28T13:02:05.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.433+0000 7fb472bdf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c06b380 con 0x7fb46c0b2e70 2023-11-28T13:02:05.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.433+0000 7fb472bdf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0cf490 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb47117c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb46c0cdc90 0x7fb46c13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cd8c0 0x7fb46c1f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb47117c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb46c0cdc90 0x7fb46c13ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41458/0 (socket says 172.21.15.153:41458) 2023-11-28T13:02:05.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb47097b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb46c0b2e70 0x7fb46c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb47117c700 1 -- 172.21.15.153:0/1977448154 learned_addr learned my addr 172.21.15.153:0/1977448154 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cd8c0 0x7fb46c1f1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59066/0 (socket says 172.21.15.153:59066) 2023-11-28T13:02:05.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 -- 172.21.15.153:0/1977448154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb46c0b2e70 msgr2=0x7fb46c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb46c0b2e70 0x7fb46c143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 -- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb46c0cdc90 msgr2=0x7fb46c13ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb46c0cdc90 0x7fb46c13ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 -- 172.21.15.153:0/1977448154 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009550 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb47117c700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb46c0cdc90 0x7fb46c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:05.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb46bfff700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cd8c0 0x7fb46c1f1e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb458016000 tx=0x7fb458004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb4697fa700 1 -- 172.21.15.153:0/1977448154 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458004510 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb4697fa700 1 -- 172.21.15.153:0/1977448154 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45801e070 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c140150 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb4697fa700 1 -- 172.21.15.153:0/1977448154 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580038e0 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.434+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0b35f0 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.436+0000 7fb4697fa700 1 -- 172.21.15.153:0/1977448154 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb45800f050 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.436+0000 7fb4697fa700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4500682f0 0x7fb45006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.436+0000 7fb47097b700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4500682f0 0x7fb45006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.436+0000 7fb4697fa700 1 -- 172.21.15.153:0/1977448154 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb458086ba0 con 0x7fb46c0cd8c0 2023-11-28T13:02:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.437+0000 7fb47097b700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4500682f0 0x7fb45006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb460009bc0 tx=0x7fb4600091c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.438+0000 7fb47ed68b80 1 Processor -- start 2023-11-28T13:02:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.438+0000 7fb47ed68b80 1 -- start start 2023-11-28T13:02:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec191860 0x5643ec167a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec191c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec192170 0x5643ec1ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ec051c10 con 0x5643ec192170 2023-11-28T13:02:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ec0c03a0 con 0x5643ec191860 2023-11-28T13:02:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643ec0b98d0 con 0x5643ec18f180 2023-11-28T13:02:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec192170 0x5643ec1ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec192170 0x5643ec1ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41466/0 (socket says 172.21.15.153:41466) 2023-11-28T13:02:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 -- 172.21.15.153:0/3442608385 learned_addr learned my addr 172.21.15.153:0/3442608385 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb46bfff700 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec191c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47097b700 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec191860 0x5643ec167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 -- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 msgr2=0x5643ec191c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec191c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 -- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec191860 msgr2=0x5643ec167a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec191860 0x5643ec167a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 -- 172.21.15.153:0/3442608385 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643ebf3b630 con 0x5643ec192170 2023-11-28T13:02:05.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb46bfff700 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec191c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:05.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.439+0000 7fb47117c700 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec192170 0x5643ec1ad7c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb46400d5a0 tx=0x7fb464001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a3b015266e0707 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb4567fc700 1 -- 172.21.15.153:0/3442608385 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46400d220 con 0x5643ec192170 2023-11-28T13:02:05.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb4567fc700 1 -- 172.21.15.153:0/3442608385 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb464003c10 con 0x5643ec192170 2023-11-28T13:02:05.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb4567fc700 1 -- 172.21.15.153:0/3442608385 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb464005110 con 0x5643ec192170 2023-11-28T13:02:05.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 -- 172.21.15.153:0/3442608385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec192170 msgr2=0x5643ec1ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec192170 0x5643ec1ad7c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb46400d5a0 tx=0x7fb464001b40 comp rx=0 tx=0).stop 2023-11-28T13:02:05.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 -- 172.21.15.153:0/3442608385 shutdown_connections 2023-11-28T13:02:05.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec192170 0x5643ec1ad7c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec191860 0x5643ec167a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/3442608385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec191c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 -- 172.21.15.153:0/3442608385 >> 172.21.15.153:0/3442608385 conn(0x5643ec164db0 msgr2=0x5643ec165740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 -- 172.21.15.153:0/3442608385 shutdown_connections 2023-11-28T13:02:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.440+0000 7fb47ed68b80 1 -- 172.21.15.153:0/3442608385 wait complete. 2023-11-28T13:02:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.441+0000 7fb47ed68b80 1 -- start start 2023-11-28T13:02:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.441+0000 7fb47ed68b80 1 Processor -- start 2023-11-28T13:02:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec18f180 0x5643ec1870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec188180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec192170 0x5643ec1b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ebf3d920 con 0x5643ec191860 2023-11-28T13:02:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ec187610 con 0x5643ec18f180 2023-11-28T13:02:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643ec1935c0 con 0x5643ec192170 2023-11-28T13:02:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47ed68b80 10 client.0 _ll_register_callbacks cb 0x5643ebf5f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec188180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec188180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41468/0 (socket says 172.21.15.153:41468) 2023-11-28T13:02:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 -- 172.21.15.153:0/1268029544 learned_addr learned my addr 172.21.15.153:0/1268029544 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47117c700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec192170 0x5643ec1b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 -- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec192170 msgr2=0x5643ec1b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47097b700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec18f180 0x5643ec1870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec192170 0x5643ec1b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 -- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec18f180 msgr2=0x5643ec1870d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec18f180 0x5643ec1870d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 -- 172.21.15.153:0/1268029544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c13ff00 con 0x5643ec191860 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb47117c700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec192170 0x5643ec1b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb46bfff700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec188180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb4580569d0 tx=0x7fb45802c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f28d50546d0bbe3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.442+0000 7fb44b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb44b7fe700 1 -- 172.21.15.153:0/1268029544 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45805eb50 con 0x5643ec191860 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb44b7fe700 1 -- 172.21.15.153:0/1268029544 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb45805ecb0 con 0x5643ec191860 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643ec087190 con 0x5643ec191860 2023-11-28T13:02:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb44b7fe700 1 -- 172.21.15.153:0/1268029544 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb458052d40 con 0x5643ec191860 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 10 client.5763 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5643ebee7de0 con 0x5643ec191860 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb44b7fe700 1 -- 172.21.15.153:0/1268029544 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5643ec087190 con 0x5643ec191860 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb44b7fe700 1 -- 172.21.15.153:0/1268029544 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb458052ea0 con 0x5643ec191860 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 10 client.5763 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 10 client.5763 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 20 client.5763 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec192f10 0x5643ec1afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 4 client.5763 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643ebf3df60 con 0x5643ec192f10 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47097b700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec192f10 0x5643ec1afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb44b7fe700 10 client.5763 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.443+0000 7fb47097b700 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec192f10 0x5643ec1afb80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a608b40672b9509 server_cookie=7c662eb9bf74779d in_seq=0 out_seq=0 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb44b7fe700 1 -- 172.21.15.153:0/1268029544 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5643ebf3df60 con 0x5643ec192f10 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb44b7fe700 10 client.5763 handle_command_reply: tid=0 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 client.5763 shutdown 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 msgr2=0x5643ec188180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec188180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb4580569d0 tx=0x7fb45802c460 comp rx=0 tx=0).stop 2023-11-28T13:02:05.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 shutdown_connections 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec192f10 0x5643ec1afb80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec192170 0x5643ec1b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec18f180 0x5643ec1870d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb44b7fe700 0 client.5763 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/1268029544 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec188180 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 >> 172.21.15.153:0/1268029544 conn(0x5643ec164db0 msgr2=0x5643ec1679e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 shutdown_connections 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 1 -- 172.21.15.153:0/1268029544 wait complete. 2023-11-28T13:02:05.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.444+0000 7fb47ed68b80 20 client.5763 trim_cache size 0 max 16384 2023-11-28T13:02:05.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.461+0000 7fb47ed68b80 1 Processor -- start 2023-11-28T13:02:05.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.461+0000 7fb47ed68b80 1 -- start start 2023-11-28T13:02:05.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec1882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec190fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec186ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ec190d30 con 0x5643ec191860 2023-11-28T13:02:05.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ec1632f0 con 0x5643ec192170 2023-11-28T13:02:05.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643ec1b3920 con 0x5643ec18f180 2023-11-28T13:02:05.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec190fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec190fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41478/0 (socket says 172.21.15.153:41478) 2023-11-28T13:02:05.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 -- 172.21.15.153:0/627092980 learned_addr learned my addr 172.21.15.153:0/627092980 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47097b700 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec1882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47117c700 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec186ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 -- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 msgr2=0x5643ec1882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec1882e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 -- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 msgr2=0x5643ec186ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec186ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 -- 172.21.15.153:0/627092980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643ec080190 con 0x5643ec191860 2023-11-28T13:02:05.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb47097b700 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec1882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:05.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb46bfff700 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec190fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb45805eab0 tx=0x7fb45805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed0fca919b58fd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb456ffd700 1 -- 172.21.15.153:0/627092980 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45805efe0 con 0x5643ec191860 2023-11-28T13:02:05.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb456ffd700 1 -- 172.21.15.153:0/627092980 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45805eb50 con 0x5643ec191860 2023-11-28T13:02:05.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.462+0000 7fb456ffd700 1 -- 172.21.15.153:0/627092980 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458052f60 con 0x5643ec191860 2023-11-28T13:02:05.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 -- 172.21.15.153:0/627092980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 msgr2=0x5643ec190fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec190fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb45805eab0 tx=0x7fb45805c000 comp rx=0 tx=0).stop 2023-11-28T13:02:05.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 -- 172.21.15.153:0/627092980 shutdown_connections 2023-11-28T13:02:05.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec186ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec1882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/627092980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec190fa0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 -- 172.21.15.153:0/627092980 >> 172.21.15.153:0/627092980 conn(0x5643ec164db0 msgr2=0x5643ec168a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 -- 172.21.15.153:0/627092980 shutdown_connections 2023-11-28T13:02:05.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 -- 172.21.15.153:0/627092980 wait complete. 2023-11-28T13:02:05.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.463+0000 7fb47ed68b80 1 -- start start 2023-11-28T13:02:05.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 1 Processor -- start 2023-11-28T13:02:05.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec18f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec18fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec1c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ec1c5fc0 con 0x5643ec191860 2023-11-28T13:02:05.482 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ec165bc0 con 0x5643ec192170 2023-11-28T13:02:05.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643ec165910 con 0x5643ec18f180 2023-11-28T13:02:05.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.464+0000 7fb47ed68b80 10 client.0 _ll_register_callbacks cb 0x5643ebf5f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:05.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec18fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec18fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41484/0 (socket says 172.21.15.153:41484) 2023-11-28T13:02:05.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 -- 172.21.15.153:0/2195581802 learned_addr learned my addr 172.21.15.153:0/2195581802 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:05.483 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb47117c700 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec1c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb47097b700 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec18f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 -- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 msgr2=0x5643ec18f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec18f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 -- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 msgr2=0x5643ec1c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:05.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec1c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 -- 172.21.15.153:0/2195581802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb460005af0 con 0x5643ec191860 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb46bfff700 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec18fa90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb458056c50 tx=0x7fb4580094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23c602ecaae4c8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb44b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb44b7fe700 1 -- 172.21.15.153:0/2195581802 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45805f060 con 0x5643ec191860 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb44b7fe700 1 -- 172.21.15.153:0/2195581802 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb45806a230 con 0x5643ec191860 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643ec0771b0 con 0x5643ec191860 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb44b7fe700 1 -- 172.21.15.153:0/2195581802 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb45806a430 con 0x5643ec191860 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb47ed68b80 10 client.5769 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5643ebee7de0 con 0x5643ec191860 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.465+0000 7fb44b7fe700 1 -- 172.21.15.153:0/2195581802 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5643ec0771b0 con 0x5643ec191860 2023-11-28T13:02:05.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb44b7fe700 1 -- 172.21.15.153:0/2195581802 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb45807f120 con 0x5643ec191860 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 10 client.5769 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 10 client.5769 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 20 client.5769 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec194f80 0x5643ec1c5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 4 client.5769 mds_command: new command op to 5547 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5643ec0611c0 con 0x5643ec194f80 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47097b700 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec194f80 0x5643ec1c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb44b7fe700 10 client.5769 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47097b700 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec194f80 0x5643ec1c5460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abf420dfa168eb9 server_cookie=d0c2e8e31586e13e in_seq=0 out_seq=0 2023-11-28T13:02:05.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb44b7fe700 1 -- 172.21.15.153:0/2195581802 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5643ec0611c0 con 0x5643ec194f80 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb44b7fe700 10 client.5769 handle_command_reply: tid=0 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 client.5769 shutdown 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 msgr2=0x5643ec18fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec18fa90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb458056c50 tx=0x7fb4580094e0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 shutdown_connections 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x5643ec194f80 0x5643ec1c5460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643ec192170 0x5643ec1c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5643ec18f180 0x5643ec18f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb44b7fe700 0 client.5769 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 --2- 172.21.15.153:0/2195581802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5643ec191860 0x5643ec18fa90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.466+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 >> 172.21.15.153:0/2195581802 conn(0x5643ec164db0 msgr2=0x5643ec1906f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 shutdown_connections 2023-11-28T13:02:05.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb47ed68b80 1 -- 172.21.15.153:0/2195581802 wait complete. 2023-11-28T13:02:05.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb47ed68b80 20 client.5769 trim_cache size 0 max 16384 2023-11-28T13:02:05.488 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:02:05.489 INFO:teuthology.orchestra.run.smithi153.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T13:02:05.489 INFO:teuthology.orchestra.run.smithi153.stdout: "scrubs": { 2023-11-28T13:02:05.489 INFO:teuthology.orchestra.run.smithi153.stdout: "8843cb55-d1b5-4459-8236-16c4c9e46534": { 2023-11-28T13:02:05.489 INFO:teuthology.orchestra.run.smithi153.stdout: "path": "/", 2023-11-28T13:02:05.489 INFO:teuthology.orchestra.run.smithi153.stdout: "tag": "8843cb55-d1b5-4459-8236-16c4c9e46534", 2023-11-28T13:02:05.490 INFO:teuthology.orchestra.run.smithi153.stdout: "options": "recursive,repair" 2023-11-28T13:02:05.490 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:05.490 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:05.490 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4500682f0 msgr2=0x7fb45006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4500682f0 0x7fb45006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb460009bc0 tx=0x7fb4600091c0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cd8c0 msgr2=0x7fb46c1f1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cd8c0 0x7fb46c1f1e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb458016000 tx=0x7fb458004750 comp rx=0 tx=0).stop 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 shutdown_connections 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb4500682f0 0x7fb45006a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb46c0cdc90 0x7fb46c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb46c0b2e70 0x7fb46c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 --2- 172.21.15.153:0/1977448154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb46c0cd8c0 0x7fb46c1f1e20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:05.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 >> 172.21.15.153:0/1977448154 conn(0x7fb46c06aed0 msgr2=0x7fb46c158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:05.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 shutdown_connections 2023-11-28T13:02:05.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.467+0000 7fb472bdf700 1 -- 172.21.15.153:0/1977448154 wait complete. 2023-11-28T13:02:05.494 INFO:tasks.cephfs.filesystem:scrub status for tag:8843cb55-d1b5-4459-8236-16c4c9e46534 - {'path': '/', 'tag': '8843cb55-d1b5-4459-8236-16c4c9e46534', 'options': 'recursive,repair'} 2023-11-28T13:02:05.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:05.961+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-28T13:02:05.275476+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f5cec086380 con 0x7f5cf80b3560 2023-11-28T13:02:05.964 INFO:teuthology.orchestra.run.smithi153.stdout: cluster: 2023-11-28T13:02:05.964 INFO:teuthology.orchestra.run.smithi153.stdout: id: 47bfaa00-a151-4537-aa4c-32b0866eb248 2023-11-28T13:02:05.964 INFO:teuthology.orchestra.run.smithi153.stdout: health: HEALTH_OK 2023-11-28T13:02:05.964 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:05.965 INFO:teuthology.orchestra.run.smithi153.stdout: services: 2023-11-28T13:02:05.965 INFO:teuthology.orchestra.run.smithi153.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T13:02:05.965 INFO:teuthology.orchestra.run.smithi153.stdout: mgr: y(active, since 2m), standbys: x 2023-11-28T13:02:05.965 INFO:teuthology.orchestra.run.smithi153.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T13:02:05.965 INFO:teuthology.orchestra.run.smithi153.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T13:02:05.966 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:05.966 INFO:teuthology.orchestra.run.smithi153.stdout: data: 2023-11-28T13:02:05.966 INFO:teuthology.orchestra.run.smithi153.stdout: volumes: 1/1 healthy 2023-11-28T13:02:05.966 INFO:teuthology.orchestra.run.smithi153.stdout: pools: 4 pools, 105 pgs 2023-11-28T13:02:05.966 INFO:teuthology.orchestra.run.smithi153.stdout: objects: 24 objects, 579 KiB 2023-11-28T13:02:05.966 INFO:teuthology.orchestra.run.smithi153.stdout: usage: 77 MiB used, 720 GiB / 720 GiB avail 2023-11-28T13:02:05.967 INFO:teuthology.orchestra.run.smithi153.stdout: pgs: 105 active+clean 2023-11-28T13:02:05.967 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:05.967 INFO:teuthology.orchestra.run.smithi153.stdout: io: 2023-11-28T13:02:05.967 INFO:teuthology.orchestra.run.smithi153.stdout: client: 1.8 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-28T13:02:05.967 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:05.967 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:05.967 INFO:teuthology.orchestra.run.smithi153.stdout:2023-11-28T13:02:05.275476+0000 mds.b [INF] scrub queued for path: / 2023-11-28T13:02:05.968 INFO:teuthology.orchestra.run.smithi153.stdout:2023-11-28T13:02:05.275480+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2023-11-28T13:02:05.968 INFO:teuthology.orchestra.run.smithi153.stdout:2023-11-28T13:02:05.275485+0000 mds.b [INF] scrub summary: active paths [/] 2023-11-28T13:02:05.968 INFO:teuthology.orchestra.run.smithi153.stdout:2023-11-28T13:02:05.276301+0000 mds.b [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T13:02:05.968 INFO:teuthology.orchestra.run.smithi153.stdout:2023-11-28T13:02:05.276309+0000 mds.b [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T13:02:05.968 INFO:teuthology.orchestra.run.smithi153.stdout:2023-11-28T13:02:05.276641+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2023-11-28T13:02:08.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:08.377+0000 7f5cf57fa700 1 -- 172.21.15.153:0/1899773779 <== mon.0 v2:172.21.15.153:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-28T13:02:06.598785+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f5cec05e4d0 con 0x7f5cf80b3560 2023-11-28T13:02:08.379 INFO:teuthology.orchestra.run.smithi153.stdout:2023-11-28T13:02:06.598785+0000 mds.b [INF] scrub summary: idle 2023-11-28T13:02:16.282 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:16.285 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:35.496 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T13:02:35.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.630+0000 7f88ec8b4700 1 Processor -- start 2023-11-28T13:02:35.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.630+0000 7f88ec8b4700 1 -- start start 2023-11-28T13:02:35.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.631+0000 7f88ec8b4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88e806e500 0x7f88e815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.631+0000 7f88ec8b4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 0x7f88e806aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.631+0000 7f88ec8b4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80aee40 con 0x7f88e80ae9d0 2023-11-28T13:02:35.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.631+0000 7f88ec8b4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e815bf10 con 0x7f88e806e500 2023-11-28T13:02:35.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.631+0000 7f88ec8b4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e806c930 con 0x7f88e806a5e0 2023-11-28T13:02:35.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.631+0000 7f88e5d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f88e80ae9d0 0x7f88e806e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59654/0 (socket says 172.21.15.153:59654) 2023-11-28T13:02:35.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.631+0000 7f88e5d9b700 1 -- 172.21.15.153:0/3756228502 learned_addr learned my addr 172.21.15.153:0/3756228502 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:35.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 0x7f88e806aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e659c700 1 --2- 172.21.15.153:0/3756228502 >> v2:172.21.15.177:3300/0 conn(0x7f88e806e500 0x7f88e815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804012972 0 0) 0x7f88e80aee40 con 0x7f88e80ae9d0 2023-11-28T13:02:35.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d00035b0 con 0x7f88e80ae9d0 2023-11-28T13:02:35.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2053519843 0 0) 0x7f88d00035b0 con 0x7f88e80ae9d0 2023-11-28T13:02:35.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 msgr2=0x7f88e806aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 0x7f88e806aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 >> v2:172.21.15.177:3300/0 conn(0x7f88e806e500 msgr2=0x7f88e815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 --2- 172.21.15.153:0/3756228502 >> v2:172.21.15.177:3300/0 conn(0x7f88e806e500 0x7f88e815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e806cc10 con 0x7f88e80ae9d0 2023-11-28T13:02:35.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f88d8001670 con 0x7f88e80ae9d0 2023-11-28T13:02:35.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 >> v1:172.21.15.153:6789/0 conn(0x7f88e80ae9d0 legacy=0x7f88e806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 0x7f88d0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d0003f80 0x7f88d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d0004d90 0x7f88d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e815bf10 con 0x7f88d0003f80 2023-11-28T13:02:35.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e806c930 con 0x7f88d0004d90 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d000baf0 con 0x7f88e806a5e0 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d0003f80 0x7f88d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e659c700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 0x7f88d0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d0004d90 0x7f88d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e5d9b700 1 -- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 msgr2=0x7f88d0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e806a5e0 0x7f88d0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e5d9b700 1 -- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d0004d90 msgr2=0x7f88d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d0004d90 0x7f88d0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.632+0000 7f88e5d9b700 1 -- 172.21.15.153:0/3756228502 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00035b0 con 0x7f88d0003f80 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d0003f80 0x7f88d0004850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f88d8001050 tx=0x7f88d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ead0de1cc40018 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8007c60 con 0x7f88d0003f80 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800cd90 con 0x7f88d0003f80 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88e559a700 1 -- 172.21.15.153:0/3756228502 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8005ba0 con 0x7f88d0003f80 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88ec8b4700 1 -- 172.21.15.153:0/3756228502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d0003f80 msgr2=0x7f88d0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88ec8b4700 1 --2- 172.21.15.153:0/3756228502 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88d0003f80 0x7f88d0004850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f88d8001050 tx=0x7f88d0005880 comp rx=0 tx=0).stop 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88e6d9d700 1 -- 172.21.15.153:0/3756228502 reap_dead start 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88ec8b4700 1 -- 172.21.15.153:0/3756228502 shutdown_connections 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88ec8b4700 1 -- 172.21.15.153:0/3756228502 >> 172.21.15.153:0/3756228502 conn(0x7f88e80b12b0 msgr2=0x7f88e80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88ec8b4700 1 -- 172.21.15.153:0/3756228502 shutdown_connections 2023-11-28T13:02:35.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.633+0000 7f88ec8b4700 1 -- 172.21.15.153:0/3756228502 wait complete. 2023-11-28T13:02:35.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 Processor -- start 2023-11-28T13:02:35.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 -- start start 2023-11-28T13:02:35.653 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d0003f80 0x7f88e814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80c7d60 0x7f88e80b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e80c81b0 0x7f88e80b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80c9330 con 0x7f88e80c81b0 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8150330 con 0x7f88e80c7d60 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.635+0000 7f88ec8b4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80c8720 con 0x7f88d0003f80 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e80c81b0 0x7f88e80b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e80c81b0 0x7f88e80b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41502/0 (socket says 172.21.15.153:41502) 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 -- 172.21.15.153:0/4226938889 learned_addr learned my addr 172.21.15.153:0/4226938889 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80c7d60 0x7f88e80b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e659c700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d0003f80 0x7f88e814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 -- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d0003f80 msgr2=0x7f88e814fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d0003f80 0x7f88e814fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 -- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80c7d60 msgr2=0x7f88e80b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80c7d60 0x7f88e80b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 -- 172.21.15.153:0/4226938889 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e806cef0 con 0x7f88e80c81b0 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e659c700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d0003f80 0x7f88e814fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e80c81b0 0x7f88e80b0b40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f88e80b2390 tx=0x7f88e0011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88d6ffd700 1 -- 172.21.15.153:0/4226938889 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e000e040 con 0x7f88e80c81b0 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88d6ffd700 1 -- 172.21.15.153:0/4226938889 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e000f040 con 0x7f88e80c81b0 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e80664c0 con 0x7f88e80c81b0 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88d6ffd700 1 -- 172.21.15.153:0/4226938889 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e000fd50 con 0x7f88e80c81b0 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.636+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e815ac80 con 0x7f88e80c81b0 2023-11-28T13:02:35.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.638+0000 7f88d6ffd700 1 -- 172.21.15.153:0/4226938889 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88e0022460 con 0x7f88e80c81b0 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.638+0000 7f88d6ffd700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88c80682f0 0x7f88c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.638+0000 7f88d6ffd700 1 -- 172.21.15.153:0/4226938889 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88e0086240 con 0x7f88e80c81b0 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.638+0000 7f88e659c700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88c80682f0 0x7f88c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.638+0000 7f88e659c700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88c80682f0 0x7f88c806a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009b50 tx=0x7f88dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.640+0000 7f88f8a3db80 1 Processor -- start 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.640+0000 7f88f8a3db80 1 -- start start 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c738860 0x56249c70ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c738c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c739170 0x56249c7547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c5f8c10 con 0x56249c736180 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c6673a0 con 0x56249c739170 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56249c6608d0 con 0x56249c738860 2023-11-28T13:02:35.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c738c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c738c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41508/0 (socket says 172.21.15.153:41508) 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 -- 172.21.15.153:0/148826902 learned_addr learned my addr 172.21.15.153:0/148826902 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e659c700 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c738860 0x56249c70ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c739170 0x56249c7547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 -- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c738860 msgr2=0x56249c70ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c738860 0x56249c70ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 -- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c739170 msgr2=0x56249c7547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c739170 0x56249c7547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 -- 172.21.15.153:0/148826902 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56249c4e2630 con 0x56249c736180 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.641+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c738c30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f88d8008fd0 tx=0x7f88d800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912a127a7da16e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88c7fff700 1 -- 172.21.15.153:0/148826902 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8007b80 con 0x56249c736180 2023-11-28T13:02:35.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88c7fff700 1 -- 172.21.15.153:0/148826902 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8001430 con 0x56249c736180 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88c7fff700 1 -- 172.21.15.153:0/148826902 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80017e0 con 0x56249c736180 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 -- 172.21.15.153:0/148826902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 msgr2=0x56249c738c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c738c30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f88d8008fd0 tx=0x7f88d800f000 comp rx=0 tx=0).stop 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 -- 172.21.15.153:0/148826902 shutdown_connections 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c739170 0x56249c7547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c738860 0x56249c70ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/148826902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c738c30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 -- 172.21.15.153:0/148826902 >> 172.21.15.153:0/148826902 conn(0x56249c70bdb0 msgr2=0x56249c70c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 -- 172.21.15.153:0/148826902 shutdown_connections 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.642+0000 7f88f8a3db80 1 -- 172.21.15.153:0/148826902 wait complete. 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.643+0000 7f88f8a3db80 1 -- start start 2023-11-28T13:02:35.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.643+0000 7f88f8a3db80 1 Processor -- start 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c73c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c729d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c4e4920 con 0x56249c738860 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c72a710 con 0x56249c736180 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56249c73a880 con 0x56249c739170 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88f8a3db80 10 client.0 _ll_register_callbacks cb 0x56249c506290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c729d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c729d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41514/0 (socket says 172.21.15.153:41514) 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 -- 172.21.15.153:0/1647797394 learned_addr learned my addr 172.21.15.153:0/1647797394 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e659c700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c73c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 -- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 msgr2=0x56249c708340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 -- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 msgr2=0x56249c73c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c73c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e5d9b700 1 -- 172.21.15.153:0/1647797394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d00035b0 con 0x56249c738860 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.644+0000 7f88e659c700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c73c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c729d00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f88d000ba40 tx=0x7f88d8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47170dfe58d454f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88c57fa700 1 -- 172.21.15.153:0/1647797394 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800c9b0 con 0x56249c738860 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88c57fa700 1 -- 172.21.15.153:0/1647797394 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88d8033c20 con 0x56249c738860 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56249c62e190 con 0x56249c738860 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88c57fa700 1 -- 172.21.15.153:0/1647797394 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88d8033de0 con 0x56249c738860 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 10 client.5781 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56249c48ede0 con 0x56249c738860 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88c57fa700 1 -- 172.21.15.153:0/1647797394 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56249c62e190 con 0x56249c738860 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88c57fa700 1 -- 172.21.15.153:0/1647797394 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f88d8004870 con 0x56249c738860 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 10 client.5781 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 10 client.5781 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 20 client.5781 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7287e0 0x56249c730560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 4 client.5781 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:02:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56249c4e4f60 con 0x56249c7287e0 2023-11-28T13:02:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.645+0000 7f88e659c700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7287e0 0x56249c730560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.646+0000 7f88c57fa700 10 client.5781 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.646+0000 7f88e659c700 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7287e0 0x56249c730560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89d02f295e7abbe server_cookie=47ff673e01a9e25f in_seq=0 out_seq=0 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.646+0000 7f88c57fa700 1 -- 172.21.15.153:0/1647797394 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x56249c4e4f60 con 0x56249c7287e0 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.646+0000 7f88c57fa700 10 client.5781 handle_command_reply: tid=0 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.646+0000 7f88f8a3db80 1 client.5781 shutdown 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 msgr2=0x56249c729d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c729d00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f88d000ba40 tx=0x7f88d8001a00 comp rx=0 tx=0).stop 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 shutdown_connections 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7287e0 0x56249c730560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c73c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/1647797394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c729d00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88c57fa700 0 client.5781 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:35.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 >> 172.21.15.153:0/1647797394 conn(0x56249c70bdb0 msgr2=0x56249c70d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 shutdown_connections 2023-11-28T13:02:35.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 1 -- 172.21.15.153:0/1647797394 wait complete. 2023-11-28T13:02:35.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.647+0000 7f88f8a3db80 20 client.5781 trim_cache size 0 max 16384 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.663+0000 7f88f8a3db80 1 Processor -- start 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.663+0000 7f88f8a3db80 1 -- start start 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c756b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c739eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c70d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c73b6d0 con 0x56249c738860 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c737690 con 0x56249c736180 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56249c7318a0 con 0x56249c739170 2023-11-28T13:02:35.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c739eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c739eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41522/0 (socket says 172.21.15.153:41522) 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 -- 172.21.15.153:0/2779478365 learned_addr learned my addr 172.21.15.153:0/2779478365 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c70d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e659c700 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c756b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 -- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 msgr2=0x56249c70d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c70d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 -- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 msgr2=0x56249c756b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c756b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.664+0000 7f88e5d9b700 1 -- 172.21.15.153:0/2779478365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56249c627190 con 0x56249c738860 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c739eb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000fb0 tx=0x7f88d802b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7fc9a05f20cf3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88d4ff9700 1 -- 172.21.15.153:0/2779478365 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8004d30 con 0x56249c738860 2023-11-28T13:02:35.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88d4ff9700 1 -- 172.21.15.153:0/2779478365 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8001640 con 0x56249c738860 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88d4ff9700 1 -- 172.21.15.153:0/2779478365 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8007310 con 0x56249c738860 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 -- 172.21.15.153:0/2779478365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 msgr2=0x56249c739eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c739eb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000fb0 tx=0x7f88d802b050 comp rx=0 tx=0).stop 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 -- 172.21.15.153:0/2779478365 shutdown_connections 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c70d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c736180 0x56249c756b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/2779478365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c738860 0x56249c739eb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 -- 172.21.15.153:0/2779478365 >> 172.21.15.153:0/2779478365 conn(0x56249c70bdb0 msgr2=0x56249c728c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 -- 172.21.15.153:0/2779478365 shutdown_connections 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.665+0000 7f88f8a3db80 1 -- 172.21.15.153:0/2779478365 wait complete. 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.666+0000 7f88f8a3db80 1 -- start start 2023-11-28T13:02:35.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.666+0000 7f88f8a3db80 1 Processor -- start 2023-11-28T13:02:35.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c70cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c738860 0x56249c7b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c7096d0 con 0x56249c736180 2023-11-28T13:02:35.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56249c76c7c0 con 0x56249c738860 2023-11-28T13:02:35.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56249c76ca00 con 0x56249c739170 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 10 client.0 _ll_register_callbacks cb 0x56249c506290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c70cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c70cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41526/0 (socket says 172.21.15.153:41526) 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 -- 172.21.15.153:0/399151354 learned_addr learned my addr 172.21.15.153:0/399151354 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c738860 0x56249c7b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e6d9d700 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 -- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 msgr2=0x56249c708b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 -- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c738860 msgr2=0x56249c7b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c738860 0x56249c7b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 -- 172.21.15.153:0/399151354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56249c48ede0 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88e659c700 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c70cae0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f88dc018040 tx=0x7f88dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be299c884326bb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88c57fa700 1 -- 172.21.15.153:0/399151354 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc02a040 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88c57fa700 1 -- 172.21.15.153:0/399151354 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f88dc02b420 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.667+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56249c61e1b0 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88c57fa700 1 -- 172.21.15.153:0/399151354 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88dc016c00 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 10 client.5787 fetch_fsmap learned FSMap version 23 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56249c4b5250 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88c57fa700 1 -- 172.21.15.153:0/399151354 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56249c61e1b0 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88c57fa700 1 -- 172.21.15.153:0/399151354 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 23) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f88dc02b660 con 0x56249c736180 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 10 client.5787 fetch_fsmap finished waiting for FSMap version 23 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 10 client.5787 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2023-11-28T13:02:35.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 20 client.5787 populate_metadata read hostname 'smithi153' 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7b28b0 0x56249c70d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 4 client.5787 mds_command: new command op to 5547 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 --> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56249c6081c0 con 0x56249c7b28b0 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.668+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7b28b0 0x56249c70d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88c57fa700 10 client.5787 ms_handle_connect on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88e5d9b700 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7b28b0 0x56249c70d280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e5376a2d9e9c37 server_cookie=cc1cd451c0fe75ea in_seq=0 out_seq=0 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88c57fa700 1 -- 172.21.15.153:0/399151354 <== mds.0 v2:172.21.15.177:6832/3478129537 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56249c6081c0 con 0x56249c7b28b0 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88c57fa700 10 client.5787 handle_command_reply: tid=0 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 client.5787 shutdown 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 msgr2=0x56249c70cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c70cae0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f88dc018040 tx=0x7f88dc0047c0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 shutdown_connections 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x56249c736180 0x56249c70cae0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:6832/3478129537,v1:172.21.15.177:6835/3478129537] conn(0x56249c7b28b0 0x56249c70d280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56249c739170 0x56249c708b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 --2- 172.21.15.153:0/399151354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56249c738860 0x56249c7b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88c57fa700 0 client.5787 ms_handle_reset on v2:172.21.15.177:6832/3478129537 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 >> 172.21.15.153:0/399151354 conn(0x56249c70bdb0 msgr2=0x56249c73a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 shutdown_connections 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 1 -- 172.21.15.153:0/399151354 wait complete. 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.669+0000 7f88f8a3db80 20 client.5787 trim_cache size 0 max 16384 2023-11-28T13:02:35.677 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:02:35.678 INFO:teuthology.orchestra.run.smithi153.stdout: "status": "no active scrubs running", 2023-11-28T13:02:35.678 INFO:teuthology.orchestra.run.smithi153.stdout: "scrubs": {} 2023-11-28T13:02:35.678 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88c80682f0 msgr2=0x7f88c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88c80682f0 0x7f88c806a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009b50 tx=0x7f88dc008040 comp rx=0 tx=0).stop 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e80c81b0 msgr2=0x7f88e80b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e80c81b0 0x7f88e80b0b40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f88e80b2390 tx=0x7f88e0011f80 comp rx=0 tx=0).stop 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 shutdown_connections 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88c80682f0 0x7f88c806a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88e80c81b0 0x7f88e80b0b40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d0003f80 0x7f88e814fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 --2- 172.21.15.153:0/4226938889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80c7d60 0x7f88e80b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 >> 172.21.15.153:0/4226938889 conn(0x7f88e80b12b0 msgr2=0x7f88e80af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 shutdown_connections 2023-11-28T13:02:35.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.670+0000 7f88ec8b4700 1 -- 172.21.15.153:0/4226938889 wait complete. 2023-11-28T13:02:35.682 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T13:02:35.783 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:35.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.903+0000 7ff0de573700 1 Processor -- start 2023-11-28T13:02:35.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.903+0000 7ff0de573700 1 -- start start 2023-11-28T13:02:35.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0de573700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0de573700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0d8160540 0x7ff0d81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0de573700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d8071950 con 0x7ff0d806dee0 2023-11-28T13:02:35.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0de573700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d815d060 con 0x7ff0d8160540 2023-11-28T13:02:35.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0de573700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d815d340 con 0x7ff0d8071040 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0d8160540 0x7ff0d81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0d8160540 0x7ff0d81609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41538/0 (socket says 172.21.15.153:41538) 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 -- 172.21.15.153:0/2268426539 learned_addr learned my addr 172.21.15.153:0/2268426539 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0d7fff700 1 --2- 172.21.15.153:0/2268426539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 -- 172.21.15.153:0/2268426539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 msgr2=0x7ff0d8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 --2- 172.21.15.153:0/2268426539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 -- 172.21.15.153:0/2268426539 >> v1:172.21.15.177:6789/0 conn(0x7ff0d806dee0 legacy=0x7ff0d806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 -- 172.21.15.153:0/2268426539 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d815d620 con 0x7ff0d8160540 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0dcb10700 1 --2- 172.21.15.153:0/2268426539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d81609c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009e50 tx=0x7ff0d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d567230f7a74f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.904+0000 7ff0d6ffd700 1 -- 172.21.15.153:0/2268426539 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d000e070 con 0x7ff0d8160540 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0d6ffd700 1 -- 172.21.15.153:0/2268426539 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d0003a20 con 0x7ff0d8160540 2023-11-28T13:02:35.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0d6ffd700 1 -- 172.21.15.153:0/2268426539 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0003dd0 con 0x7ff0d8160540 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 -- 172.21.15.153:0/2268426539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 msgr2=0x7ff0d81609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 --2- 172.21.15.153:0/2268426539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d81609c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009e50 tx=0x7ff0d000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 -- 172.21.15.153:0/2268426539 shutdown_connections 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 --2- 172.21.15.153:0/2268426539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d81609c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 --2- 172.21.15.153:0/2268426539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 -- 172.21.15.153:0/2268426539 >> 172.21.15.153:0/2268426539 conn(0x7ff0d80d0000 msgr2=0x7ff0d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 -- 172.21.15.153:0/2268426539 shutdown_connections 2023-11-28T13:02:35.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.905+0000 7ff0de573700 1 -- 172.21.15.153:0/2268426539 wait complete. 2023-11-28T13:02:35.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.907+0000 7ff0de573700 1 Processor -- start 2023-11-28T13:02:35.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.907+0000 7ff0de573700 1 -- start start 2023-11-28T13:02:35.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0de573700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d806dee0 0x7ff0d80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0de573700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d80c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0de573700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d809b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0de573700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d815bec0 con 0x7ff0d8160540 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0de573700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80b33b0 con 0x7ff0d806dee0 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0de573700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80b3690 con 0x7ff0d8071040 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d809b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d809b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41544/0 (socket says 172.21.15.153:41544) 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 -- 172.21.15.153:0/3313093267 learned_addr learned my addr 172.21.15.153:0/3313093267 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0d77fe700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d80c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0d7fff700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d806dee0 0x7ff0d80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 -- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 msgr2=0x7ff0d80c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d80c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 -- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d806dee0 msgr2=0x7ff0d80b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d806dee0 0x7ff0d80b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 -- 172.21.15.153:0/3313093267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0009b30 con 0x7ff0d8160540 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0dcb10700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d809b4f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff0d806c7b0 tx=0x7ff0d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0d4ff9700 1 -- 172.21.15.153:0/3313093267 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00092e0 con 0x7ff0d8160540 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0d4ff9700 1 -- 172.21.15.153:0/3313093267 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d0004240 con 0x7ff0d8160540 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.908+0000 7ff0d4ff9700 1 -- 172.21.15.153:0/3313093267 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0004610 con 0x7ff0d8160540 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.909+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d815ccd0 con 0x7ff0d8160540 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.909+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d815d900 con 0x7ff0d8160540 2023-11-28T13:02:35.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.909+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d806a270 con 0x7ff0d8160540 2023-11-28T13:02:35.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.910+0000 7ff0d4ff9700 1 -- 172.21.15.153:0/3313093267 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff0d002c070 con 0x7ff0d8160540 2023-11-28T13:02:35.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.910+0000 7ff0d4ff9700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0c40682f0 0x7ff0c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.910+0000 7ff0d7fff700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0c40682f0 0x7ff0c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.910+0000 7ff0d4ff9700 1 -- 172.21.15.153:0/3313093267 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0d0086f20 con 0x7ff0d8160540 2023-11-28T13:02:35.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.911+0000 7ff0d7fff700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0c40682f0 0x7ff0c406a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80cdb20 tx=0x7ff0cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:35.915+0000 7ff0d4ff9700 1 -- 172.21.15.153:0/3313093267 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0d00b85e0 con 0x7ff0d8160540 2023-11-28T13:02:36.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.138+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0d809dea0 con 0x7ff0d8160540 2023-11-28T13:02:36.139 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.138+0000 7ff0d4ff9700 1 -- 172.21.15.153:0/3313093267 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7ff0d0020090 con 0x7ff0d8160540 2023-11-28T13:02:36.139 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:36.139 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:36.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0c40682f0 msgr2=0x7ff0c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0c40682f0 0x7ff0c406a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80cdb20 tx=0x7ff0cc008040 comp rx=0 tx=0).stop 2023-11-28T13:02:36.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 msgr2=0x7ff0d809b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d809b4f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff0d806c7b0 tx=0x7ff0d0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 shutdown_connections 2023-11-28T13:02:36.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0c40682f0 0x7ff0c406a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0d8160540 0x7ff0d809b4f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0d806dee0 0x7ff0d80b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 --2- 172.21.15.153:0/3313093267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0d8071040 0x7ff0d80c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 >> 172.21.15.153:0/3313093267 conn(0x7ff0d80d0000 msgr2=0x7ff0d806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 shutdown_connections 2023-11-28T13:02:36.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.140+0000 7ff0de573700 1 -- 172.21.15.153:0/3313093267 wait complete. 2023-11-28T13:02:36.143 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:36.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-11-28T13:02:36.314 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2023-11-28T13:02:36.314 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:02:36.315 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:02:36.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.358+0000 7f81405e2080 1 Processor -- start 2023-11-28T13:02:36.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81405e2080 1 -- start start 2023-11-28T13:02:36.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81405e2080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b76e9d0 0x55916b726f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81405e2080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b7fc450 0x55916b7542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81405e2080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b800de0 0x55916b723c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81405e2080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916b65efe0 con 0x55916b76e9d0 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81405e2080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916b76a420 con 0x55916b7fc450 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81405e2080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55916b888560 con 0x55916b800de0 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81366e5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b76e9d0 0x55916b726f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81366e5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b76e9d0 0x55916b726f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41552/0 (socket says 172.21.15.153:41552) 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.359+0000 7f81366e5700 1 -- 172.21.15.153:0/3714979307 learned_addr learned my addr 172.21.15.153:0/3714979307 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f8135ee4700 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b7fc450 0x55916b7542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f8136ee6700 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b800de0 0x55916b723c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81366e5700 1 -- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b800de0 msgr2=0x55916b723c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81366e5700 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b800de0 0x55916b723c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81366e5700 1 -- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b7fc450 msgr2=0x55916b7542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81366e5700 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b7fc450 0x55916b7542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81366e5700 1 -- 172.21.15.153:0/3714979307 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c005d20 con 0x55916b76e9d0 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81366e5700 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b76e9d0 0x55916b726f20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f812c0099a0 tx=0x7f812c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520378b1fbc5542d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f8134ee2700 1 -- 172.21.15.153:0/3714979307 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c00f040 con 0x55916b76e9d0 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f8134ee2700 1 -- 172.21.15.153:0/3714979307 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c008ca0 con 0x55916b76e9d0 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f8134ee2700 1 -- 172.21.15.153:0/3714979307 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c013620 con 0x55916b76e9d0 2023-11-28T13:02:36.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 -- 172.21.15.153:0/3714979307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b76e9d0 msgr2=0x55916b726f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b76e9d0 0x55916b726f20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f812c0099a0 tx=0x7f812c005a00 comp rx=0 tx=0).stop 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 -- 172.21.15.153:0/3714979307 shutdown_connections 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b76e9d0 0x55916b726f20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b800de0 0x55916b723c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 --2- 172.21.15.153:0/3714979307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b7fc450 0x55916b7542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 -- 172.21.15.153:0/3714979307 >> 172.21.15.153:0/3714979307 conn(0x55916b750b60 msgr2=0x55916b765100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.360+0000 7f81405e2080 1 -- 172.21.15.153:0/3714979307 shutdown_connections 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.361+0000 7f81405e2080 1 -- 172.21.15.153:0/3714979307 wait complete. 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.361+0000 7f81405e2080 1 Processor -- start 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.361+0000 7f81405e2080 1 -- start start 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81405e2080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b76e9d0 0x55916b7f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81405e2080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b7fc450 0x55916b752220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81405e2080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b800de0 0x55916b752760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81405e2080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916b765910 con 0x55916b7fc450 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81405e2080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916b7f1e30 con 0x55916b800de0 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81405e2080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55916b765100 con 0x55916b76e9d0 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b7fc450 0x55916b752220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b7fc450 0x55916b752220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41560/0 (socket says 172.21.15.153:41560) 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 -- 172.21.15.153:0/2461773024 learned_addr learned my addr 172.21.15.153:0/2461773024 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81366e5700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b76e9d0 0x55916b7f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8136ee6700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b800de0 0x55916b752760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 -- 172.21.15.153:0/2461773024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b76e9d0 msgr2=0x55916b7f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55916b76e9d0 0x55916b7f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 -- 172.21.15.153:0/2461773024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b800de0 msgr2=0x55916b752760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55916b800de0 0x55916b752760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 -- 172.21.15.153:0/2461773024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c009600 con 0x55916b7fc450 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f8135ee4700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b7fc450 0x55916b752220 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8128001ed0 tx=0x7f812800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.362+0000 7f81277fe700 1 -- 172.21.15.153:0/2461773024 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800bcb0 con 0x55916b7fc450 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.363+0000 7f81277fe700 1 -- 172.21.15.153:0/2461773024 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812800be30 con 0x55916b7fc450 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.363+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55916b66a830 con 0x55916b7fc450 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.363+0000 7f81277fe700 1 -- 172.21.15.153:0/2461773024 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800f690 con 0x55916b7fc450 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.363+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813000a100 con 0x55916b7fc450 2023-11-28T13:02:36.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.364+0000 7f81277fe700 1 -- 172.21.15.153:0/2461773024 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f812800f7f0 con 0x55916b7fc450 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.364+0000 7f81277fe700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8118066c10 0x7f81180690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.364+0000 7f81277fe700 1 -- 172.21.15.153:0/2461773024 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8128084de0 con 0x55916b7fc450 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.364+0000 7f81366e5700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8118066c10 0x7f81180690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.364+0000 7f81405e2080 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x55916b7ff2b0 0x55916b768860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.364+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 --> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v8 -- 0x55916b754d00 con 0x55916b7ff2b0 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f8136ee6700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x55916b7ff2b0 0x55916b768860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81366e5700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8118066c10 0x7f81180690c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f812c005a40 tx=0x7f812c012000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f8136ee6700 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x55916b7ff2b0 0x55916b768860 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f8136ee6700 1 -- 172.21.15.153:0/2461773024 <== osd.1 v2:172.21.15.153:6816/103564 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f813000f610 con 0x55916b7ff2b0 2023-11-28T13:02:36.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x55916b7ff2b0 msgr2=0x55916b768860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x55916b7ff2b0 0x55916b768860 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8118066c10 msgr2=0x7f81180690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8118066c10 0x7f81180690c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f812c005a40 tx=0x7f812c012000 comp rx=0 tx=0).stop 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b7fc450 msgr2=0x55916b752220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 --2- 172.21.15.153:0/2461773024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55916b7fc450 0x55916b752220 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8128001ed0 tx=0x7f812800df80 comp rx=0 tx=0).stop 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f8136ee6700 1 -- 172.21.15.153:0/2461773024 reap_dead start 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 shutdown_connections 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 >> 172.21.15.153:0/2461773024 conn(0x55916b750b60 msgr2=0x55916b7f20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 shutdown_connections 2023-11-28T13:02:36.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.365+0000 7f81405e2080 1 -- 172.21.15.153:0/2461773024 wait complete. 2023-11-28T13:02:36.374 INFO:teuthology.orchestra.run.smithi153.stdout:Yalphaparent_a 2023-11-28T13:02:36.375 DEBUG:teuthology.orchestra.run.smithi153:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:02:36.431 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:02:36.431 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 1099511627777, 2023-11-28T13:02:36.431 INFO:teuthology.orchestra.run.smithi153.stdout: "ancestors": [ 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi153.stdout: "dirino": 1099511627776, 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi153.stdout: "dname": "alpha", 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi153.stdout: "version": 4 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi153.stdout: "dirino": 1, 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout: "dname": "parent_a", 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout: "version": 7 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout: "pool": 8, 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout: "old_pools": [] 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:02:36.433 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:36.435 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:02:36.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:36.521 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:36.559 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:02:36.560 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:02:36.560 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:36.578 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:02:36.578 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:36.636 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:36.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.794+0000 7f09095cb700 1 Processor -- start 2023-11-28T13:02:36.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.794+0000 7f09095cb700 1 -- start start 2023-11-28T13:02:36.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09095cb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09095cb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f09040cd7c0 0x7f09040cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09095cb700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040ce160 con 0x7f0904161340 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09095cb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040b5800 con 0x7f09040cd7c0 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09095cb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040d1950 con 0x7f090415c9b0 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09037fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f09040cd7c0 0x7f09040cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09037fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f09040cd7c0 0x7f09040cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41572/0 (socket says 172.21.15.153:41572) 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09027fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0904161340 0x7f09040d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59734/0 (socket says 172.21.15.153:59734) 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09037fe700 1 -- 172.21.15.153:0/1957781102 learned_addr learned my addr 172.21.15.153:0/1957781102 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:36.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f0902ffd700 1 --2- 172.21.15.153:0/1957781102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09037fe700 1 -- 172.21.15.153:0/1957781102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090415c9b0 msgr2=0x7f09040b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:36.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09037fe700 1 --2- 172.21.15.153:0/1957781102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09037fe700 1 -- 172.21.15.153:0/1957781102 >> v1:172.21.15.153:6789/0 conn(0x7f0904161340 legacy=0x7f09040d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f0901ffb700 1 -- 172.21.15.153:0/1957781102 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987470955 0 0) 0x7f09040ce160 con 0x7f0904161340 2023-11-28T13:02:36.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.795+0000 7f09037fe700 1 -- 172.21.15.153:0/1957781102 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040d24f0 con 0x7f09040cd7c0 2023-11-28T13:02:36.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09037fe700 1 --2- 172.21.15.153:0/1957781102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09040cd7c0 0x7f09040cdc20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009e50 tx=0x7f08fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c2407b7f27921b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f0901ffb700 1 -- 172.21.15.153:0/1957781102 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00e070 con 0x7f09040cd7c0 2023-11-28T13:02:36.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f0901ffb700 1 -- 172.21.15.153:0/1957781102 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc00b9c0 con 0x7f09040cd7c0 2023-11-28T13:02:36.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f0901ffb700 1 -- 172.21.15.153:0/1957781102 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00bd70 con 0x7f09040cd7c0 2023-11-28T13:02:36.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 -- 172.21.15.153:0/1957781102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09040cd7c0 msgr2=0x7f09040cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 --2- 172.21.15.153:0/1957781102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09040cd7c0 0x7f09040cdc20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009e50 tx=0x7f08fc00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:36.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 -- 172.21.15.153:0/1957781102 shutdown_connections 2023-11-28T13:02:36.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 --2- 172.21.15.153:0/1957781102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f090415c9b0 0x7f09040b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 --2- 172.21.15.153:0/1957781102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f09040cd7c0 0x7f09040cdc20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 -- 172.21.15.153:0/1957781102 >> 172.21.15.153:0/1957781102 conn(0x7f0904072600 msgr2=0x7f09040cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 -- 172.21.15.153:0/1957781102 shutdown_connections 2023-11-28T13:02:36.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.796+0000 7f09095cb700 1 -- 172.21.15.153:0/1957781102 wait complete. 2023-11-28T13:02:36.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.798+0000 7f09095cb700 1 Processor -- start 2023-11-28T13:02:36.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.798+0000 7f09095cb700 1 -- start start 2023-11-28T13:02:36.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09095cb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09040cd7c0 0x7f0904160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09095cb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f090415c9b0 0x7f09040b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09095cb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0904161340 0x7f090415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09095cb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040b8d50 con 0x7f090415c9b0 2023-11-28T13:02:36.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09095cb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c9a90 con 0x7f0904161340 2023-11-28T13:02:36.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09095cb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c91f0 con 0x7f09040cd7c0 2023-11-28T13:02:36.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f090415c9b0 0x7f09040b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f090415c9b0 0x7f09040b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41576/0 (socket says 172.21.15.153:41576) 2023-11-28T13:02:36.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 -- 172.21.15.153:0/160674640 learned_addr learned my addr 172.21.15.153:0/160674640 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:36.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f0902ffd700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09040cd7c0 0x7f0904160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09037fe700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0904161340 0x7f090415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 -- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09040cd7c0 msgr2=0x7f0904160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09040cd7c0 0x7f0904160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 -- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0904161340 msgr2=0x7f090415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0904161340 0x7f090415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.799+0000 7f09027fc700 1 -- 172.21.15.153:0/160674640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc009b30 con 0x7f090415c9b0 2023-11-28T13:02:36.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.800+0000 7f09027fc700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f090415c9b0 0x7f09040b8810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c550 tx=0x7f08f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.800+0000 7f08eb7fe700 1 -- 172.21.15.153:0/160674640 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8004c80 con 0x7f090415c9b0 2023-11-28T13:02:36.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.800+0000 7f08eb7fe700 1 -- 172.21.15.153:0/160674640 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8004e00 con 0x7f090415c9b0 2023-11-28T13:02:36.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.800+0000 7f08eb7fe700 1 -- 172.21.15.153:0/160674640 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8007310 con 0x7f090415c9b0 2023-11-28T13:02:36.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.800+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09040d1300 con 0x7f090415c9b0 2023-11-28T13:02:36.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.800+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040b5520 con 0x7f090415c9b0 2023-11-28T13:02:36.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.801+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040d15e0 con 0x7f090415c9b0 2023-11-28T13:02:36.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.801+0000 7f08eb7fe700 1 -- 172.21.15.153:0/160674640 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f08f801a020 con 0x7f090415c9b0 2023-11-28T13:02:36.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.801+0000 7f08eb7fe700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08f00682f0 0x7f08f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.801+0000 7f08eb7fe700 1 -- 172.21.15.153:0/160674640 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08f80856b0 con 0x7f090415c9b0 2023-11-28T13:02:36.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.801+0000 7f0902ffd700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08f00682f0 0x7f08f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.802+0000 7f0902ffd700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08f00682f0 0x7f08f006a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006b20 tx=0x7f08f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.806+0000 7f08eb7fe700 1 -- 172.21.15.153:0/160674640 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f08f8051f10 con 0x7f090415c9b0 2023-11-28T13:02:36.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.992+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09041e6510 con 0x7f090415c9b0 2023-11-28T13:02:36.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.994+0000 7f08eb7fe700 1 -- 172.21.15.153:0/160674640 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f09041e6510 con 0x7f090415c9b0 2023-11-28T13:02:36.996 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:36.996 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":51,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:57.932997+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:54.715348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:55.897576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:37.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08f00682f0 msgr2=0x7f08f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08f00682f0 0x7f08f006a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006b20 tx=0x7f08f4008040 comp rx=0 tx=0).stop 2023-11-28T13:02:37.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f090415c9b0 msgr2=0x7f09040b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f090415c9b0 0x7f09040b8810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c550 tx=0x7f08f800c860 comp rx=0 tx=0).stop 2023-11-28T13:02:37.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 shutdown_connections 2023-11-28T13:02:37.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0904161340 0x7f090415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09040cd7c0 0x7f0904160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f08f00682f0 0x7f08f006a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 --2- 172.21.15.153:0/160674640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f090415c9b0 0x7f09040b8810 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 >> 172.21.15.153:0/160674640 conn(0x7f0904072600 msgr2=0x7f09040b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 shutdown_connections 2023-11-28T13:02:37.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:36.996+0000 7f09095cb700 1 -- 172.21.15.153:0/160674640 wait complete. 2023-11-28T13:02:37.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:01:54.715348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:01:55.897576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:02:37.013 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:02:37.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.134+0000 7efe5311c700 1 Processor -- start 2023-11-28T13:02:37.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.134+0000 7efe5311c700 1 -- start start 2023-11-28T13:02:37.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.135+0000 7efe5311c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efe4c071040 0x7efe4c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.135+0000 7efe5311c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06dee0 0x7efe4c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.135+0000 7efe5311c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c071950 con 0x7efe4c06cdc0 2023-11-28T13:02:37.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.135+0000 7efe5311c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4c161550 con 0x7efe4c071040 2023-11-28T13:02:37.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.135+0000 7efe5311c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c15d060 con 0x7efe4c06dee0 2023-11-28T13:02:37.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.135+0000 7efe50eb8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efe4c071040 0x7efe4c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe50eb8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efe4c071040 0x7efe4c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41582/0 (socket says 172.21.15.153:41582) 2023-11-28T13:02:37.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe50eb8700 1 -- 172.21.15.153:0/3166062791 learned_addr learned my addr 172.21.15.153:0/3166062791 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:37.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe4bfff700 1 --2- 172.21.15.153:0/3166062791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06dee0 0x7efe4c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe50eb8700 1 -- 172.21.15.153:0/3166062791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06dee0 msgr2=0x7efe4c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe50eb8700 1 --2- 172.21.15.153:0/3166062791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06dee0 0x7efe4c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe50eb8700 1 -- 172.21.15.153:0/3166062791 >> v1:172.21.15.177:6789/0 conn(0x7efe4c06cdc0 legacy=0x7efe4c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe50eb8700 1 -- 172.21.15.153:0/3166062791 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c15d340 con 0x7efe4c071040 2023-11-28T13:02:37.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe50eb8700 1 --2- 172.21.15.153:0/3166062791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c071410 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efe400098f0 tx=0x7efe40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918d8ee1030ad0e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe4b7fe700 1 -- 172.21.15.153:0/3166062791 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40011830 con 0x7efe4c071040 2023-11-28T13:02:37.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe4b7fe700 1 -- 172.21.15.153:0/3166062791 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe40011990 con 0x7efe4c071040 2023-11-28T13:02:37.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.136+0000 7efe4b7fe700 1 -- 172.21.15.153:0/3166062791 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40011d20 con 0x7efe4c071040 2023-11-28T13:02:37.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 -- 172.21.15.153:0/3166062791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 msgr2=0x7efe4c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 --2- 172.21.15.153:0/3166062791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c071410 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efe400098f0 tx=0x7efe40009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:37.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 -- 172.21.15.153:0/3166062791 shutdown_connections 2023-11-28T13:02:37.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 --2- 172.21.15.153:0/3166062791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c071410 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 --2- 172.21.15.153:0/3166062791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06dee0 0x7efe4c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 -- 172.21.15.153:0/3166062791 >> 172.21.15.153:0/3166062791 conn(0x7efe4c0d0000 msgr2=0x7efe4c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 -- 172.21.15.153:0/3166062791 shutdown_connections 2023-11-28T13:02:37.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.137+0000 7efe5311c700 1 -- 172.21.15.153:0/3166062791 wait complete. 2023-11-28T13:02:37.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.138+0000 7efe5311c700 1 Processor -- start 2023-11-28T13:02:37.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.138+0000 7efe5311c700 1 -- start start 2023-11-28T13:02:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe5311c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06cdc0 0x7efe4c0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe5311c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe4c06dee0 0x7efe4c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe5311c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe5311c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0b4ce0 con 0x7efe4c071040 2023-11-28T13:02:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe5311c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0ae500 con 0x7efe4c06dee0 2023-11-28T13:02:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe5311c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0ad880 con 0x7efe4c06cdc0 2023-11-28T13:02:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c0b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41592/0 (socket says 172.21.15.153:41592) 2023-11-28T13:02:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 -- 172.21.15.153:0/3528366183 learned_addr learned my addr 172.21.15.153:0/3528366183 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe50eb8700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06cdc0 0x7efe4c0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe4bfff700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe4c06dee0 0x7efe4c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 -- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06cdc0 msgr2=0x7efe4c0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06cdc0 0x7efe4c0ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 -- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe4c06dee0 msgr2=0x7efe4c0ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe4c06dee0 0x7efe4c0ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.139+0000 7efe516b9700 1 -- 172.21.15.153:0/3528366183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe40009550 con 0x7efe4c071040 2023-11-28T13:02:37.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.140+0000 7efe516b9700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c0b3640 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efe4400cb50 tx=0x7efe4400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.140+0000 7efe497fa700 1 -- 172.21.15.153:0/3528366183 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44003c00 con 0x7efe4c071040 2023-11-28T13:02:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.140+0000 7efe497fa700 1 -- 172.21.15.153:0/3528366183 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe44003d60 con 0x7efe4c071040 2023-11-28T13:02:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.140+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c15d900 con 0x7efe4c071040 2023-11-28T13:02:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.140+0000 7efe497fa700 1 -- 172.21.15.153:0/3528366183 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400aa20 con 0x7efe4c071040 2023-11-28T13:02:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.140+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c15d620 con 0x7efe4c071040 2023-11-28T13:02:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.141+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c06e870 con 0x7efe4c071040 2023-11-28T13:02:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.141+0000 7efe497fa700 1 -- 172.21.15.153:0/3528366183 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efe44010050 con 0x7efe4c071040 2023-11-28T13:02:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.141+0000 7efe497fa700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efe300682f0 0x7efe3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.142+0000 7efe497fa700 1 -- 172.21.15.153:0/3528366183 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe44085b20 con 0x7efe4c071040 2023-11-28T13:02:37.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.142+0000 7efe50eb8700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efe300682f0 0x7efe3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.142+0000 7efe50eb8700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efe300682f0 0x7efe3006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efe40000c00 tx=0x7efe40008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.146+0000 7efe497fa700 1 -- 172.21.15.153:0/3528366183 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efe44052380 con 0x7efe4c071040 2023-11-28T13:02:37.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.326+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe4c15b900 con 0x7efe4c071040 2023-11-28T13:02:37.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.326+0000 7efe497fa700 1 -- 172.21.15.153:0/3528366183 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe440559a0 con 0x7efe4c071040 2023-11-28T13:02:37.327 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:02:37.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efe300682f0 msgr2=0x7efe3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efe300682f0 0x7efe3006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efe40000c00 tx=0x7efe40008030 comp rx=0 tx=0).stop 2023-11-28T13:02:37.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 msgr2=0x7efe4c0b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c0b3640 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efe4400cb50 tx=0x7efe4400ce60 comp rx=0 tx=0).stop 2023-11-28T13:02:37.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 shutdown_connections 2023-11-28T13:02:37.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efe300682f0 0x7efe3006a7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe4c071040 0x7efe4c0b3640 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe4c06cdc0 0x7efe4c0ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 --2- 172.21.15.153:0/3528366183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe4c06dee0 0x7efe4c0ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 >> 172.21.15.153:0/3528366183 conn(0x7efe4c0d0000 msgr2=0x7efe4c06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 shutdown_connections 2023-11-28T13:02:37.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.328+0000 7efe5311c700 1 -- 172.21.15.153:0/3528366183 wait complete. 2023-11-28T13:02:37.341 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:02:37.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.464+0000 7f2774774700 1 Processor -- start 2023-11-28T13:02:37.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.464+0000 7f2774774700 1 -- start start 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.464+0000 7f2774774700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f276c06dee0 0x7f276c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2774774700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276c15d060 con 0x7f276c160540 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2774774700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276c15d340 con 0x7f276c071040 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2774774700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f276c15d620 con 0x7f276c06dee0 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2771d0f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f276c06dee0 0x7f276c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2772d11700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f276c160540 0x7f276c160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59764/0 (socket says 172.21.15.153:59764) 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2771d0f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f276c06dee0 0x7f276c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41596/0 (socket says 172.21.15.153:41596) 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2772d11700 1 -- 172.21.15.153:0/2418562708 learned_addr learned my addr 172.21.15.153:0/2418562708 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2771d0f700 1 -- 172.21.15.153:0/2418562708 >> v1:172.21.15.177:6789/0 conn(0x7f276c071040 legacy=0x7f276c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2771d0f700 1 -- 172.21.15.153:0/2418562708 >> v1:172.21.15.153:6789/0 conn(0x7f276c160540 legacy=0x7f276c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2771d0f700 1 -- 172.21.15.153:0/2418562708 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c15d900 con 0x7f276c06dee0 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f277150e700 1 -- 172.21.15.153:0/2418562708 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164308383 0 0) 0x7f276c15d060 con 0x7f276c160540 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f2771d0f700 1 --2- 172.21.15.153:0/2418562708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c06dee0 0x7f276c06e330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2768009840 tx=0x7f2768009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a077edd5c3e621 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f277150e700 1 -- 172.21.15.153:0/2418562708 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768004230 con 0x7f276c06dee0 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f277150e700 1 -- 172.21.15.153:0/2418562708 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768004390 con 0x7f276c06dee0 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.465+0000 7f277150e700 1 -- 172.21.15.153:0/2418562708 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680036a0 con 0x7f276c06dee0 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.466+0000 7f2774774700 1 -- 172.21.15.153:0/2418562708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c06dee0 msgr2=0x7f276c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.466+0000 7f2774774700 1 --2- 172.21.15.153:0/2418562708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c06dee0 0x7f276c06e330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2768009840 tx=0x7f2768009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.466+0000 7f2774774700 1 -- 172.21.15.153:0/2418562708 shutdown_connections 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.466+0000 7f2774774700 1 --2- 172.21.15.153:0/2418562708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c06dee0 0x7f276c06e330 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.466+0000 7f2774774700 1 -- 172.21.15.153:0/2418562708 >> 172.21.15.153:0/2418562708 conn(0x7f276c0d0000 msgr2=0x7f276c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.466+0000 7f2774774700 1 -- 172.21.15.153:0/2418562708 shutdown_connections 2023-11-28T13:02:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.466+0000 7f2774774700 1 -- 172.21.15.153:0/2418562708 wait complete. 2023-11-28T13:02:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.467+0000 7f2774774700 1 Processor -- start 2023-11-28T13:02:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.467+0000 7f2774774700 1 -- start start 2023-11-28T13:02:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2774774700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276c06dee0 0x7f276c0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2774774700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c071040 0x7f276c0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2774774700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f276c160540 0x7f276c0c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2774774700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0ae760 con 0x7f276c071040 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2774774700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0b0030 con 0x7f276c160540 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2774774700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c0b0310 con 0x7f276c06dee0 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c071040 0x7f276c0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c071040 0x7f276c0ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41606/0 (socket says 172.21.15.153:41606) 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 -- 172.21.15.153:0/4065513470 learned_addr learned my addr 172.21.15.153:0/4065513470 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2772510700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276c06dee0 0x7f276c0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2772d11700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f276c160540 0x7f276c0c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 -- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276c06dee0 msgr2=0x7f276c0b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276c06dee0 0x7f276c0b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 -- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f276c160540 msgr2=0x7f276c0c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f276c160540 0x7f276c0c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 -- 172.21.15.153:0/4065513470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27680094a0 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2772510700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276c06dee0 0x7f276c0b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.468+0000 7f2771d0f700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c071040 0x7f276c0ae220 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f27680090b0 tx=0x7f27680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.469+0000 7f2762ffd700 1 -- 172.21.15.153:0/4065513470 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768011830 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.469+0000 7f2762ffd700 1 -- 172.21.15.153:0/4065513470 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768011990 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.469+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c15ccd0 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.469+0000 7f2762ffd700 1 -- 172.21.15.153:0/4065513470 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768011d40 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.469+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c06e870 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.470+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c06a270 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.470+0000 7f2762ffd700 1 -- 172.21.15.153:0/4065513470 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2768017020 con 0x7f276c071040 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.470+0000 7f2762ffd700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2750068320 0x7f275006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.474 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.471+0000 7f2772510700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2750068320 0x7f275006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.471+0000 7f2762ffd700 1 -- 172.21.15.153:0/4065513470 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2768087420 con 0x7f276c071040 2023-11-28T13:02:37.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.471+0000 7f2772510700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2750068320 0x7f275006a7d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f275c006fd0 tx=0x7f275c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.475 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.475+0000 7f2762ffd700 1 -- 172.21.15.153:0/4065513470 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2768053b90 con 0x7f276c071040 2023-11-28T13:02:37.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.657+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f276c0c6ad0 con 0x7f276c071040 2023-11-28T13:02:37.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.657+0000 7f2762ffd700 1 -- 172.21.15.153:0/4065513470 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f27680571b0 con 0x7f276c071040 2023-11-28T13:02:37.658 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:02:37.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2750068320 msgr2=0x7f275006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2750068320 0x7f275006a7d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f275c006fd0 tx=0x7f275c008040 comp rx=0 tx=0).stop 2023-11-28T13:02:37.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c071040 msgr2=0x7f276c0ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c071040 0x7f276c0ae220 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f27680090b0 tx=0x7f27680047c0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 shutdown_connections 2023-11-28T13:02:37.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f276c160540 0x7f276c0c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f276c06dee0 0x7f276c0b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2750068320 0x7f275006a7d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 --2- 172.21.15.153:0/4065513470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f276c071040 0x7f276c0ae220 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 >> 172.21.15.153:0/4065513470 conn(0x7f276c0d0000 msgr2=0x7f276c0cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 shutdown_connections 2023-11-28T13:02:37.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.659+0000 7f2774774700 1 -- 172.21.15.153:0/4065513470 wait complete. 2023-11-28T13:02:37.672 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:02:37.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.795+0000 7fdae7b21700 1 Processor -- start 2023-11-28T13:02:37.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.795+0000 7fdae7b21700 1 -- start start 2023-11-28T13:02:37.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae7b21700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 0x7fdae00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae7b21700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae013ff00 con 0x7fdae00cd8c0 2023-11-28T13:02:37.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae7b21700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae01404c0 con 0x7fdae00b2e70 2023-11-28T13:02:37.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae7b21700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0140dc0 con 0x7fdae0140950 2023-11-28T13:02:37.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae58bd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fdae00cd8c0 0x7fdae00cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59774/0 (socket says 172.21.15.153:59774) 2023-11-28T13:02:37.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae58bd700 1 -- 172.21.15.153:0/3490155368 learned_addr learned my addr 172.21.15.153:0/3490155368 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:37.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae50bc700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 0x7fdae00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944645893 0 0) 0x7fdae013ff00 con 0x7fdae00cd8c0 2023-11-28T13:02:37.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdacc0035b0 con 0x7fdae00cd8c0 2023-11-28T13:02:37.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996255413 0 0) 0x7fdacc0035b0 con 0x7fdae00cd8c0 2023-11-28T13:02:37.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 msgr2=0x7fdae00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 0x7fdae00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 >> v1:172.21.15.177:6789/0 conn(0x7fdae00b2e70 legacy=0x7fdae006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae015aa80 con 0x7fdae00cd8c0 2023-11-28T13:02:37.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fdadc0017f0 con 0x7fdae00cd8c0 2023-11-28T13:02:37.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 >> v1:172.21.15.153:6789/0 conn(0x7fdae00cd8c0 legacy=0x7fdae00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 0x7fdacc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc003f70 0x7fdacc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdacc004d90 0x7fdacc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0140dc0 con 0x7fdacc004d90 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc00ff00 con 0x7fdacc003f70 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.796+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc009a00 con 0x7fdae0140950 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae60be700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdacc004d90 0x7fdacc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae50bc700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc003f70 0x7fdacc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae58bd700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 0x7fdacc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae60be700 1 -- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 msgr2=0x7fdacc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae60be700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae0140950 0x7fdacc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae60be700 1 -- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc003f70 msgr2=0x7fdacc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae60be700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdacc003f70 0x7fdacc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae60be700 1 -- 172.21.15.153:0/3490155368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc0035b0 con 0x7fdacc004d90 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae60be700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdacc004d90 0x7fdacc005200 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdae00b3240 tx=0x7fdacc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557fa36b87869ddf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8001e70 con 0x7fdacc004d90 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad800cd50 con 0x7fdacc004d90 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.797+0000 7fdae48bb700 1 -- 172.21.15.153:0/3490155368 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8012660 con 0x7fdacc004d90 2023-11-28T13:02:37.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.798+0000 7fdae7b21700 1 -- 172.21.15.153:0/3490155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdacc004d90 msgr2=0x7fdacc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.798+0000 7fdae7b21700 1 --2- 172.21.15.153:0/3490155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdacc004d90 0x7fdacc005200 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdae00b3240 tx=0x7fdacc005880 comp rx=0 tx=0).stop 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.798+0000 7fdae60be700 1 -- 172.21.15.153:0/3490155368 reap_dead start 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.798+0000 7fdae7b21700 1 -- 172.21.15.153:0/3490155368 shutdown_connections 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.798+0000 7fdae7b21700 1 -- 172.21.15.153:0/3490155368 >> 172.21.15.153:0/3490155368 conn(0x7fdae006aed0 msgr2=0x7fdae006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.798+0000 7fdae7b21700 1 -- 172.21.15.153:0/3490155368 shutdown_connections 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.798+0000 7fdae7b21700 1 -- 172.21.15.153:0/3490155368 wait complete. 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.799+0000 7fdae7b21700 1 Processor -- start 2023-11-28T13:02:37.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.799+0000 7fdae7b21700 1 -- start start 2023-11-28T13:02:37.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae01f11c0 0x7fdae00b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae01f1590 0x7fdae00b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae00b6040 0x7fdae00b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01413d0 con 0x7fdae00b6040 2023-11-28T13:02:37.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00b69f0 con 0x7fdae01f11c0 2023-11-28T13:02:37.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01583f0 con 0x7fdae01f1590 2023-11-28T13:02:37.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae00b6040 0x7fdae00b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae00b6040 0x7fdae00b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41626/0 (socket says 172.21.15.153:41626) 2023-11-28T13:02:37.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 -- 172.21.15.153:0/2621949677 learned_addr learned my addr 172.21.15.153:0/2621949677 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:37.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae58bd700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae01f11c0 0x7fdae00b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae50bc700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae01f1590 0x7fdae00b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 -- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae01f1590 msgr2=0x7fdae00b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae01f1590 0x7fdae00b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 -- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae01f11c0 msgr2=0x7fdae00b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae01f11c0 0x7fdae00b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 -- 172.21.15.153:0/2621949677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae00b35f0 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae60be700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae00b6040 0x7fdae00b64b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdad8001520 tx=0x7fdad8001490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdad67fc700 1 -- 172.21.15.153:0/2621949677 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8003e10 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdad67fc700 1 -- 172.21.15.153:0/2621949677 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad8005070 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae00b3310 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdad67fc700 1 -- 172.21.15.153:0/2621949677 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8005420 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.800+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae0140150 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.801+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae0141510 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.802+0000 7fdad67fc700 1 -- 172.21.15.153:0/2621949677 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdad8010050 con 0x7fdae00b6040 2023-11-28T13:02:37.812 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.802+0000 7fdad67fc700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdac40682f0 0x7fdac406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.802+0000 7fdad67fc700 1 -- 172.21.15.153:0/2621949677 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdad80870e0 con 0x7fdae00b6040 2023-11-28T13:02:37.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.802+0000 7fdae58bd700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdac40682f0 0x7fdac406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.803+0000 7fdae58bd700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdac40682f0 0x7fdac406a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdadc000c00 tx=0x7fdadc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.813 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.807+0000 7fdad67fc700 1 -- 172.21.15.153:0/2621949677 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdae0141510 con 0x7fdae00b6040 2023-11-28T13:02:37.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.987+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdae0141510 con 0x7fdae00b6040 2023-11-28T13:02:37.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.988+0000 7fdad67fc700 1 -- 172.21.15.153:0/2621949677 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdae0141510 con 0x7fdae00b6040 2023-11-28T13:02:37.989 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.989+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdac40682f0 msgr2=0x7fdac406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.989+0000 7fdae7b21700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdac40682f0 0x7fdac406a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdadc000c00 tx=0x7fdadc009040 comp rx=0 tx=0).stop 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.989+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae00b6040 msgr2=0x7fdae00b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.989+0000 7fdae7b21700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae00b6040 0x7fdae00b64b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdad8001520 tx=0x7fdad8001490 comp rx=0 tx=0).stop 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 shutdown_connections 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fdac40682f0 0x7fdac406a7a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae00b6040 0x7fdae00b64b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae01f11c0 0x7fdae00b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 --2- 172.21.15.153:0/2621949677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae01f1590 0x7fdae00b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 >> 172.21.15.153:0/2621949677 conn(0x7fdae006aed0 msgr2=0x7fdae01589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 shutdown_connections 2023-11-28T13:02:37.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:37.990+0000 7fdae7b21700 1 -- 172.21.15.153:0/2621949677 wait complete. 2023-11-28T13:02:38.002 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:02:38.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.125+0000 7f40dddc0700 1 Processor -- start 2023-11-28T13:02:38.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.125+0000 7f40dddc0700 1 -- start start 2023-11-28T13:02:38.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.125+0000 7f40dddc0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40d8154e70 0x7f40d80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.125+0000 7f40dddc0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d80cf360 0x7f40d8071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.125+0000 7f40dddc0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d80719e0 con 0x7f40d8071580 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.125+0000 7f40dddc0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d8155240 con 0x7f40d8154e70 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.125+0000 7f40dddc0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d815b100 con 0x7f40d80cf360 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d80cf360 0x7f40d8071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40d8154e70 0x7f40d80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d80cf360 0x7f40d8071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59240/0 (socket says 172.21.15.153:59240) 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 -- 172.21.15.153:0/941687807 learned_addr learned my addr 172.21.15.153:0/941687807 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 -- 172.21.15.153:0/941687807 >> v2:172.21.15.177:3300/0 conn(0x7f40d8154e70 msgr2=0x7f40d80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 --2- 172.21.15.153:0/941687807 >> v2:172.21.15.177:3300/0 conn(0x7f40d8154e70 0x7f40d80cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 -- 172.21.15.153:0/941687807 >> v1:172.21.15.177:6789/0 conn(0x7f40d8071580 legacy=0x7f40d815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 -- 172.21.15.153:0/941687807 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d815b3e0 con 0x7f40d80cf360 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d6ffd700 1 --2- 172.21.15.153:0/941687807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d80cf360 0x7f40d8071040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40c80098f0 tx=0x7f40c8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b06c0716489103 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d67fc700 1 -- 172.21.15.153:0/941687807 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8011830 con 0x7f40d80cf360 2023-11-28T13:02:38.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d67fc700 1 -- 172.21.15.153:0/941687807 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c8011990 con 0x7f40d80cf360 2023-11-28T13:02:38.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.126+0000 7f40d67fc700 1 -- 172.21.15.153:0/941687807 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8011d20 con 0x7f40d80cf360 2023-11-28T13:02:38.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 -- 172.21.15.153:0/941687807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d80cf360 msgr2=0x7f40d8071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 --2- 172.21.15.153:0/941687807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d80cf360 0x7f40d8071040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40c80098f0 tx=0x7f40c8009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:38.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 -- 172.21.15.153:0/941687807 shutdown_connections 2023-11-28T13:02:38.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 --2- 172.21.15.153:0/941687807 >> v2:172.21.15.177:3300/0 conn(0x7f40d8154e70 0x7f40d80cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 --2- 172.21.15.153:0/941687807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d80cf360 0x7f40d8071040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 -- 172.21.15.153:0/941687807 >> 172.21.15.153:0/941687807 conn(0x7f40d80d0000 msgr2=0x7f40d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:38.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 -- 172.21.15.153:0/941687807 shutdown_connections 2023-11-28T13:02:38.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.127+0000 7f40dddc0700 1 -- 172.21.15.153:0/941687807 wait complete. 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.128+0000 7f40dddc0700 1 Processor -- start 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.128+0000 7f40dddc0700 1 -- start start 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40dddc0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d8071580 0x7f40d80c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40dddc0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80cf360 0x7f40d8152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40dddc0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d8154e70 0x7f40d80c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40dddc0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d815a420 con 0x7f40d80cf360 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40dddc0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8152d70 con 0x7f40d8071580 2023-11-28T13:02:38.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40dddc0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c96e0 con 0x7f40d8154e70 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80cf360 0x7f40d8152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80cf360 0x7f40d8152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41638/0 (socket says 172.21.15.153:41638) 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40d6ffd700 1 -- 172.21.15.153:0/605026328 learned_addr learned my addr 172.21.15.153:0/605026328 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40d77fe700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d8071580 0x7f40d80c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40d7fff700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d8154e70 0x7f40d80c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.129+0000 7f40d6ffd700 1 -- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d8154e70 msgr2=0x7f40d80c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40d6ffd700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d8154e70 0x7f40d80c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40d6ffd700 1 -- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d8071580 msgr2=0x7f40d80c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40d6ffd700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d8071580 0x7f40d80c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40d6ffd700 1 -- 172.21.15.153:0/605026328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8009550 con 0x7f40d80cf360 2023-11-28T13:02:38.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40d6ffd700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80cf360 0x7f40d8152f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000c00 tx=0x7f40c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40bffff700 1 -- 172.21.15.153:0/605026328 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8004400 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40bffff700 1 -- 172.21.15.153:0/605026328 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c801f070 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d815b9a0 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40bffff700 1 -- 172.21.15.153:0/605026328 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c80038e0 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.130+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d815b6c0 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.131+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d80af520 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.131+0000 7f40bffff700 1 -- 172.21.15.153:0/605026328 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f40c802a020 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.132+0000 7f40bffff700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40c40682f0 0x7f40c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.132+0000 7f40bffff700 1 -- 172.21.15.153:0/605026328 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40c8087220 con 0x7f40d80cf360 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.132+0000 7f40d77fe700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40c40682f0 0x7f40c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.132+0000 7f40d77fe700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40c40682f0 0x7f40c406a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f40d81588f0 tx=0x7f40cc009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.136+0000 7f40bffff700 1 -- 172.21.15.153:0/605026328 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40c8053ab0 con 0x7f40d80cf360 2023-11-28T13:02:38.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.320+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40d8066a20 con 0x7f40d80cf360 2023-11-28T13:02:38.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.321+0000 7f40bffff700 1 -- 172.21.15.153:0/605026328 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40c8015020 con 0x7f40d80cf360 2023-11-28T13:02:38.322 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:02:38.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40c40682f0 msgr2=0x7f40c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40c40682f0 0x7f40c406a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f40d81588f0 tx=0x7f40cc009210 comp rx=0 tx=0).stop 2023-11-28T13:02:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80cf360 msgr2=0x7f40d8152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80cf360 0x7f40d8152f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000c00 tx=0x7f40c8004750 comp rx=0 tx=0).stop 2023-11-28T13:02:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 shutdown_connections 2023-11-28T13:02:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d8154e70 0x7f40d80c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d8071580 0x7f40d80c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40c40682f0 0x7f40c406a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 --2- 172.21.15.153:0/605026328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40d80cf360 0x7f40d8152f00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 >> 172.21.15.153:0/605026328 conn(0x7f40d80d0000 msgr2=0x7f40d80b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:38.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 shutdown_connections 2023-11-28T13:02:38.326 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.323+0000 7f40dddc0700 1 -- 172.21.15.153:0/605026328 wait complete. 2023-11-28T13:02:38.336 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:02:38.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.458+0000 7f5af5db6700 1 Processor -- start 2023-11-28T13:02:38.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.458+0000 7f5af5db6700 1 -- start start 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5af5db6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af006dee0 0x7f5af006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5af5db6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5af0160540 0x7f5af01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5af5db6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af015d060 con 0x7f5af0071040 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5af5db6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af015d340 con 0x7f5af0160540 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5af5db6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af015d620 con 0x7f5af006dee0 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5aeffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5af0160540 0x7f5af01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5aeffff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5af0160540 0x7f5af01609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41648/0 (socket says 172.21.15.153:41648) 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.459+0000 7f5aeffff700 1 -- 172.21.15.153:0/1861326057 learned_addr learned my addr 172.21.15.153:0/1861326057 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:38.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aeeffd700 1 --2- 172.21.15.153:0/1861326057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af006dee0 0x7f5af006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aeffff700 1 -- 172.21.15.153:0/1861326057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af006dee0 msgr2=0x7f5af006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aeffff700 1 --2- 172.21.15.153:0/1861326057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af006dee0 0x7f5af006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aeffff700 1 -- 172.21.15.153:0/1861326057 >> v1:172.21.15.177:6789/0 conn(0x7f5af0071040 legacy=0x7f5af0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aeffff700 1 -- 172.21.15.153:0/1861326057 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af015d900 con 0x7f5af0160540 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aeffff700 1 --2- 172.21.15.153:0/1861326057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0160540 0x7f5af01609c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009e50 tx=0x7f5ae800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d15228b98e352c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aee7fc700 1 -- 172.21.15.153:0/1861326057 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800e070 con 0x7f5af0160540 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aee7fc700 1 -- 172.21.15.153:0/1861326057 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8003a20 con 0x7f5af0160540 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5aee7fc700 1 -- 172.21.15.153:0/1861326057 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8003dd0 con 0x7f5af0160540 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5af5db6700 1 -- 172.21.15.153:0/1861326057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0160540 msgr2=0x7f5af01609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.460+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1861326057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0160540 0x7f5af01609c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009e50 tx=0x7f5ae800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:38.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.461+0000 7f5af5db6700 1 -- 172.21.15.153:0/1861326057 shutdown_connections 2023-11-28T13:02:38.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.461+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1861326057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0160540 0x7f5af01609c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.461+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1861326057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af006dee0 0x7f5af006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.461+0000 7f5af5db6700 1 -- 172.21.15.153:0/1861326057 >> 172.21.15.153:0/1861326057 conn(0x7f5af00d0000 msgr2=0x7f5af00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:38.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.461+0000 7f5af5db6700 1 -- 172.21.15.153:0/1861326057 shutdown_connections 2023-11-28T13:02:38.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.461+0000 7f5af5db6700 1 -- 172.21.15.153:0/1861326057 wait complete. 2023-11-28T13:02:38.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.462+0000 7f5af5db6700 1 Processor -- start 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.462+0000 7f5af5db6700 1 -- start start 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af006dee0 0x7f5af01f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0071040 0x7f5af00b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af0160540 0x7f5af00cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01f6ac0 con 0x7f5af0071040 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01f5f40 con 0x7f5af006dee0 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af01f6220 con 0x7f5af0160540 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0071040 0x7f5af00b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0071040 0x7f5af00b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41652/0 (socket says 172.21.15.153:41652) 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 -- 172.21.15.153:0/1498594066 learned_addr learned my addr 172.21.15.153:0/1498594066 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:38.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeffff700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af0160540 0x7f5af00cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aef7fe700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af006dee0 0x7f5af01f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 -- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af0160540 msgr2=0x7f5af00cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af0160540 0x7f5af00cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 -- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af006dee0 msgr2=0x7f5af01f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af006dee0 0x7f5af01f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 -- 172.21.15.153:0/1498594066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8009b30 con 0x7f5af0071040 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5aeeffd700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0071040 0x7f5af00b48b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d800 tx=0x7f5ae000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5ae7fff700 1 -- 172.21.15.153:0/1498594066 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00125f0 con 0x7f5af0071040 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5ae7fff700 1 -- 172.21.15.153:0/1498594066 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0012750 con 0x7f5af0071040 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5ae7fff700 1 -- 172.21.15.153:0/1498594066 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0012b00 con 0x7f5af0071040 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af015ccd0 con 0x7f5af0071040 2023-11-28T13:02:38.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.463+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af006e870 con 0x7f5af0071040 2023-11-28T13:02:38.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.464+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af006a270 con 0x7f5af0071040 2023-11-28T13:02:38.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.465+0000 7f5ae7fff700 1 -- 172.21.15.153:0/1498594066 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ae0012c60 con 0x7f5af0071040 2023-11-28T13:02:38.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.465+0000 7f5ae7fff700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5ad00682f0 0x7f5ad006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.465+0000 7f5ae7fff700 1 -- 172.21.15.153:0/1498594066 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ae0085e70 con 0x7f5af0071040 2023-11-28T13:02:38.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.465+0000 7f5aef7fe700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5ad00682f0 0x7f5ad006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.466+0000 7f5aef7fe700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5ad00682f0 0x7f5ad006a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009960 tx=0x7f5ad80090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.469+0000 7f5ae7fff700 1 -- 172.21.15.153:0/1498594066 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ae00526d0 con 0x7f5af0071040 2023-11-28T13:02:38.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.667+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5af00b2fd0 con 0x7f5af0071040 2023-11-28T13:02:38.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.667+0000 7f5ae7fff700 1 -- 172.21.15.153:0/1498594066 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ae0055cf0 con 0x7f5af0071040 2023-11-28T13:02:38.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5ad00682f0 msgr2=0x7f5ad006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5ad00682f0 0x7f5ad006a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009960 tx=0x7f5ad80090e0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0071040 msgr2=0x7f5af00b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0071040 0x7f5af00b48b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d800 tx=0x7f5ae000db10 comp rx=0 tx=0).stop 2023-11-28T13:02:38.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 shutdown_connections 2023-11-28T13:02:38.674 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5af0160540 0x7f5af00cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5af006dee0 0x7f5af01f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5ad00682f0 0x7f5ad006a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 --2- 172.21.15.153:0/1498594066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5af0071040 0x7f5af00b48b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 >> 172.21.15.153:0/1498594066 conn(0x7f5af00d0000 msgr2=0x7f5af015f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:38.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 shutdown_connections 2023-11-28T13:02:38.676 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.669+0000 7f5af5db6700 1 -- 172.21.15.153:0/1498594066 wait complete. 2023-11-28T13:02:38.676 INFO:teuthology.orchestra.run.smithi153.stderr:listed 0 entries 2023-11-28T13:02:38.683 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:02:38.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:38.745 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:02:38.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.746 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:02:38.774 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:02:38.774 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:38.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:38.794 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:02:38.794 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:38.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:38.832 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:02:38.833 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:38.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:38.853 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:38.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.991+0000 7f5134517700 1 Processor -- start 2023-11-28T13:02:38.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.991+0000 7f5134517700 1 -- start start 2023-11-28T13:02:38.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.992+0000 7f5134517700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f512c0b0d80 0x7f512c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.992+0000 7f5134517700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0ccc00 con 0x7f512c159490 2023-11-28T13:02:38.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.992+0000 7f5134517700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0ccec0 con 0x7f512c159860 2023-11-28T13:02:38.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.992+0000 7f5134517700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c0cd1a0 con 0x7f512c0b0d80 2023-11-28T13:02:38.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.992+0000 7f51322b3700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f512c159490 0x7f512c0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59822/0 (socket says 172.21.15.153:59822) 2023-11-28T13:02:38.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.992+0000 7f51322b3700 1 -- 172.21.15.153:0/1442000377 learned_addr learned my addr 172.21.15.153:0/1442000377 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.992+0000 7f5132ab4700 1 --2- 172.21.15.153:0/1442000377 >> v2:172.21.15.177:3300/0 conn(0x7f512c0b0d80 0x7f512c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f51312b1700 1 -- 172.21.15.153:0/1442000377 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835049030 0 0) 0x7f512c0ccc00 con 0x7f512c159490 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f51312b1700 1 -- 172.21.15.153:0/1442000377 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51180035b0 con 0x7f512c159490 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f51312b1700 1 -- 172.21.15.153:0/1442000377 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482620197 0 0) 0x7f51180035b0 con 0x7f512c159490 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f5132ab4700 1 -- 172.21.15.153:0/1442000377 >> v1:172.21.15.177:6789/0 conn(0x7f512c159860 legacy=0x7f512c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f5132ab4700 1 -- 172.21.15.153:0/1442000377 >> v1:172.21.15.153:6789/0 conn(0x7f512c159490 legacy=0x7f512c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f5132ab4700 1 -- 172.21.15.153:0/1442000377 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0cd480 con 0x7f512c0b0d80 2023-11-28T13:02:38.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f5132ab4700 1 --2- 172.21.15.153:0/1442000377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c0b0d80 0x7f512c0c9bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f512400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d7bc59c3bfd98a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f51312b1700 1 -- 172.21.15.153:0/1442000377 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400e070 con 0x7f512c0b0d80 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f51312b1700 1 -- 172.21.15.153:0/1442000377 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5124003a20 con 0x7f512c0b0d80 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.993+0000 7f51312b1700 1 -- 172.21.15.153:0/1442000377 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c001670 con 0x7f512c0b0d80 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.994+0000 7f5134517700 1 -- 172.21.15.153:0/1442000377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c0b0d80 msgr2=0x7f512c0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.994+0000 7f5134517700 1 --2- 172.21.15.153:0/1442000377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c0b0d80 0x7f512c0c9bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5124009d90 tx=0x7f512400b330 comp rx=0 tx=0).stop 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.994+0000 7f5134517700 1 -- 172.21.15.153:0/1442000377 shutdown_connections 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.994+0000 7f5134517700 1 --2- 172.21.15.153:0/1442000377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c0b0d80 0x7f512c0c9bd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.994+0000 7f5134517700 1 -- 172.21.15.153:0/1442000377 >> 172.21.15.153:0/1442000377 conn(0x7f512c0cb670 msgr2=0x7f512c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.994+0000 7f5134517700 1 -- 172.21.15.153:0/1442000377 shutdown_connections 2023-11-28T13:02:38.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.994+0000 7f5134517700 1 -- 172.21.15.153:0/1442000377 wait complete. 2023-11-28T13:02:39.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.995+0000 7f5134517700 1 Processor -- start 2023-11-28T13:02:39.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.995+0000 7f5134517700 1 -- start start 2023-11-28T13:02:39.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5134517700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512c0b0d80 0x7f512c0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5134517700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c159490 0x7f512c071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5134517700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c13f180 0x7f512c13f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5134517700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0b4960 con 0x7f512c13f180 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5134517700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0b2970 con 0x7f512c159490 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5134517700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0b38c0 con 0x7f512c0b0d80 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c13f180 0x7f512c13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c13f180 0x7f512c13f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41668/0 (socket says 172.21.15.153:41668) 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 -- 172.21.15.153:0/2532328026 learned_addr learned my addr 172.21.15.153:0/2532328026 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f51322b3700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512c0b0d80 0x7f512c0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5131ab2700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c159490 0x7f512c071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 -- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512c0b0d80 msgr2=0x7f512c0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512c0b0d80 0x7f512c0b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 -- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c159490 msgr2=0x7f512c071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c159490 0x7f512c071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.001 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 -- 172.21.15.153:0/2532328026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124009a70 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5131ab2700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c159490 0x7f512c071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5132ab4700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c13f180 0x7f512c13f620 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f512c067360 tx=0x7f5124004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5122ffd700 1 -- 172.21.15.153:0/2532328026 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512401f070 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5122ffd700 1 -- 172.21.15.153:0/2532328026 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240043c0 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0cdc80 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.996+0000 7f5122ffd700 1 -- 172.21.15.153:0/2532328026 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124004770 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.997+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c0cd760 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.997+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c0cde10 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.998+0000 7f5122ffd700 1 -- 172.21.15.153:0/2532328026 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5124019470 con 0x7f512c13f180 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.998+0000 7f5122ffd700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51100682f0 0x7f511006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.002 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.998+0000 7f5122ffd700 1 -- 172.21.15.153:0/2532328026 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5124088ea0 con 0x7f512c13f180 2023-11-28T13:02:39.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.998+0000 7f51322b3700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51100682f0 0x7f511006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:38.999+0000 7f51322b3700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51100682f0 0x7f511006a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f511c0011e0 tx=0x7f511c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.003 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.003+0000 7f5122ffd700 1 -- 172.21.15.153:0/2532328026 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f51240556a0 con 0x7f512c13f180 2023-11-28T13:02:39.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.225+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f512c0745c0 con 0x7f512c13f180 2023-11-28T13:02:39.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.226+0000 7f5122ffd700 1 -- 172.21.15.153:0/2532328026 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f5124058cc0 con 0x7f512c13f180 2023-11-28T13:02:39.227 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:39.227 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51100682f0 msgr2=0x7f511006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51100682f0 0x7f511006a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f511c0011e0 tx=0x7f511c009040 comp rx=0 tx=0).stop 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c13f180 msgr2=0x7f512c13f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c13f180 0x7f512c13f620 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f512c067360 tx=0x7f5124004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 shutdown_connections 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f51100682f0 0x7f511006a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f512c13f180 0x7f512c13f620 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f512c0b0d80 0x7f512c0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 --2- 172.21.15.153:0/2532328026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f512c159490 0x7f512c071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 >> 172.21.15.153:0/2532328026 conn(0x7f512c0cb670 msgr2=0x7f512c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:39.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 shutdown_connections 2023-11-28T13:02:39.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.228+0000 7f5134517700 1 -- 172.21.15.153:0/2532328026 wait complete. 2023-11-28T13:02:39.230 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:39.241 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:39.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.359+0000 7fb98da14700 1 Processor -- start 2023-11-28T13:02:39.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.359+0000 7fb98da14700 1 -- start start 2023-11-28T13:02:39.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.360+0000 7fb98da14700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.360+0000 7fb98da14700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb9880cbdf0 0x7fb9880ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.360+0000 7fb98da14700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880cc5f0 con 0x7fb98815cef0 2023-11-28T13:02:39.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.360+0000 7fb98da14700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98815d470 con 0x7fb9880cbdf0 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.360+0000 7fb98da14700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98815d750 con 0x7fb98806ba90 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9867fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb9880cbdf0 0x7fb9880ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb9880cbdf0 0x7fb9880ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:51142/0 (socket says 172.21.15.153:51142) 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9867fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59284/0 (socket says 172.21.15.153:59284) 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 -- 172.21.15.153:0/1421364134 learned_addr learned my addr 172.21.15.153:0/1421364134 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 -- 172.21.15.153:0/1421364134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 msgr2=0x7fb9880af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 --2- 172.21.15.153:0/1421364134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 -- 172.21.15.153:0/1421364134 >> v1:172.21.15.177:6789/0 conn(0x7fb98815cef0 legacy=0x7fb98806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 -- 172.21.15.153:0/1421364134 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98815da30 con 0x7fb9880cbdf0 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb985ffb700 1 -- 172.21.15.153:0/1421364134 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364026497 0 0) 0x7fb9880cc5f0 con 0x7fb98815cef0 2023-11-28T13:02:39.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb9877fe700 1 --2- 172.21.15.153:0/1421364134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 0x7fb9880ca650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d150 tx=0x7fb98000d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b5c0b16a2496e5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb985ffb700 1 -- 172.21.15.153:0/1421364134 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980010070 con 0x7fb9880cbdf0 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb985ffb700 1 -- 172.21.15.153:0/1421364134 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98000dd10 con 0x7fb9880cbdf0 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.361+0000 7fb985ffb700 1 -- 172.21.15.153:0/1421364134 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000b310 con 0x7fb9880cbdf0 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 -- 172.21.15.153:0/1421364134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 msgr2=0x7fb9880ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 --2- 172.21.15.153:0/1421364134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 0x7fb9880ca650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d150 tx=0x7fb98000d460 comp rx=0 tx=0).stop 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 -- 172.21.15.153:0/1421364134 shutdown_connections 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 --2- 172.21.15.153:0/1421364134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 0x7fb9880ca650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 --2- 172.21.15.153:0/1421364134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 -- 172.21.15.153:0/1421364134 >> 172.21.15.153:0/1421364134 conn(0x7fb9880d0000 msgr2=0x7fb9880d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 -- 172.21.15.153:0/1421364134 shutdown_connections 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.362+0000 7fb98da14700 1 -- 172.21.15.153:0/1421364134 wait complete. 2023-11-28T13:02:39.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.363+0000 7fb98da14700 1 Processor -- start 2023-11-28T13:02:39.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.363+0000 7fb98da14700 1 -- start start 2023-11-28T13:02:39.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb98da14700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb98da14700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 0x7fb988159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb98da14700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb98815cef0 0x7fb98806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb98da14700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880b3680 con 0x7fb98815cef0 2023-11-28T13:02:39.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb98da14700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880b3960 con 0x7fb9880cbdf0 2023-11-28T13:02:39.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb98da14700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880b3c40 con 0x7fb98806ba90 2023-11-28T13:02:39.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb98815cef0 0x7fb98806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb98815cef0 0x7fb98806e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41682/0 (socket says 172.21.15.153:41682) 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 -- 172.21.15.153:0/3970825847 learned_addr learned my addr 172.21.15.153:0/3970825847 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9867fc700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 0x7fb988159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb986ffd700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 -- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 msgr2=0x7fb9880b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 -- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 msgr2=0x7fb988159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 0x7fb988159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.364+0000 7fb9877fe700 1 -- 172.21.15.153:0/3970825847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98000bd40 con 0x7fb98815cef0 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.365+0000 7fb986ffd700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.365+0000 7fb9877fe700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb98815cef0 0x7fb98806e0f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb98806c7b0 tx=0x7fb980004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.365+0000 7fb97ffff700 1 -- 172.21.15.153:0/3970825847 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980009a10 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.365+0000 7fb97ffff700 1 -- 172.21.15.153:0/3970825847 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb980009b70 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.365+0000 7fb97ffff700 1 -- 172.21.15.153:0/3970825847 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980004040 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.365+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98815dff0 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.365+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98815dd10 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.366+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9880cab90 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.367+0000 7fb97ffff700 1 -- 172.21.15.153:0/3970825847 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9800041a0 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.367+0000 7fb97ffff700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9680682f0 0x7fb96806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.367+0000 7fb97ffff700 1 -- 172.21.15.153:0/3970825847 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb980088000 con 0x7fb98815cef0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.367+0000 7fb986ffd700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9680682f0 0x7fb96806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.367+0000 7fb986ffd700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9680682f0 0x7fb96806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb970000f60 tx=0x7fb970009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.371+0000 7fb97ffff700 1 -- 172.21.15.153:0/3970825847 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb980054900 con 0x7fb98815cef0 2023-11-28T13:02:39.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.556+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9880d0e60 con 0x7fb98815cef0 2023-11-28T13:02:39.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.558+0000 7fb97ffff700 1 -- 172.21.15.153:0/3970825847 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18027 (secure 0 0 0) 0x7fb98000b540 con 0x7fb98815cef0 2023-11-28T13:02:39.559 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:39.559 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":51,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:57.932997+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:54.715348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:55.897576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9680682f0 msgr2=0x7fb96806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9680682f0 0x7fb96806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb970000f60 tx=0x7fb970009040 comp rx=0 tx=0).stop 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb98815cef0 msgr2=0x7fb98806e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb98815cef0 0x7fb98806e0f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb98806c7b0 tx=0x7fb980004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 shutdown_connections 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9680682f0 0x7fb96806a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb98815cef0 0x7fb98806e0f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98806ba90 0x7fb9880b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 --2- 172.21.15.153:0/3970825847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9880cbdf0 0x7fb988159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 >> 172.21.15.153:0/3970825847 conn(0x7fb9880d0000 msgr2=0x7fb9880cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:39.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 shutdown_connections 2023-11-28T13:02:39.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.560+0000 7fb98da14700 1 -- 172.21.15.153:0/3970825847 wait complete. 2023-11-28T13:02:39.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:01:54.715348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:01:55.897576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:02:39.573 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:02:39.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.696+0000 7f5ed11ee700 1 Processor -- start 2023-11-28T13:02:39.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.696+0000 7f5ed11ee700 1 -- start start 2023-11-28T13:02:39.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ed11ee700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5ecc071040 0x7f5ecc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ed11ee700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5ecc06dee0 0x7f5ecc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ed11ee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc071950 con 0x7f5ecc06cdc0 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ed11ee700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc161550 con 0x7f5ecc071040 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ed11ee700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc15d060 con 0x7f5ecc06dee0 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ecad9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5ecc071040 0x7f5ecc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ecad9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5ecc071040 0x7f5ecc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41686/0 (socket says 172.21.15.153:41686) 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ecad9d700 1 -- 172.21.15.153:0/2748277167 learned_addr learned my addr 172.21.15.153:0/2748277167 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5eca59c700 1 --2- 172.21.15.153:0/2748277167 >> v2:172.21.15.177:3300/0 conn(0x7f5ecc06dee0 0x7f5ecc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ecad9d700 1 -- 172.21.15.153:0/2748277167 >> v2:172.21.15.177:3300/0 conn(0x7f5ecc06dee0 msgr2=0x7f5ecc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ecad9d700 1 --2- 172.21.15.153:0/2748277167 >> v2:172.21.15.177:3300/0 conn(0x7f5ecc06dee0 0x7f5ecc06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ecad9d700 1 -- 172.21.15.153:0/2748277167 >> v1:172.21.15.177:6789/0 conn(0x7f5ecc06cdc0 legacy=0x7f5ecc06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.697+0000 7f5ecad9d700 1 -- 172.21.15.153:0/2748277167 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc15d340 con 0x7f5ecc071040 2023-11-28T13:02:39.700 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ecad9d700 1 --2- 172.21.15.153:0/2748277167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc071040 0x7f5ecc071410 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06fb00 tx=0x7f5ec0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09079e91828ce5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ec9d9b700 1 -- 172.21.15.153:0/2748277167 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000b870 con 0x7f5ecc071040 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ec9d9b700 1 -- 172.21.15.153:0/2748277167 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000b9d0 con 0x7f5ecc071040 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ec9d9b700 1 -- 172.21.15.153:0/2748277167 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000bd60 con 0x7f5ecc071040 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 -- 172.21.15.153:0/2748277167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc071040 msgr2=0x7f5ecc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/2748277167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc071040 0x7f5ecc071410 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06fb00 tx=0x7f5ec0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 -- 172.21.15.153:0/2748277167 shutdown_connections 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/2748277167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc071040 0x7f5ecc071410 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/2748277167 >> v2:172.21.15.177:3300/0 conn(0x7f5ecc06dee0 0x7f5ecc06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 -- 172.21.15.153:0/2748277167 >> 172.21.15.153:0/2748277167 conn(0x7f5ecc0d0000 msgr2=0x7f5ecc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 -- 172.21.15.153:0/2748277167 shutdown_connections 2023-11-28T13:02:39.701 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.698+0000 7f5ed11ee700 1 -- 172.21.15.153:0/2748277167 wait complete. 2023-11-28T13:02:39.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.700+0000 7f5ed11ee700 1 Processor -- start 2023-11-28T13:02:39.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.700+0000 7f5ed11ee700 1 -- start start 2023-11-28T13:02:39.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ecc06cdc0 0x7f5ecc1472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc06dee0 0x7f5ecc0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ecc071040 0x7f5ecc1e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc147820 con 0x7f5ecc06dee0 2023-11-28T13:02:39.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0adff0 con 0x7f5ecc071040 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0ae2d0 con 0x7f5ecc06cdc0 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc06dee0 0x7f5ecc0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc06dee0 0x7f5ecc0ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41694/0 (socket says 172.21.15.153:41694) 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 -- 172.21.15.153:0/445485957 learned_addr learned my addr 172.21.15.153:0/445485957 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ecad9d700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ecc06cdc0 0x7f5ecc1472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ecb59e700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ecc071040 0x7f5ecc1e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 -- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ecc06cdc0 msgr2=0x7f5ecc1472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ecc06cdc0 0x7f5ecc1472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 -- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ecc071040 msgr2=0x7f5ecc1e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ecc071040 0x7f5ecc1e6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 -- 172.21.15.153:0/445485957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009550 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ecb59e700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ecc071040 0x7f5ecc1e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eca59c700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc06dee0 0x7f5ecc0ad570 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d800 tx=0x7f5ebc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eb37fe700 1 -- 172.21.15.153:0/445485957 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0125f0 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eb37fe700 1 -- 172.21.15.153:0/445485957 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc012750 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc15d900 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5eb37fe700 1 -- 172.21.15.153:0/445485957 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc012b00 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.701+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc15d620 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.702+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc06e870 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.703+0000 7f5eb37fe700 1 -- 172.21.15.153:0/445485957 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ebc012c60 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.703+0000 7f5eb37fe700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5eb80682f0 0x7f5eb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.703+0000 7f5eb37fe700 1 -- 172.21.15.153:0/445485957 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5ebc0860f0 con 0x7f5ecc06dee0 2023-11-28T13:02:39.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.703+0000 7f5ecad9d700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5eb80682f0 0x7f5eb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:39.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.703+0000 7f5ecad9d700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5eb80682f0 0x7f5eb806a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0014040 tx=0x7f5ec0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:39.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.707+0000 7f5eb37fe700 1 -- 172.21.15.153:0/445485957 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ebc052950 con 0x7f5ecc06dee0 2023-11-28T13:02:39.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.890+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ecc0720a0 con 0x7f5ecc06dee0 2023-11-28T13:02:39.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.890+0000 7f5eb37fe700 1 -- 172.21.15.153:0/445485957 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ebc055f70 con 0x7f5ecc06dee0 2023-11-28T13:02:39.891 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:02:39.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5eb80682f0 msgr2=0x7f5eb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5eb80682f0 0x7f5eb806a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0014040 tx=0x7f5ec0008030 comp rx=0 tx=0).stop 2023-11-28T13:02:39.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc06dee0 msgr2=0x7f5ecc0ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:39.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc06dee0 0x7f5ecc0ad570 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d800 tx=0x7f5ebc00db10 comp rx=0 tx=0).stop 2023-11-28T13:02:39.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 shutdown_connections 2023-11-28T13:02:39.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ecc071040 0x7f5ecc1e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ecc06cdc0 0x7f5ecc1472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5eb80682f0 0x7f5eb806a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 --2- 172.21.15.153:0/445485957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ecc06dee0 0x7f5ecc0ad570 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:39.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 >> 172.21.15.153:0/445485957 conn(0x7f5ecc0d0000 msgr2=0x7f5ecc1575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:39.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 shutdown_connections 2023-11-28T13:02:39.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:39.892+0000 7f5ed11ee700 1 -- 172.21.15.153:0/445485957 wait complete. 2023-11-28T13:02:39.905 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:02:40.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.026+0000 7f3f701dc700 1 Processor -- start 2023-11-28T13:02:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.026+0000 7f3f701dc700 1 -- start start 2023-11-28T13:02:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f701dc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f680cd8c0 0x7f3f68140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f701dc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3f680cdc90 0x7f3f680b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f701dc700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6813ff00 con 0x7f3f680b2e70 2023-11-28T13:02:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f701dc700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f681404c0 con 0x7f3f680cd8c0 2023-11-28T13:02:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f701dc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6815aa80 con 0x7f3f680cdc90 2023-11-28T13:02:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6df78700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f680cd8c0 0x7f3f68140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6e779700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3f680b2e70 0x7f3f6806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59866/0 (socket says 172.21.15.153:59866) 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6e779700 1 -- 172.21.15.153:0/3975191627 learned_addr learned my addr 172.21.15.153:0/3975191627 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6df78700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f680cd8c0 0x7f3f68140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41700/0 (socket says 172.21.15.153:41700) 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6d777700 1 --2- 172.21.15.153:0/3975191627 >> v2:172.21.15.177:3300/0 conn(0x7f3f680cdc90 0x7f3f680b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6df78700 1 -- 172.21.15.153:0/3975191627 >> v2:172.21.15.177:3300/0 conn(0x7f3f680cdc90 msgr2=0x7f3f680b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6df78700 1 --2- 172.21.15.153:0/3975191627 >> v2:172.21.15.177:3300/0 conn(0x7f3f680cdc90 0x7f3f680b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6df78700 1 -- 172.21.15.153:0/3975191627 >> v1:172.21.15.153:6789/0 conn(0x7f3f680b2e70 legacy=0x7f3f6806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6df78700 1 -- 172.21.15.153:0/3975191627 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680b3310 con 0x7f3f680cd8c0 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6d777700 1 --2- 172.21.15.153:0/3975191627 >> v2:172.21.15.177:3300/0 conn(0x7f3f680cdc90 0x7f3f680b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6df78700 1 --2- 172.21.15.153:0/3975191627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680cd8c0 0x7f3f68140950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3f68071c40 tx=0x7f3f64009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd38df23cca2bbb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.027+0000 7f3f6cf76700 1 -- 172.21.15.153:0/3975191627 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64005e30 con 0x7f3f680cd8c0 2023-11-28T13:02:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f6cf76700 1 -- 172.21.15.153:0/3975191627 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f64006030 con 0x7f3f680cd8c0 2023-11-28T13:02:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f6cf76700 1 -- 172.21.15.153:0/3975191627 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f640063c0 con 0x7f3f680cd8c0 2023-11-28T13:02:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 -- 172.21.15.153:0/3975191627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680cd8c0 msgr2=0x7f3f68140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 --2- 172.21.15.153:0/3975191627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680cd8c0 0x7f3f68140950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3f68071c40 tx=0x7f3f64009b00 comp rx=0 tx=0).stop 2023-11-28T13:02:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 -- 172.21.15.153:0/3975191627 shutdown_connections 2023-11-28T13:02:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 --2- 172.21.15.153:0/3975191627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680cd8c0 0x7f3f68140950 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 --2- 172.21.15.153:0/3975191627 >> v2:172.21.15.177:3300/0 conn(0x7f3f680cdc90 0x7f3f680b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 -- 172.21.15.153:0/3975191627 >> 172.21.15.153:0/3975191627 conn(0x7f3f6806aed0 msgr2=0x7f3f6806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 -- 172.21.15.153:0/3975191627 shutdown_connections 2023-11-28T13:02:40.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.028+0000 7f3f701dc700 1 -- 172.21.15.153:0/3975191627 wait complete. 2023-11-28T13:02:40.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.029+0000 7f3f701dc700 1 Processor -- start 2023-11-28T13:02:40.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.029+0000 7f3f701dc700 1 -- start start 2023-11-28T13:02:40.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f701dc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680b2e70 0x7f3f68157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f701dc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f680cd8c0 0x7f3f6813cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f701dc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f680cdc90 0x7f3f681567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f701dc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68159110 con 0x7f3f680b2e70 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f701dc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680c46a0 con 0x7f3f680cdc90 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f701dc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68158500 con 0x7f3f680cd8c0 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680b2e70 0x7f3f68157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680b2e70 0x7f3f68157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41704/0 (socket says 172.21.15.153:41704) 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 -- 172.21.15.153:0/2327683428 learned_addr learned my addr 172.21.15.153:0/2327683428 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 -- 172.21.15.153:0/2327683428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f680cd8c0 msgr2=0x7f3f6813cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f680cd8c0 0x7f3f6813cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 -- 172.21.15.153:0/2327683428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f680cdc90 msgr2=0x7f3f681567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f680cdc90 0x7f3f681567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 -- 172.21.15.153:0/2327683428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64009430 con 0x7f3f680b2e70 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.030+0000 7f3f6df78700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680b2e70 0x7f3f68157e00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3f680b3aa0 tx=0x7f3f64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.031+0000 7f3f5e7fc700 1 -- 172.21.15.153:0/2327683428 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64004240 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.031+0000 7f3f5e7fc700 1 -- 172.21.15.153:0/2327683428 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f640043c0 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.031+0000 7f3f5e7fc700 1 -- 172.21.15.153:0/2327683428 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f640036a0 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.031+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f68140150 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.031+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f680b35f0 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.032+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f680ce100 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.032+0000 7f3f5e7fc700 1 -- 172.21.15.153:0/2327683428 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f6400f050 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.032+0000 7f3f5e7fc700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f4c0682f0 0x7f3f4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.032+0000 7f3f6d777700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f4c0682f0 0x7f3f4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.032+0000 7f3f5e7fc700 1 -- 172.21.15.153:0/2327683428 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f640878e0 con 0x7f3f680b2e70 2023-11-28T13:02:40.036 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.033+0000 7f3f6d777700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f4c0682f0 0x7f3f4c06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f6815dc90 tx=0x7f3f58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.037+0000 7f3f5e7fc700 1 -- 172.21.15.153:0/2327683428 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f64054140 con 0x7f3f680b2e70 2023-11-28T13:02:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.218+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3f68068690 con 0x7f3f680b2e70 2023-11-28T13:02:40.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.218+0000 7f3f5e7fc700 1 -- 172.21.15.153:0/2327683428 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f6402a020 con 0x7f3f680b2e70 2023-11-28T13:02:40.219 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:02:40.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f4c0682f0 msgr2=0x7f3f4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f4c0682f0 0x7f3f4c06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f6815dc90 tx=0x7f3f58008040 comp rx=0 tx=0).stop 2023-11-28T13:02:40.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680b2e70 msgr2=0x7f3f68157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680b2e70 0x7f3f68157e00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3f680b3aa0 tx=0x7f3f64004750 comp rx=0 tx=0).stop 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 shutdown_connections 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3f4c0682f0 0x7f3f4c06a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f680b2e70 0x7f3f68157e00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f680cdc90 0x7f3f681567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 --2- 172.21.15.153:0/2327683428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f680cd8c0 0x7f3f6813cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 >> 172.21.15.153:0/2327683428 conn(0x7f3f6806aed0 msgr2=0x7f3f6813e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 shutdown_connections 2023-11-28T13:02:40.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.220+0000 7f3f701dc700 1 -- 172.21.15.153:0/2327683428 wait complete. 2023-11-28T13:02:40.233 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:02:40.357 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.356+0000 7f6230198700 1 Processor -- start 2023-11-28T13:02:40.357 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.356+0000 7f6230198700 1 -- start start 2023-11-28T13:02:40.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f6230198700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6228140950 0x7f62280b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f6230198700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62280b2e70 0x7f622806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f6230198700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622813ff00 con 0x7f62280cd8c0 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f6230198700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62281404c0 con 0x7f6228140950 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f6230198700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6228140dc0 con 0x7f62280b2e70 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622df34700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f62280cd8c0 0x7f62280cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59876/0 (socket says 172.21.15.153:59876) 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622df34700 1 -- 172.21.15.153:0/1582547446 learned_addr learned my addr 172.21.15.153:0/1582547446 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622e735700 1 --2- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62280b2e70 0x7f622806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622d733700 1 --2- 172.21.15.153:0/1582547446 >> v2:172.21.15.177:3300/0 conn(0x7f6228140950 0x7f62280b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622cf32700 1 -- 172.21.15.153:0/1582547446 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241720194 0 0) 0x7f622813ff00 con 0x7f62280cd8c0 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622cf32700 1 -- 172.21.15.153:0/1582547446 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62140035b0 con 0x7f62280cd8c0 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622cf32700 1 -- 172.21.15.153:0/1582547446 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205547242 0 0) 0x7f62140035b0 con 0x7f62280cd8c0 2023-11-28T13:02:40.360 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622d733700 1 -- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62280b2e70 msgr2=0x7f622806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622d733700 1 --2- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62280b2e70 0x7f622806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622d733700 1 -- 172.21.15.153:0/1582547446 >> v1:172.21.15.153:6789/0 conn(0x7f62280cd8c0 legacy=0x7f62280cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622d733700 1 -- 172.21.15.153:0/1582547446 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622815aa80 con 0x7f6228140950 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.357+0000 7f622e735700 1 --2- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62280b2e70 0x7f622806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f622d733700 1 --2- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6228140950 0x7f62280b2930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6218009840 tx=0x7f6218009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2af66616847a4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f622cf32700 1 -- 172.21.15.153:0/1582547446 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240017f0 con 0x7f6228140950 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f622cf32700 1 -- 172.21.15.153:0/1582547446 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218004030 con 0x7f6228140950 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f622cf32700 1 -- 172.21.15.153:0/1582547446 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62180043c0 con 0x7f6228140950 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 -- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6228140950 msgr2=0x7f62280b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 --2- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6228140950 0x7f62280b2930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6218009840 tx=0x7f6218009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 -- 172.21.15.153:0/1582547446 shutdown_connections 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 --2- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62280b2e70 0x7f622806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 --2- 172.21.15.153:0/1582547446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6228140950 0x7f62280b2930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 -- 172.21.15.153:0/1582547446 >> 172.21.15.153:0/1582547446 conn(0x7f622806aed0 msgr2=0x7f622806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 -- 172.21.15.153:0/1582547446 shutdown_connections 2023-11-28T13:02:40.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.358+0000 7f6230198700 1 -- 172.21.15.153:0/1582547446 wait complete. 2023-11-28T13:02:40.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.359+0000 7f6230198700 1 Processor -- start 2023-11-28T13:02:40.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f6230198700 1 -- start start 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f6230198700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62280b2e70 0x7f6228143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f6230198700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62280cd8c0 0x7f622815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f6230198700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6228140950 0x7f622815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f6230198700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228143c80 con 0x7f62280cd8c0 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f6230198700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62280c87d0 con 0x7f62280b2e70 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f6230198700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62280c8ab0 con 0x7f6228140950 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f622d733700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62280cd8c0 0x7f622815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f622d733700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62280cd8c0 0x7f622815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41720/0 (socket says 172.21.15.153:41720) 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.360+0000 7f622d733700 1 -- 172.21.15.153:0/2750502080 learned_addr learned my addr 172.21.15.153:0/2750502080 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622e735700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6228140950 0x7f622815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622df34700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62280b2e70 0x7f6228143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622d733700 1 -- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6228140950 msgr2=0x7f622815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622d733700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6228140950 0x7f622815b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622d733700 1 -- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62280b2e70 msgr2=0x7f6228143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622d733700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62280b2e70 0x7f6228143740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622d733700 1 -- 172.21.15.153:0/2750502080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6220009a70 con 0x7f62280cd8c0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f622d733700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62280cd8c0 0x7f622815ac70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f621800a010 tx=0x7f6218004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f621e7fc700 1 -- 172.21.15.153:0/2750502080 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218003c00 con 0x7f62280cd8c0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f621e7fc700 1 -- 172.21.15.153:0/2750502080 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621801f070 con 0x7f62280cd8c0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62180094a0 con 0x7f62280cd8c0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f621e7fc700 1 -- 172.21.15.153:0/2750502080 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218011630 con 0x7f62280cd8c0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.361+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62280b35f0 con 0x7f62280cd8c0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.362+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62280b3310 con 0x7f62280cd8c0 2023-11-28T13:02:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.362+0000 7f621e7fc700 1 -- 172.21.15.153:0/2750502080 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f621802a020 con 0x7f62280cd8c0 2023-11-28T13:02:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.363+0000 7f621e7fc700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f620c0682f0 0x7f620c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.363+0000 7f621e7fc700 1 -- 172.21.15.153:0/2750502080 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62180870f0 con 0x7f62280cd8c0 2023-11-28T13:02:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.363+0000 7f622df34700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f620c0682f0 0x7f620c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.363+0000 7f622df34700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f620c0682f0 0x7f620c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f62240019a0 tx=0x7f6224009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.367+0000 7f621e7fc700 1 -- 172.21.15.153:0/2750502080 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62180538f0 con 0x7f62280cd8c0 2023-11-28T13:02:40.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.553+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6228140150 con 0x7f62280cd8c0 2023-11-28T13:02:40.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.553+0000 7f621e7fc700 1 -- 172.21.15.153:0/2750502080 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6218056f10 con 0x7f62280cd8c0 2023-11-28T13:02:40.554 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:02:40.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f620c0682f0 msgr2=0x7f620c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f620c0682f0 0x7f620c06a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f62240019a0 tx=0x7f6224009040 comp rx=0 tx=0).stop 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62280cd8c0 msgr2=0x7f622815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62280cd8c0 0x7f622815ac70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f621800a010 tx=0x7f6218004750 comp rx=0 tx=0).stop 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 shutdown_connections 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6228140950 0x7f622815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62280b2e70 0x7f6228143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f620c0682f0 0x7f620c06a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 --2- 172.21.15.153:0/2750502080 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62280cd8c0 0x7f622815ac70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.555+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 >> 172.21.15.153:0/2750502080 conn(0x7f622806aed0 msgr2=0x7f622815a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.556+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 shutdown_connections 2023-11-28T13:02:40.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.556+0000 7f6230198700 1 -- 172.21.15.153:0/2750502080 wait complete. 2023-11-28T13:02:40.568 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:02:40.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.690+0000 7fe0b7fcf700 1 Processor -- start 2023-11-28T13:02:40.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.690+0000 7fe0b7fcf700 1 -- start start 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.691+0000 7fe0b7fcf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe0b006b060 0x7fe0b006b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.691+0000 7fe0b7fcf700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00c9e10 con 0x7fe0b0160b10 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.691+0000 7fe0b7fcf700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b006efc0 con 0x7fe0b0160740 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.691+0000 7fe0b7fcf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b00ca720 con 0x7fe0b006b060 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.691+0000 7fe0b556a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe0b0160b10 0x7fe0b00cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59890/0 (socket says 172.21.15.153:59890) 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.691+0000 7fe0b556a700 1 -- 172.21.15.153:0/173178166 learned_addr learned my addr 172.21.15.153:0/173178166 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.691+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183955210 0 0) 0x7fe0b00c9e10 con 0x7fe0b0160b10 2023-11-28T13:02:40.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe09c0035b0 con 0x7fe0b0160b10 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b656c700 1 --2- 172.21.15.153:0/173178166 >> v2:172.21.15.177:3300/0 conn(0x7fe0b006b060 0x7fe0b006b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3368717942 0 0) 0x7fe09c0035b0 con 0x7fe0b0160b10 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 >> v2:172.21.15.177:3300/0 conn(0x7fe0b006b060 msgr2=0x7fe0b006b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 --2- 172.21.15.153:0/173178166 >> v2:172.21.15.177:3300/0 conn(0x7fe0b006b060 0x7fe0b006b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 >> v1:172.21.15.177:6789/0 conn(0x7fe0b0160740 legacy=0x7fe0b00cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00ca0d0 con 0x7fe0b0160b10 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fe0ac001670 con 0x7fe0b0160b10 2023-11-28T13:02:40.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b656c700 1 --2- 172.21.15.153:0/173178166 >> v2:172.21.15.177:3300/0 conn(0x7fe0b006b060 0x7fe0b006b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 >> v1:172.21.15.153:6789/0 conn(0x7fe0b0160b10 legacy=0x7fe0b00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe09c003a00 0x7fe09c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c004850 0x7fe09c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe09c005200 0x7fe09c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c160 con 0x7fe09c003a00 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c2d0 con 0x7fe09c005200 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c490 con 0x7fe09c004850 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b5d6b700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe09c003a00 0x7fe09c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b556a700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c004850 0x7fe09c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b656c700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe09c005200 0x7fe09c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b5d6b700 1 -- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c004850 msgr2=0x7fe09c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b5d6b700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c004850 0x7fe09c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b5d6b700 1 -- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe09c005200 msgr2=0x7fe09c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b5d6b700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe09c005200 0x7fe09c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b5d6b700 1 -- 172.21.15.153:0/173178166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c0035b0 con 0x7fe09c003a00 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b656c700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe09c005200 0x7fe09c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b656c700 1 -- 172.21.15.153:0/173178166 reap_dead start 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.692+0000 7fe0b5d6b700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe09c003a00 0x7fe09c003e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0007fd0 tx=0x7fe09c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488fa146790d02e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00052d0 con 0x7fe09c003a00 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a0005430 con 0x7fe09c003a00 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b4d69700 1 -- 172.21.15.153:0/173178166 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00038c0 con 0x7fe09c003a00 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/173178166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe09c003a00 msgr2=0x7fe09c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe09c003a00 0x7fe09c003e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0007fd0 tx=0x7fe09c005880 comp rx=0 tx=0).stop 2023-11-28T13:02:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/173178166 shutdown_connections 2023-11-28T13:02:40.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/173178166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe09c003a00 0x7fe09c003e10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/173178166 >> 172.21.15.153:0/173178166 conn(0x7fe0b00b12b0 msgr2=0x7fe0b00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/173178166 shutdown_connections 2023-11-28T13:02:40.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.693+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/173178166 wait complete. 2023-11-28T13:02:40.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.694+0000 7fe0b7fcf700 1 Processor -- start 2023-11-28T13:02:40.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.694+0000 7fe0b7fcf700 1 -- start start 2023-11-28T13:02:40.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b7fcf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c003a00 0x7fe0b00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b7fcf700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b0153c50 0x7fe0b01540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b7fcf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0b00c7830 0x7fe0b00c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b7fcf700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b006c7b0 con 0x7fe0b0153c50 2023-11-28T13:02:40.702 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b7fcf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0154600 con 0x7fe0b00c7830 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b7fcf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00c8680 con 0x7fe09c003a00 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b556a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b0153c50 0x7fe0b01540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b556a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b0153c50 0x7fe0b01540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41738/0 (socket says 172.21.15.153:41738) 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b556a700 1 -- 172.21.15.153:0/251528427 learned_addr learned my addr 172.21.15.153:0/251528427 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b5d6b700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c003a00 0x7fe0b00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.695+0000 7fe0b656c700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0b00c7830 0x7fe0b00c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b556a700 1 -- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c003a00 msgr2=0x7fe0b00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b556a700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c003a00 0x7fe0b00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b556a700 1 -- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0b00c7830 msgr2=0x7fe0b00c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b556a700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0b00c7830 0x7fe0b00c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b556a700 1 -- 172.21.15.153:0/251528427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b0160f60 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b656c700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0b00c7830 0x7fe0b00c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b556a700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b0153c50 0x7fe0b01540c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac007290 tx=0x7fe0ac0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0a67fc700 1 -- 172.21.15.153:0/251528427 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00f600 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0a67fc700 1 -- 172.21.15.153:0/251528427 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00f780 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b00ca3b0 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0a67fc700 1 -- 172.21.15.153:0/251528427 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00fb30 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.696+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b006ec00 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.697+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b006c8f0 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.697+0000 7fe0a67fc700 1 -- 172.21.15.153:0/251528427 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0ac00fc90 con 0x7fe0b0153c50 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.697+0000 7fe0a67fc700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe0940682f0 0x7fe09406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.698+0000 7fe0a67fc700 1 -- 172.21.15.153:0/251528427 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0ac085e30 con 0x7fe0b0153c50 2023-11-28T13:02:40.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.698+0000 7fe0b5d6b700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe0940682f0 0x7fe09406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.698+0000 7fe0b5d6b700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe0940682f0 0x7fe09406a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0000fb0 tx=0x7fe0a0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.705 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.702+0000 7fe0a67fc700 1 -- 172.21.15.153:0/251528427 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0ac052600 con 0x7fe0b0153c50 2023-11-28T13:02:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.885+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0b0066110 con 0x7fe0b0153c50 2023-11-28T13:02:40.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.885+0000 7fe0a67fc700 1 -- 172.21.15.153:0/251528427 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0ac055c20 con 0x7fe0b0153c50 2023-11-28T13:02:40.886 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:02:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe0940682f0 msgr2=0x7fe09406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe0940682f0 0x7fe09406a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0000fb0 tx=0x7fe0a0009030 comp rx=0 tx=0).stop 2023-11-28T13:02:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b0153c50 msgr2=0x7fe0b01540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b0153c50 0x7fe0b01540c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac007290 tx=0x7fe0ac0075a0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 shutdown_connections 2023-11-28T13:02:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0b00c7830 0x7fe0b00c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe09c003a00 0x7fe0b00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe0940682f0 0x7fe09406a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 --2- 172.21.15.153:0/251528427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0b0153c50 0x7fe0b01540c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 >> 172.21.15.153:0/251528427 conn(0x7fe0b00b12b0 msgr2=0x7fe0b0154f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 shutdown_connections 2023-11-28T13:02:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:40.887+0000 7fe0b7fcf700 1 -- 172.21.15.153:0/251528427 wait complete. 2023-11-28T13:02:40.900 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:41.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.020+0000 7f107b6f9700 1 Processor -- start 2023-11-28T13:02:41.021 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.020+0000 7f107b6f9700 1 -- start start 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f107b6f9700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10741658c0 0x7f1074167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f107b6f9700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107415d060 con 0x7f107406dee0 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f107b6f9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107415d340 con 0x7f1074071040 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f107b6f9700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1074161550 con 0x7f10741658c0 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f1079c96700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f10741658c0 0x7f1074167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f1078c94700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f107406dee0 0x7f107406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59908/0 (socket says 172.21.15.153:59908) 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f1078c94700 1 -- 172.21.15.153:0/2339841374 learned_addr learned my addr 172.21.15.153:0/2339841374 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f1079c96700 1 -- 172.21.15.153:0/2339841374 >> v1:172.21.15.177:6789/0 conn(0x7f1074071040 legacy=0x7f1074071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.023 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f1079c96700 1 -- 172.21.15.153:0/2339841374 >> v1:172.21.15.153:6789/0 conn(0x7f107406dee0 legacy=0x7f107406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f106bfff700 1 -- 172.21.15.153:0/2339841374 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463258852 0 0) 0x7f107415d060 con 0x7f107406dee0 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f1079c96700 1 -- 172.21.15.153:0/2339841374 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107415d620 con 0x7f10741658c0 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f1079c96700 1 --2- 172.21.15.153:0/2339841374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10741658c0 0x7f1074167ca0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d90 tx=0x7f106c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4825904713b15a20 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f106bfff700 1 -- 172.21.15.153:0/2339841374 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00e070 con 0x7f10741658c0 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f106bfff700 1 -- 172.21.15.153:0/2339841374 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c00bbd0 con 0x7f10741658c0 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.021+0000 7f106bfff700 1 -- 172.21.15.153:0/2339841374 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c012430 con 0x7f10741658c0 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.022+0000 7f107b6f9700 1 -- 172.21.15.153:0/2339841374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10741658c0 msgr2=0x7f1074167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.022+0000 7f107b6f9700 1 --2- 172.21.15.153:0/2339841374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10741658c0 0x7f1074167ca0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d90 tx=0x7f106c00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.022+0000 7f107b6f9700 1 -- 172.21.15.153:0/2339841374 shutdown_connections 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.022+0000 7f107b6f9700 1 --2- 172.21.15.153:0/2339841374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10741658c0 0x7f1074167ca0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.024 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.022+0000 7f107b6f9700 1 -- 172.21.15.153:0/2339841374 >> 172.21.15.153:0/2339841374 conn(0x7f10740d0000 msgr2=0x7f10740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.022+0000 7f107b6f9700 1 -- 172.21.15.153:0/2339841374 shutdown_connections 2023-11-28T13:02:41.025 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.022+0000 7f107b6f9700 1 -- 172.21.15.153:0/2339841374 wait complete. 2023-11-28T13:02:41.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 Processor -- start 2023-11-28T13:02:41.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 -- start start 2023-11-28T13:02:41.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f107406dee0 0x7f10740b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1074071040 0x7f10740b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10741658c0 0x7f10741f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741499d0 con 0x7f107406dee0 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1074146cb0 con 0x7f10741658c0 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.024+0000 7f107b6f9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10741490a0 con 0x7f1074071040 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f107406dee0 0x7f10740b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f107406dee0 0x7f10740b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41748/0 (socket says 172.21.15.153:41748) 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 -- 172.21.15.153:0/1616856136 learned_addr learned my addr 172.21.15.153:0/1616856136 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1078c94700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1074071040 0x7f10740b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079c96700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10741658c0 0x7f10741f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 -- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1074071040 msgr2=0x7f10740b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1074071040 0x7f10740b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 -- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10741658c0 msgr2=0x7f10741f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10741658c0 0x7f10741f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 -- 172.21.15.153:0/1616856136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c009a70 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1079495700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f107406dee0 0x7f10740b2e30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f107000c4d0 tx=0x7f107000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1069ffb700 1 -- 172.21.15.153:0/1616856136 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107000cea0 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1069ffb700 1 -- 172.21.15.153:0/1616856136 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10700052d0 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f1069ffb700 1 -- 172.21.15.153:0/1616856136 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070003680 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107415ccd0 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.025+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107415d900 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.026+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107406a270 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.027+0000 7f1069ffb700 1 -- 172.21.15.153:0/1616856136 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1070019030 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.027+0000 7f1069ffb700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10580682f0 0x7f105806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.027+0000 7f1069ffb700 1 -- 172.21.15.153:0/1616856136 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1070084d90 con 0x7f107406dee0 2023-11-28T13:02:41.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.027+0000 7f1078c94700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10580682f0 0x7f105806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.031 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.027+0000 7f1078c94700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10580682f0 0x7f105806a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10740c3450 tx=0x7f1064004a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.031+0000 7f1069ffb700 1 -- 172.21.15.153:0/1616856136 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f107004b5e0 con 0x7f107406dee0 2023-11-28T13:02:41.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.257+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10740720a0 con 0x7f107406dee0 2023-11-28T13:02:41.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.257+0000 7f1069ffb700 1 -- 172.21.15.153:0/1616856136 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f1070017090 con 0x7f107406dee0 2023-11-28T13:02:41.258 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:41.258 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:41.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10580682f0 msgr2=0x7f105806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10580682f0 0x7f105806a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10740c3450 tx=0x7f1064004a50 comp rx=0 tx=0).stop 2023-11-28T13:02:41.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f107406dee0 msgr2=0x7f10740b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f107406dee0 0x7f10740b2e30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f107000c4d0 tx=0x7f107000c7e0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 shutdown_connections 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10580682f0 0x7f105806a7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f107406dee0 0x7f10740b2e30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10741658c0 0x7f10741f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 --2- 172.21.15.153:0/1616856136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1074071040 0x7f10740b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 >> 172.21.15.153:0/1616856136 conn(0x7f10740d0000 msgr2=0x7f10740cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 shutdown_connections 2023-11-28T13:02:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.259+0000 7f107b6f9700 1 -- 172.21.15.153:0/1616856136 wait complete. 2023-11-28T13:02:41.262 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:41.272 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:41.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9099577700 1 Processor -- start 2023-11-28T13:02:41.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9099577700 1 -- start start 2023-11-28T13:02:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9099577700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9094160b10 0x7f90940cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9099577700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909406bbd0 con 0x7f9094160740 2023-11-28T13:02:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9099577700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90940c9e10 con 0x7f909406b060 2023-11-28T13:02:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9099577700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90940ca720 con 0x7f9094160b10 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9092ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9094160740 0x7f90940cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59920/0 (socket says 172.21.15.153:59920) 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f90927fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9094160b10 0x7f90940cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.391+0000 7f9092ffd700 1 -- 172.21.15.153:0/1058702974 learned_addr learned my addr 172.21.15.153:0/1058702974 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.392+0000 7f90927fc700 1 -- 172.21.15.153:0/1058702974 >> v1:172.21.15.177:6789/0 conn(0x7f909406b060 legacy=0x7f909406b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.392+0000 7f90927fc700 1 -- 172.21.15.153:0/1058702974 >> v1:172.21.15.153:6789/0 conn(0x7f9094160740 legacy=0x7f90940cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.392+0000 7f90927fc700 1 -- 172.21.15.153:0/1058702974 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909406efc0 con 0x7f9094160b10 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.392+0000 7f90927fc700 1 --2- 172.21.15.153:0/1058702974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940cd390 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9084009840 tx=0x7f9084009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921312b0fc165b8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.392+0000 7f9091ffb700 1 -- 172.21.15.153:0/1058702974 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084011830 con 0x7f9094160b10 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.392+0000 7f9091ffb700 1 -- 172.21.15.153:0/1058702974 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9084011990 con 0x7f9094160b10 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.392+0000 7f9091ffb700 1 -- 172.21.15.153:0/1058702974 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084011d20 con 0x7f9094160b10 2023-11-28T13:02:41.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.393+0000 7f9099577700 1 -- 172.21.15.153:0/1058702974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 msgr2=0x7f90940cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.393+0000 7f9099577700 1 --2- 172.21.15.153:0/1058702974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940cd390 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9084009840 tx=0x7f9084009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.393+0000 7f9099577700 1 -- 172.21.15.153:0/1058702974 shutdown_connections 2023-11-28T13:02:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.393+0000 7f9099577700 1 --2- 172.21.15.153:0/1058702974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940cd390 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.393+0000 7f9099577700 1 -- 172.21.15.153:0/1058702974 >> 172.21.15.153:0/1058702974 conn(0x7f90940b12b0 msgr2=0x7f90940b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.393+0000 7f9099577700 1 -- 172.21.15.153:0/1058702974 shutdown_connections 2023-11-28T13:02:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.393+0000 7f9099577700 1 -- 172.21.15.153:0/1058702974 wait complete. 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.394+0000 7f9099577700 1 Processor -- start 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.394+0000 7f9099577700 1 -- start start 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9099577700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f909406b060 0x7f9094148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9099577700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094160740 0x7f90940ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9099577700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9099577700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094148800 con 0x7f9094160b10 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9099577700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094148540 con 0x7f9094160740 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9099577700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909415c180 con 0x7f909406b060 2023-11-28T13:02:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41766/0 (socket says 172.21.15.153:41766) 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 -- 172.21.15.153:0/73425547 learned_addr learned my addr 172.21.15.153:0/73425547 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9092ffd700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f909406b060 0x7f9094148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90927fc700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094160740 0x7f90940ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 -- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f909406b060 msgr2=0x7f9094148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f909406b060 0x7f9094148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 -- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094160740 msgr2=0x7f90940ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094160740 0x7f90940ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 -- 172.21.15.153:0/73425547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840094a0 con 0x7f9094160b10 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90927fc700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094160740 0x7f90940ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f9092ffd700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f909406b060 0x7f9094148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f90937fe700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940ad910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f908c00cb80 tx=0x7f908c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f907b7fe700 1 -- 172.21.15.153:0/73425547 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c003c00 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.395+0000 7f907b7fe700 1 -- 172.21.15.153:0/73425547 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c003d60 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.396+0000 7f907b7fe700 1 -- 172.21.15.153:0/73425547 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00abb0 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.396+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90940ca3b0 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.396+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90940ca0d0 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.396+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90940cd8d0 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.397+0000 7f907b7fe700 1 -- 172.21.15.153:0/73425547 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f908c010050 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.397+0000 7f907b7fe700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90800682f0 0x7f908006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.397+0000 7f907b7fe700 1 -- 172.21.15.153:0/73425547 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f908c085b60 con 0x7f9094160b10 2023-11-28T13:02:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.397+0000 7f9092ffd700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90800682f0 0x7f908006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.401 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.398+0000 7f9092ffd700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90800682f0 0x7f908006a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9088005910 tx=0x7f9088009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.402 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.401+0000 7f907b7fe700 1 -- 172.21.15.153:0/73425547 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f908c0523c0 con 0x7f9094160b10 2023-11-28T13:02:41.587 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.587+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90941489b0 con 0x7f9094160b10 2023-11-28T13:02:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.588+0000 7f907b7fe700 1 -- 172.21.15.153:0/73425547 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18027 (secure 0 0 0) 0x7f90941489b0 con 0x7f9094160b10 2023-11-28T13:02:41.589 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:41.589 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":51,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:57.932997+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:54.715348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:55.897576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:41.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90800682f0 msgr2=0x7f908006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90800682f0 0x7f908006a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9088005910 tx=0x7f9088009040 comp rx=0 tx=0).stop 2023-11-28T13:02:41.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 msgr2=0x7f90940ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940ad910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f908c00cb80 tx=0x7f908c00cf40 comp rx=0 tx=0).stop 2023-11-28T13:02:41.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 shutdown_connections 2023-11-28T13:02:41.591 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90800682f0 0x7f908006a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9094160b10 0x7f90940ad910 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f909406b060 0x7f9094148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 --2- 172.21.15.153:0/73425547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094160740 0x7f90940ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 >> 172.21.15.153:0/73425547 conn(0x7f90940b12b0 msgr2=0x7f909406e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 shutdown_connections 2023-11-28T13:02:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.590+0000 7f9099577700 1 -- 172.21.15.153:0/73425547 wait complete. 2023-11-28T13:02:41.603 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:02:41.603 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:41.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.723+0000 7f9655ef1700 1 Processor -- start 2023-11-28T13:02:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.723+0000 7f9655ef1700 1 -- start start 2023-11-28T13:02:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.723+0000 7f9655ef1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9650154e70 0x7f96500cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.723+0000 7f9655ef1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f96500cf360 0x7f9650071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.723+0000 7f9655ef1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9650071980 con 0x7f9650071580 2023-11-28T13:02:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f9655ef1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9650155240 con 0x7f9650154e70 2023-11-28T13:02:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f9655ef1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965015b100 con 0x7f96500cf360 2023-11-28T13:02:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9650154e70 0x7f96500cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9650071580 0x7f965015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59938/0 (socket says 172.21.15.153:59938) 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964ffff700 1 -- 172.21.15.153:0/3088230084 learned_addr learned my addr 172.21.15.153:0/3088230084 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964f7fe700 1 -- 172.21.15.153:0/3088230084 >> v2:172.21.15.177:3300/0 conn(0x7f96500cf360 msgr2=0x7f9650071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964effd700 1 --2- 172.21.15.153:0/3088230084 >> v2:172.21.15.177:3300/0 conn(0x7f96500cf360 0x7f9650071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964e7fc700 1 -- 172.21.15.153:0/3088230084 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980309250 0 0) 0x7f9650071980 con 0x7f9650071580 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964f7fe700 1 --2- 172.21.15.153:0/3088230084 >> v2:172.21.15.177:3300/0 conn(0x7f96500cf360 0x7f9650071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964f7fe700 1 -- 172.21.15.153:0/3088230084 >> v1:172.21.15.153:6789/0 conn(0x7f9650071580 legacy=0x7f965015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964f7fe700 1 -- 172.21.15.153:0/3088230084 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965015b3e0 con 0x7f9650154e70 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964effd700 1 --2- 172.21.15.153:0/3088230084 >> v2:172.21.15.177:3300/0 conn(0x7f96500cf360 0x7f9650071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964f7fe700 1 --2- 172.21.15.153:0/3088230084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650154e70 0x7f96500cdbd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f96500c7060 tx=0x7f9640009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616588ca1c5a948a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964e7fc700 1 -- 172.21.15.153:0/3088230084 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640011830 con 0x7f9650154e70 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964e7fc700 1 -- 172.21.15.153:0/3088230084 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9640011990 con 0x7f9650154e70 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.724+0000 7f964e7fc700 1 -- 172.21.15.153:0/3088230084 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640011d20 con 0x7f9650154e70 2023-11-28T13:02:41.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 -- 172.21.15.153:0/3088230084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650154e70 msgr2=0x7f96500cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 --2- 172.21.15.153:0/3088230084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650154e70 0x7f96500cdbd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f96500c7060 tx=0x7f9640009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 -- 172.21.15.153:0/3088230084 shutdown_connections 2023-11-28T13:02:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 --2- 172.21.15.153:0/3088230084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650154e70 0x7f96500cdbd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 --2- 172.21.15.153:0/3088230084 >> v2:172.21.15.177:3300/0 conn(0x7f96500cf360 0x7f9650071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 -- 172.21.15.153:0/3088230084 >> 172.21.15.153:0/3088230084 conn(0x7f96500d0000 msgr2=0x7f96500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 -- 172.21.15.153:0/3088230084 shutdown_connections 2023-11-28T13:02:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.725+0000 7f9655ef1700 1 -- 172.21.15.153:0/3088230084 wait complete. 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.726+0000 7f9655ef1700 1 Processor -- start 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.726+0000 7f9655ef1700 1 -- start start 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f9655ef1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650071580 0x7f96500c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f9655ef1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96500cf360 0x7f96501f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f9655ef1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9650154e70 0x7f96500cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f9655ef1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965006d7d0 con 0x7f9650071580 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f9655ef1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965006dab0 con 0x7f9650154e70 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f9655ef1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965006dd90 con 0x7f96500cf360 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650071580 0x7f96500c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650071580 0x7f96500c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41776/0 (socket says 172.21.15.153:41776) 2023-11-28T13:02:41.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 -- 172.21.15.153:0/1311668494 learned_addr learned my addr 172.21.15.153:0/1311668494 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964effd700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96500cf360 0x7f96501f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964ffff700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9650154e70 0x7f96500cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 -- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96500cf360 msgr2=0x7f96501f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96500cf360 0x7f96501f1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 -- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9650154e70 msgr2=0x7f96500cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9650154e70 0x7f96500cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964f7fe700 1 -- 172.21.15.153:0/1311668494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400094a0 con 0x7f9650071580 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.727+0000 7f964ffff700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9650154e70 0x7f96500cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.728+0000 7f964f7fe700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650071580 0x7f96500c2be0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9640014040 tx=0x7f9640004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.728+0000 7f9637fff700 1 -- 172.21.15.153:0/1311668494 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400038a0 con 0x7f9650071580 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.728+0000 7f9637fff700 1 -- 172.21.15.153:0/1311668494 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964001e070 con 0x7f9650071580 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.728+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965015b9a0 con 0x7f9650071580 2023-11-28T13:02:41.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.728+0000 7f9637fff700 1 -- 172.21.15.153:0/1311668494 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640003c20 con 0x7f9650071580 2023-11-28T13:02:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.728+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965015b6c0 con 0x7f9650071580 2023-11-28T13:02:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.729+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96500af520 con 0x7f9650071580 2023-11-28T13:02:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.729+0000 7f9637fff700 1 -- 172.21.15.153:0/1311668494 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f964000f050 con 0x7f9650071580 2023-11-28T13:02:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.730+0000 7f9637fff700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f963c0682f0 0x7f963c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.730+0000 7f964effd700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f963c0682f0 0x7f963c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.730+0000 7f9637fff700 1 -- 172.21.15.153:0/1311668494 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9640086f10 con 0x7f9650071580 2023-11-28T13:02:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.730+0000 7f964effd700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f963c0682f0 0x7f963c06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9644006fd0 tx=0x7f9644008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.734+0000 7f9637fff700 1 -- 172.21.15.153:0/1311668494 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9640053680 con 0x7f9650071580 2023-11-28T13:02:41.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.956+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96500cae20 con 0x7f9650071580 2023-11-28T13:02:41.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.957+0000 7f9637fff700 1 -- 172.21.15.153:0/1311668494 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f9640015090 con 0x7f9650071580 2023-11-28T13:02:41.958 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:41.958 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:41.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f963c0682f0 msgr2=0x7f963c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f963c0682f0 0x7f963c06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9644006fd0 tx=0x7f9644008040 comp rx=0 tx=0).stop 2023-11-28T13:02:41.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650071580 msgr2=0x7f96500c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650071580 0x7f96500c2be0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9640014040 tx=0x7f9640004750 comp rx=0 tx=0).stop 2023-11-28T13:02:41.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 shutdown_connections 2023-11-28T13:02:41.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f963c0682f0 0x7f963c06a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9650071580 0x7f96500c2be0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9650154e70 0x7f96500cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 --2- 172.21.15.153:0/1311668494 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96500cf360 0x7f96501f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 >> 172.21.15.153:0/1311668494 conn(0x7f96500d0000 msgr2=0x7f965006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 shutdown_connections 2023-11-28T13:02:41.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:41.959+0000 7f9655ef1700 1 -- 172.21.15.153:0/1311668494 wait complete. 2023-11-28T13:02:41.961 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:41.972 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:42.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.091+0000 7f072c5a7700 1 Processor -- start 2023-11-28T13:02:42.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.091+0000 7f072c5a7700 1 -- start start 2023-11-28T13:02:42.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f072c5a7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 0x7f072415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f072c5a7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072406b420 con 0x7f07240d0a00 2023-11-28T13:02:42.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f072c5a7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072406b700 con 0x7f07240ce040 2023-11-28T13:02:42.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f072c5a7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072406ff60 con 0x7f07240b3040 2023-11-28T13:02:42.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729b42700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f07240d0a00 0x7f07240b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59950/0 (socket says 172.21.15.153:59950) 2023-11-28T13:02:42.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729b42700 1 -- 172.21.15.153:0/2838416817 learned_addr learned my addr 172.21.15.153:0/2838416817 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:42.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f072ab44700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 0x7f072415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848944402 0 0) 0x7f072406b420 con 0x7f07240d0a00 2023-11-28T13:02:42.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07100035b0 con 0x7f07240d0a00 2023-11-28T13:02:42.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92904851 0 0) 0x7f072406b700 con 0x7f07240ce040 2023-11-28T13:02:42.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f072406b420 con 0x7f07240ce040 2023-11-28T13:02:42.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320943763 0 0) 0x7f07100035b0 con 0x7f07240d0a00 2023-11-28T13:02:42.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 msgr2=0x7f072415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:42.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 0x7f072415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.092+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 >> v1:172.21.15.177:6789/0 conn(0x7f07240ce040 legacy=0x7f072406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072406b9e0 con 0x7f07240d0a00 2023-11-28T13:02:42.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f0714001670 con 0x7f07240d0a00 2023-11-28T13:02:42.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 >> v1:172.21.15.153:6789/0 conn(0x7f07240d0a00 legacy=0x7f07240b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0710003bc0 0x7f0710003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 0x7f0710004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f0710005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072406ff60 con 0x7f0710004d90 2023-11-28T13:02:42.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071000bac0 con 0x7f0710003bc0 2023-11-28T13:02:42.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071000bc50 con 0x7f07240b3040 2023-11-28T13:02:42.102 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072ab44700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f0710005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729b42700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 0x7f0710004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072a343700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0710003bc0 0x7f0710003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072ab44700 1 -- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 msgr2=0x7f0710004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:42.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072ab44700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 0x7f0710004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072ab44700 1 -- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0710003bc0 msgr2=0x7f0710003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:42.103 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072ab44700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0710003bc0 0x7f0710003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072ab44700 1 -- 172.21.15.153:0/2838416817 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100035b0 con 0x7f0710004d90 2023-11-28T13:02:42.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729b42700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07240b3040 0x7f0710004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:42.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f072ab44700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f0710005200 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0724071ce0 tx=0x7f0710005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8009f1e299a921da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.104 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00ebd0 con 0x7f0710004d90 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c00f040 con 0x7f0710004d90 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.093+0000 7f0729341700 1 -- 172.21.15.153:0/2838416817 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c014480 con 0x7f0710004d90 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.094+0000 7f072c5a7700 1 -- 172.21.15.153:0/2838416817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 msgr2=0x7f0710005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.094+0000 7f072c5a7700 1 --2- 172.21.15.153:0/2838416817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f0710005200 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0724071ce0 tx=0x7f0710005880 comp rx=0 tx=0).stop 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.094+0000 7f072ab44700 1 -- 172.21.15.153:0/2838416817 reap_dead start 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.094+0000 7f072c5a7700 1 -- 172.21.15.153:0/2838416817 shutdown_connections 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.094+0000 7f072c5a7700 1 -- 172.21.15.153:0/2838416817 >> 172.21.15.153:0/2838416817 conn(0x7f072406aed0 msgr2=0x7f072406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.094+0000 7f072c5a7700 1 -- 172.21.15.153:0/2838416817 shutdown_connections 2023-11-28T13:02:42.105 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.094+0000 7f072c5a7700 1 -- 172.21.15.153:0/2838416817 wait complete. 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.095+0000 7f072c5a7700 1 Processor -- start 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.095+0000 7f072c5a7700 1 -- start start 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072c5a7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f07240c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072c5a7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0724152da0 0x7f07240ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072c5a7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072413ee20 0x7f072413f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072c5a7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0724153260 con 0x7f0710004d90 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072c5a7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07240c6040 con 0x7f072413ee20 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072c5a7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07240c6320 con 0x7f0724152da0 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f07240c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f07240c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41796/0 (socket says 172.21.15.153:41796) 2023-11-28T13:02:42.106 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 -- 172.21.15.153:0/2050011855 learned_addr learned my addr 172.21.15.153:0/2050011855 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 -- 172.21.15.153:0/2050011855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0724152da0 msgr2=0x7f07240ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0724152da0 0x7f07240ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 -- 172.21.15.153:0/2050011855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072413ee20 msgr2=0x7f072413f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072ab44700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072413ee20 0x7f072413f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072413ee20 0x7f072413f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 -- 172.21.15.153:0/2050011855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072406bfa0 con 0x7f0710004d90 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.096+0000 7f072a343700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f07240c6b20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f072000f7a0 tx=0x7f072000fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.097+0000 7f071affd700 1 -- 172.21.15.153:0/2050011855 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200052e0 con 0x7f0710004d90 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.097+0000 7f071affd700 1 -- 172.21.15.153:0/2050011855 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0720005440 con 0x7f0710004d90 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.097+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072406bcc0 con 0x7f0710004d90 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.097+0000 7f071affd700 1 -- 172.21.15.153:0/2050011855 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200055b0 con 0x7f0710004d90 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.097+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072415df10 con 0x7f0710004d90 2023-11-28T13:02:42.107 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.098+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07240c6810 con 0x7f0710004d90 2023-11-28T13:02:42.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.098+0000 7f071affd700 1 -- 172.21.15.153:0/2050011855 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f07200038f0 con 0x7f0710004d90 2023-11-28T13:02:42.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.099+0000 7f071affd700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f07080682f0 0x7f070806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.099+0000 7f071affd700 1 -- 172.21.15.153:0/2050011855 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0720086700 con 0x7f0710004d90 2023-11-28T13:02:42.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.099+0000 7f0729b42700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f07080682f0 0x7f070806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.099+0000 7f0729b42700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f07080682f0 0x7f070806a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0714000c00 tx=0x7f0714001110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.109 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.103+0000 7f071affd700 1 -- 172.21.15.153:0/2050011855 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0720053000 con 0x7f0710004d90 2023-11-28T13:02:42.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.329+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0724068690 con 0x7f0710004d90 2023-11-28T13:02:42.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.330+0000 7f071affd700 1 -- 172.21.15.153:0/2050011855 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f0720056620 con 0x7f0710004d90 2023-11-28T13:02:42.331 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:42.331 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:56.931710+0000","modified":"2023-11-28T13:01:59.942702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3478129537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3478129537},{"type":"v1","addr":"172.21.15.177:6835","nonce":3478129537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:42.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f07080682f0 msgr2=0x7f070806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f07080682f0 0x7f070806a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0714000c00 tx=0x7f0714001110 comp rx=0 tx=0).stop 2023-11-28T13:02:42.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 msgr2=0x7f07240c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f07240c6b20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f072000f7a0 tx=0x7f072000fb60 comp rx=0 tx=0).stop 2023-11-28T13:02:42.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 shutdown_connections 2023-11-28T13:02:42.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f07080682f0 0x7f070806a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.333 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0710004d90 0x7f07240c6b20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072413ee20 0x7f072413f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 --2- 172.21.15.153:0/2050011855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0724152da0 0x7f07240ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 >> 172.21.15.153:0/2050011855 conn(0x7f072406aed0 msgr2=0x7f07240cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:42.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 shutdown_connections 2023-11-28T13:02:42.334 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.332+0000 7f072c5a7700 1 -- 172.21.15.153:0/2050011855 wait complete. 2023-11-28T13:02:42.334 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2023-11-28T13:02:42.345 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:42.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.466+0000 7fc59403a700 1 Processor -- start 2023-11-28T13:02:42.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.466+0000 7fc59403a700 1 -- start start 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc59403a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc58c06f850 0x7fc58c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc59403a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c13ff00 con 0x7fc58c140950 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc59403a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c1404c0 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc59403a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c140dc0 con 0x7fc58c06f850 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc5925d7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc58c06f850 0x7fc58c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc5915d5700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc58c140950 0x7fc58c0b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:59970/0 (socket says 172.21.15.153:59970) 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc5915d5700 1 -- 172.21.15.153:0/3915183724 learned_addr learned my addr 172.21.15.153:0/3915183724 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc590dd4700 1 -- 172.21.15.153:0/3915183724 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077846294 0 0) 0x7fc58c13ff00 con 0x7fc58c140950 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc590dd4700 1 -- 172.21.15.153:0/3915183724 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5780035b0 con 0x7fc58c140950 2023-11-28T13:02:42.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc5925d7700 1 -- 172.21.15.153:0/3915183724 >> v1:172.21.15.177:6789/0 conn(0x7fc58c0cd8c0 legacy=0x7fc58c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc5925d7700 1 -- 172.21.15.153:0/3915183724 >> v1:172.21.15.153:6789/0 conn(0x7fc58c140950 legacy=0x7fc58c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc5925d7700 1 -- 172.21.15.153:0/3915183724 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c15aa80 con 0x7fc58c06f850 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc5925d7700 1 --2- 172.21.15.153:0/3915183724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c06f850 0x7fc58c06fcf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc584009d90 tx=0x7fc58400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47607c9981a7e0af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc590dd4700 1 -- 172.21.15.153:0/3915183724 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400e070 con 0x7fc58c06f850 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc590dd4700 1 -- 172.21.15.153:0/3915183724 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58400bae0 con 0x7fc58c06f850 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.467+0000 7fc590dd4700 1 -- 172.21.15.153:0/3915183724 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400be90 con 0x7fc58c06f850 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.468+0000 7fc59403a700 1 -- 172.21.15.153:0/3915183724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c06f850 msgr2=0x7fc58c06fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.468+0000 7fc59403a700 1 --2- 172.21.15.153:0/3915183724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c06f850 0x7fc58c06fcf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc584009d90 tx=0x7fc58400b330 comp rx=0 tx=0).stop 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.468+0000 7fc59403a700 1 -- 172.21.15.153:0/3915183724 shutdown_connections 2023-11-28T13:02:42.470 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.468+0000 7fc59403a700 1 --2- 172.21.15.153:0/3915183724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c06f850 0x7fc58c06fcf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.468+0000 7fc59403a700 1 -- 172.21.15.153:0/3915183724 >> 172.21.15.153:0/3915183724 conn(0x7fc58c06aed0 msgr2=0x7fc58c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:42.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.468+0000 7fc59403a700 1 -- 172.21.15.153:0/3915183724 shutdown_connections 2023-11-28T13:02:42.471 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.468+0000 7fc59403a700 1 -- 172.21.15.153:0/3915183724 wait complete. 2023-11-28T13:02:42.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.469+0000 7fc59403a700 1 Processor -- start 2023-11-28T13:02:42.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.469+0000 7fc59403a700 1 -- start start 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc59403a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc58c06f850 0x7fc58c0b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc59403a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c0cd8c0 0x7fc58c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc59403a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc58c140950 0x7fc58c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc59403a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0b5b70 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc59403a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0b5f90 con 0x7fc58c140950 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc59403a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0b4c20 con 0x7fc58c06f850 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c0cd8c0 0x7fc58c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c0cd8c0 0x7fc58c0b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41812/0 (socket says 172.21.15.153:41812) 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 -- 172.21.15.153:0/4062373803 learned_addr learned my addr 172.21.15.153:0/4062373803 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:42.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5925d7700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc58c140950 0x7fc58c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc591dd6700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc58c06f850 0x7fc58c0b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 -- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc58c06f850 msgr2=0x7fc58c0b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc58c06f850 0x7fc58c0b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 -- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc58c140950 msgr2=0x7fc58c158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc58c140950 0x7fc58c158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 -- 172.21.15.153:0/4062373803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc584009a70 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.470+0000 7fc5915d5700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c0cd8c0 0x7fc58c0b46e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c430 tx=0x7fc57c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.471+0000 7fc5827fc700 1 -- 172.21.15.153:0/4062373803 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c005d70 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.471+0000 7fc5827fc700 1 -- 172.21.15.153:0/4062373803 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c0070d0 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.471+0000 7fc5827fc700 1 -- 172.21.15.153:0/4062373803 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c007480 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.471+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c0b3310 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.471+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c0b3040 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.471+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c0b35f0 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.472+0000 7fc5827fc700 1 -- 172.21.15.153:0/4062373803 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc57c019030 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.472+0000 7fc5827fc700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc5700682f0 0x7fc57006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.472+0000 7fc5827fc700 1 -- 172.21.15.153:0/4062373803 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc57c084f90 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.472+0000 7fc591dd6700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc5700682f0 0x7fc57006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.473+0000 7fc591dd6700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc5700682f0 0x7fc57006a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc588004f80 tx=0x7fc588009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.477+0000 7fc5827fc700 1 -- 172.21.15.153:0/4062373803 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc57c051820 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.660+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc58c0b5cb0 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.661+0000 7fc5827fc700 1 -- 172.21.15.153:0/4062373803 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18027 (secure 0 0 0) 0x7fc58c0b5cb0 con 0x7fc58c0cd8c0 2023-11-28T13:02:42.662 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:42.662 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":51,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:01:57.932997+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:54.715348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:55.897576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:42.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc5700682f0 msgr2=0x7fc57006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc5700682f0 0x7fc57006a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc588004f80 tx=0x7fc588009040 comp rx=0 tx=0).stop 2023-11-28T13:02:42.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c0cd8c0 msgr2=0x7fc58c0b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c0cd8c0 0x7fc58c0b46e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c430 tx=0x7fc57c00c740 comp rx=0 tx=0).stop 2023-11-28T13:02:42.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 shutdown_connections 2023-11-28T13:02:42.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc58c140950 0x7fc58c158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc58c06f850 0x7fc58c0b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc5700682f0 0x7fc57006a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 --2- 172.21.15.153:0/4062373803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc58c0cd8c0 0x7fc58c0b46e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 >> 172.21.15.153:0/4062373803 conn(0x7fc58c06aed0 msgr2=0x7fc58c13e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:42.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 shutdown_connections 2023-11-28T13:02:42.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.663+0000 7fc59403a700 1 -- 172.21.15.153:0/4062373803 wait complete. 2023-11-28T13:02:42.676 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:02:42.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.796+0000 7fd195bc8700 1 Processor -- start 2023-11-28T13:02:42.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.796+0000 7fd195bc8700 1 -- start start 2023-11-28T13:02:42.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd195bc8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd19015c9b0 0x7fd1900b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd195bc8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd190161340 0x7fd1900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd195bc8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1900d2320 0x7fd1900cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd195bc8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900b5800 con 0x7fd190161340 2023-11-28T13:02:42.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd195bc8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900d1950 con 0x7fd19015c9b0 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd195bc8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900b5520 con 0x7fd1900d2320 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd190161340 0x7fd1900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd190161340 0x7fd1900d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41820/0 (socket says 172.21.15.153:41820) 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 -- 172.21.15.153:0/1124943122 learned_addr learned my addr 172.21.15.153:0/1124943122 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18f7fe700 1 --2- 172.21.15.153:0/1124943122 >> v2:172.21.15.177:3300/0 conn(0x7fd19015c9b0 0x7fd1900b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18ffff700 1 --2- 172.21.15.153:0/1124943122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1900d2320 0x7fd1900cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 -- 172.21.15.153:0/1124943122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1900d2320 msgr2=0x7fd1900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 --2- 172.21.15.153:0/1124943122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1900d2320 0x7fd1900cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 -- 172.21.15.153:0/1124943122 >> v2:172.21.15.177:3300/0 conn(0x7fd19015c9b0 msgr2=0x7fd1900b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 --2- 172.21.15.153:0/1124943122 >> v2:172.21.15.177:3300/0 conn(0x7fd19015c9b0 0x7fd1900b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 -- 172.21.15.153:0/1124943122 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900d1300 con 0x7fd190161340 2023-11-28T13:02:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18ffff700 1 --2- 172.21.15.153:0/1124943122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1900d2320 0x7fd1900cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18f7fe700 1 --2- 172.21.15.153:0/1124943122 >> v2:172.21.15.177:3300/0 conn(0x7fd19015c9b0 0x7fd1900b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.797+0000 7fd18effd700 1 --2- 172.21.15.153:0/1124943122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190161340 0x7fd1900d1de0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd190067460 tx=0x7fd180009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621efdef38e860fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd18e7fc700 1 -- 172.21.15.153:0/1124943122 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000b870 con 0x7fd190161340 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd18e7fc700 1 -- 172.21.15.153:0/1124943122 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000b9d0 con 0x7fd190161340 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd18e7fc700 1 -- 172.21.15.153:0/1124943122 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000bd60 con 0x7fd190161340 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 -- 172.21.15.153:0/1124943122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190161340 msgr2=0x7fd1900d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 --2- 172.21.15.153:0/1124943122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190161340 0x7fd1900d1de0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd190067460 tx=0x7fd180009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 -- 172.21.15.153:0/1124943122 shutdown_connections 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 --2- 172.21.15.153:0/1124943122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1900d2320 0x7fd1900cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 --2- 172.21.15.153:0/1124943122 >> v2:172.21.15.177:3300/0 conn(0x7fd19015c9b0 0x7fd1900b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 --2- 172.21.15.153:0/1124943122 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd190161340 0x7fd1900d1de0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 -- 172.21.15.153:0/1124943122 >> 172.21.15.153:0/1124943122 conn(0x7fd190072600 msgr2=0x7fd1900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:42.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 -- 172.21.15.153:0/1124943122 shutdown_connections 2023-11-28T13:02:42.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.798+0000 7fd195bc8700 1 -- 172.21.15.153:0/1124943122 wait complete. 2023-11-28T13:02:42.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.800+0000 7fd195bc8700 1 Processor -- start 2023-11-28T13:02:42.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.800+0000 7fd195bc8700 1 -- start start 2023-11-28T13:02:42.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd195bc8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1900d2320 0x7fd190156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd195bc8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd19015c9b0 0x7fd1901462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd195bc8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd190161340 0x7fd1901f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd195bc8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190156880 con 0x7fd19015c9b0 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd195bc8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd190146820 con 0x7fd1900d2320 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd195bc8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd190156ae0 con 0x7fd190161340 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd19015c9b0 0x7fd1901462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd19015c9b0 0x7fd1901462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41828/0 (socket says 172.21.15.153:41828) 2023-11-28T13:02:42.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 -- 172.21.15.153:0/3422752540 learned_addr learned my addr 172.21.15.153:0/3422752540 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18f7fe700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1900d2320 0x7fd190156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 -- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd190161340 msgr2=0x7fd1901f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18ffff700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd190161340 0x7fd1901f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd190161340 0x7fd1901f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 -- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1900d2320 msgr2=0x7fd190156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1900d2320 0x7fd190156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 -- 172.21.15.153:0/3422752540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180009550 con 0x7fd19015c9b0 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18f7fe700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1900d2320 0x7fd190156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.801+0000 7fd18effd700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd19015c9b0 0x7fd1901462e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd180000c00 tx=0x7fd180004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.802+0000 7fd177fff700 1 -- 172.21.15.153:0/3422752540 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180004540 con 0x7fd19015c9b0 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.802+0000 7fd177fff700 1 -- 172.21.15.153:0/3422752540 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1800036a0 con 0x7fd19015c9b0 2023-11-28T13:02:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.802+0000 7fd177fff700 1 -- 172.21.15.153:0/3422752540 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180003a50 con 0x7fd19015c9b0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.802+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900b5130 con 0x7fd19015c9b0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.802+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900d15e0 con 0x7fd19015c9b0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.803+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1900b52c0 con 0x7fd19015c9b0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.803+0000 7fd177fff700 1 -- 172.21.15.153:0/3422752540 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd18000f050 con 0x7fd19015c9b0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.803+0000 7fd177fff700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd17c0682f0 0x7fd17c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.804+0000 7fd177fff700 1 -- 172.21.15.153:0/3422752540 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd180087570 con 0x7fd19015c9b0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.804+0000 7fd18f7fe700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd17c0682f0 0x7fd17c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.804+0000 7fd18f7fe700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd17c0682f0 0x7fd17c06a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1840099e0 tx=0x7fd184008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:42.808+0000 7fd177fff700 1 -- 172.21.15.153:0/3422752540 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd180053dd0 con 0x7fd19015c9b0 2023-11-28T13:02:43.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.032+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd1900745c0 con 0x7fd19015c9b0 2023-11-28T13:02:43.449 INFO:tasks.ceph.mon.a.smithi153.stderr:2023-11-28T13:02:43.448+0000 7feb34d4f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:02:43.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.452+0000 7fd177fff700 1 -- 172.21.15.153:0/3422752540 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd1800573f0 con 0x7fd19015c9b0 2023-11-28T13:02:43.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd17c0682f0 msgr2=0x7fd17c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:43.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd17c0682f0 0x7fd17c06a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1840099e0 tx=0x7fd184008040 comp rx=0 tx=0).stop 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd19015c9b0 msgr2=0x7fd1901462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd19015c9b0 0x7fd1901462e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd180000c00 tx=0x7fd180004750 comp rx=0 tx=0).stop 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 shutdown_connections 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd190161340 0x7fd1901f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1900d2320 0x7fd190156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd17c0682f0 0x7fd17c06a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 --2- 172.21.15.153:0/3422752540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd19015c9b0 0x7fd1901462e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 >> 172.21.15.153:0/3422752540 conn(0x7fd190072600 msgr2=0x7fd19015a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:43.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 shutdown_connections 2023-11-28T13:02:43.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.454+0000 7fd195bc8700 1 -- 172.21.15.153:0/3422752540 wait complete. 2023-11-28T13:02:43.457 INFO:teuthology.orchestra.run.smithi153.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:02:43.457 INFO:tasks.ceph.mds.b.smithi177.stderr: -6634> 2023-11-28T13:01:57.522+0000 7fb3262b3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:02:43.461 INFO:tasks.ceph.mds.b.smithi177.stderr: -2978> 2023-11-28T13:02:05.275+0000 7fb322aac700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5706=0-4194304@1} caps={5706=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5706 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ad36727700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:02:43.470 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:02:43.490 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T13:02:43.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.593+0000 7f17daf97700 1 Processor -- start 2023-11-28T13:02:43.594 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.593+0000 7f17daf97700 1 -- start start 2023-11-28T13:02:43.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.593+0000 7f17daf97700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f17d40ce040 0x7f17d415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:43.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.593+0000 7f17daf97700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f17d40d0a00 0x7f17d406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:43.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.593+0000 7f17daf97700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d406b420 con 0x7f17d40b2930 2023-11-28T13:02:43.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.593+0000 7f17daf97700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d406b700 con 0x7f17d40ce040 2023-11-28T13:02:43.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.593+0000 7f17daf97700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d406b9e0 con 0x7f17d40d0a00 2023-11-28T13:02:43.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f17d40ce040 0x7f17d415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:43.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f17d40ce040 0x7f17d415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41832/0 (socket says 172.21.15.153:41832) 2023-11-28T13:02:43.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 -- 172.21.15.153:0/413449784 learned_addr learned my addr 172.21.15.153:0/413449784 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:43.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d3fff700 1 --2- 172.21.15.153:0/413449784 >> v2:172.21.15.177:3300/0 conn(0x7f17d40d0a00 0x7f17d406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:43.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 -- 172.21.15.153:0/413449784 >> v2:172.21.15.177:3300/0 conn(0x7f17d40d0a00 msgr2=0x7f17d406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:43.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 --2- 172.21.15.153:0/413449784 >> v2:172.21.15.177:3300/0 conn(0x7f17d40d0a00 0x7f17d406f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 -- 172.21.15.153:0/413449784 >> v1:172.21.15.177:6789/0 conn(0x7f17d40b2930 legacy=0x7f17d406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:43.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 -- 172.21.15.153:0/413449784 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d406bcc0 con 0x7f17d40ce040 2023-11-28T13:02:43.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d3fff700 1 --2- 172.21.15.153:0/413449784 >> v2:172.21.15.177:3300/0 conn(0x7f17d40d0a00 0x7f17d406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:43.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d8d33700 1 --2- 172.21.15.153:0/413449784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40ce040 0x7f17d415df10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f17d406ed50 tx=0x7f17c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce768bb557c5a73c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:43.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d37fe700 1 -- 172.21.15.153:0/413449784 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8005e30 con 0x7f17d40ce040 2023-11-28T13:02:43.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d37fe700 1 -- 172.21.15.153:0/413449784 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c8004030 con 0x7f17d40ce040 2023-11-28T13:02:43.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.594+0000 7f17d37fe700 1 -- 172.21.15.153:0/413449784 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80043e0 con 0x7f17d40ce040 2023-11-28T13:02:43.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 -- 172.21.15.153:0/413449784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40ce040 msgr2=0x7f17d415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:43.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 --2- 172.21.15.153:0/413449784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40ce040 0x7f17d415df10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f17d406ed50 tx=0x7f17c8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 -- 172.21.15.153:0/413449784 shutdown_connections 2023-11-28T13:02:43.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 --2- 172.21.15.153:0/413449784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40ce040 0x7f17d415df10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 --2- 172.21.15.153:0/413449784 >> v2:172.21.15.177:3300/0 conn(0x7f17d40d0a00 0x7f17d406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 -- 172.21.15.153:0/413449784 >> 172.21.15.153:0/413449784 conn(0x7f17d406aed0 msgr2=0x7f17d406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:43.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 -- 172.21.15.153:0/413449784 shutdown_connections 2023-11-28T13:02:43.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.595+0000 7f17daf97700 1 -- 172.21.15.153:0/413449784 wait complete. 2023-11-28T13:02:43.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.596+0000 7f17daf97700 1 Processor -- start 2023-11-28T13:02:43.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.596+0000 7f17daf97700 1 -- start start 2023-11-28T13:02:43.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17daf97700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d40b2930 0x7f17d41f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:43.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17daf97700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d40ce040 0x7f17d4155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17daf97700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40d0a00 0x7f17d40c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17daf97700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40c6bc0 con 0x7f17d40d0a00 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17daf97700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40c6690 con 0x7f17d40b2930 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17daf97700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40c5680 con 0x7f17d40ce040 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40d0a00 0x7f17d40c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40d0a00 0x7f17d40c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41840/0 (socket says 172.21.15.153:41840) 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 -- 172.21.15.153:0/2016762776 learned_addr learned my addr 172.21.15.153:0/2016762776 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d3fff700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d40ce040 0x7f17d4155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:43.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d8d33700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d40b2930 0x7f17d41f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 -- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d40ce040 msgr2=0x7f17d4155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d40ce040 0x7f17d4155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 -- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d40b2930 msgr2=0x7f17d41f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d40b2930 0x7f17d41f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 -- 172.21.15.153:0/2016762776 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8009500 con 0x7f17d40d0a00 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d3fff700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d40ce040 0x7f17d4155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d8d33700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d40b2930 0x7f17d41f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d9534700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40d0a00 0x7f17d40c6d70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00cb50 tx=0x7f17cc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.597+0000 7f17d17fa700 1 -- 172.21.15.153:0/2016762776 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc003c00 con 0x7f17d40d0a00 2023-11-28T13:02:43.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.598+0000 7f17d17fa700 1 -- 172.21.15.153:0/2016762776 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc003d60 con 0x7f17d40d0a00 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.598+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d406fd90 con 0x7f17d40d0a00 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.598+0000 7f17d17fa700 1 -- 172.21.15.153:0/2016762776 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00ab70 con 0x7f17d40d0a00 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.598+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d406bfa0 con 0x7f17d40d0a00 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.599+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d406ff20 con 0x7f17d40d0a00 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.599+0000 7f17d17fa700 1 -- 172.21.15.153:0/2016762776 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f17cc010050 con 0x7f17d40d0a00 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.599+0000 7f17d17fa700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f17b80682f0 0x7f17b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.599+0000 7f17d17fa700 1 -- 172.21.15.153:0/2016762776 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f17cc085bd0 con 0x7f17d40d0a00 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.600+0000 7f17d8d33700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f17b80682f0 0x7f17b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:43.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.600+0000 7f17d8d33700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f17b80682f0 0x7f17b806a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17d406ed50 tx=0x7f17c8008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:43.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.604+0000 7f17d17fa700 1 -- 172.21.15.153:0/2016762776 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f17cc0523d0 con 0x7f17d40d0a00 2023-11-28T13:02:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:43.830+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f17d40c57f0 con 0x7f17d40d0a00 2023-11-28T13:02:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.463+0000 7f17d17fa700 1 -- 172.21.15.153:0/2016762776 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17cc0559f0 con 0x7f17d40d0a00 2023-11-28T13:02:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f17b80682f0 msgr2=0x7f17b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f17b80682f0 0x7f17b806a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17d406ed50 tx=0x7f17c8008070 comp rx=0 tx=0).stop 2023-11-28T13:02:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40d0a00 msgr2=0x7f17d40c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40d0a00 0x7f17d40c6d70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00cb50 tx=0x7f17cc00cf10 comp rx=0 tx=0).stop 2023-11-28T13:02:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 shutdown_connections 2023-11-28T13:02:44.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f17b80682f0 0x7f17b806a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:44.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17d40d0a00 0x7f17d40c6d70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:44.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d40b2930 0x7f17d41f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:44.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 --2- 172.21.15.153:0/2016762776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d40ce040 0x7f17d4155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:44.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 >> 172.21.15.153:0/2016762776 conn(0x7f17d406aed0 msgr2=0x7f17d415cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:44.468 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 shutdown_connections 2023-11-28T13:02:44.469 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.466+0000 7f17daf97700 1 -- 172.21.15.153:0/2016762776 wait complete. 2023-11-28T13:02:44.481 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:02:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.611+0000 7fade83e8700 1 Processor -- start 2023-11-28T13:02:44.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.611+0000 7fade83e8700 1 -- start start 2023-11-28T13:02:44.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade83e8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade0070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:44.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade83e8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade00713c0 con 0x7fade006a980 2023-11-28T13:02:44.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade83e8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade00716a0 con 0x7fade00af430 2023-11-28T13:02:44.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade83e8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade0071980 con 0x7fade00af060 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade6985700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fade006a980 0x7fade00ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60014/0 (socket says 172.21.15.153:60014) 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade6985700 1 -- 172.21.15.153:0/2640848099 learned_addr learned my addr 172.21.15.153:0/2640848099 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade6184700 1 --2- 172.21.15.153:0/2640848099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade0070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260469712 0 0) 0x7fade00713c0 con 0x7fade006a980 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc0035b0 con 0x7fade006a980 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303520267 0 0) 0x7fade00716a0 con 0x7fade00af430 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade00713c0 con 0x7fade00af430 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893900297 0 0) 0x7fadcc0035b0 con 0x7fade006a980 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade6184700 1 -- 172.21.15.153:0/2640848099 >> v1:172.21.15.177:6789/0 conn(0x7fade00af430 legacy=0x7fade006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade6184700 1 -- 172.21.15.153:0/2640848099 >> v1:172.21.15.153:6789/0 conn(0x7fade006a980 legacy=0x7fade00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:44.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.612+0000 7fade6184700 1 -- 172.21.15.153:0/2640848099 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0071c60 con 0x7fade00af060 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade6184700 1 --2- 172.21.15.153:0/2640848099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade0070d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faddc009870 tx=0x7faddc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83ec0c89ecdc1c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80019f0 con 0x7fade00af060 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc004030 con 0x7fade00af060 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade5182700 1 -- 172.21.15.153:0/2640848099 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc0043a0 con 0x7fade00af060 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade83e8700 1 -- 172.21.15.153:0/2640848099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 msgr2=0x7fade0070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade83e8700 1 --2- 172.21.15.153:0/2640848099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade0070d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faddc009870 tx=0x7faddc009b80 comp rx=0 tx=0).stop 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade83e8700 1 -- 172.21.15.153:0/2640848099 shutdown_connections 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade83e8700 1 --2- 172.21.15.153:0/2640848099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade0070d50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade83e8700 1 -- 172.21.15.153:0/2640848099 >> 172.21.15.153:0/2640848099 conn(0x7fade006b380 msgr2=0x7fade006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade83e8700 1 -- 172.21.15.153:0/2640848099 shutdown_connections 2023-11-28T13:02:44.616 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.613+0000 7fade83e8700 1 -- 172.21.15.153:0/2640848099 wait complete. 2023-11-28T13:02:44.619 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.615+0000 7fade83e8700 1 Processor -- start 2023-11-28T13:02:44.619 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.615+0000 7fade83e8700 1 -- start start 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade83e8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade006a980 0x7fade00c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade83e8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade00c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade83e8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade00af430 0x7fade015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade83e8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade00af850 con 0x7fade00af430 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade83e8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade00ca210 con 0x7fade006a980 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade83e8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00afa30 con 0x7fade00af060 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade00af430 0x7fade015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade00af430 0x7fade015ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41856/0 (socket says 172.21.15.153:41856) 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 -- 172.21.15.153:0/288968623 learned_addr learned my addr 172.21.15.153:0/288968623 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6184700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade006a980 0x7fade00c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade5983700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade00c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:44.620 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 -- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 msgr2=0x7fade00c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade00c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 -- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade006a980 msgr2=0x7fade00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade006a980 0x7fade00c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 -- 172.21.15.153:0/288968623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0094a0 con 0x7fade00af430 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.616+0000 7fade6985700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade00af430 0x7fade015ded0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fadd8001560 tx=0x7fadd800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.617+0000 7fadd6ffd700 1 -- 172.21.15.153:0/288968623 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd800aa40 con 0x7fade00af430 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.617+0000 7fadd6ffd700 1 -- 172.21.15.153:0/288968623 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd800abc0 con 0x7fade00af430 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.617+0000 7fadd6ffd700 1 -- 172.21.15.153:0/288968623 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80070d0 con 0x7fade00af430 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.617+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade006d9a0 con 0x7fade00af430 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.617+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade0071f40 con 0x7fade00af430 2023-11-28T13:02:44.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.618+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade006db30 con 0x7fade00af430 2023-11-28T13:02:44.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.618+0000 7fadd6ffd700 1 -- 172.21.15.153:0/288968623 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fadd8010070 con 0x7fade00af430 2023-11-28T13:02:44.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.618+0000 7fadd6ffd700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fadc40682f0 0x7fadc406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:44.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.619+0000 7fadd6ffd700 1 -- 172.21.15.153:0/288968623 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fadd8085520 con 0x7fade00af430 2023-11-28T13:02:44.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.619+0000 7fade6184700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fadc40682f0 0x7fadc406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:44.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.619+0000 7fade6184700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fadc40682f0 0x7fadc406a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faddc0090b0 tx=0x7faddc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:44.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.623+0000 7fadd6ffd700 1 -- 172.21.15.153:0/288968623 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fadd8051d20 con 0x7fade00af430 2023-11-28T13:02:44.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:44.808+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fade00cdf80 con 0x7fade00af430 2023-11-28T13:02:45.515 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.514+0000 7fadd6ffd700 1 -- 172.21.15.153:0/288968623 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7fadd8055340 con 0x7fade00af430 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fadc40682f0 msgr2=0x7fadc406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fadc40682f0 0x7fadc406a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faddc0090b0 tx=0x7faddc008030 comp rx=0 tx=0).stop 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade00af430 msgr2=0x7fade015ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade00af430 0x7fade015ded0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fadd8001560 tx=0x7fadd800cee0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 shutdown_connections 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fadc40682f0 0x7fadc406a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fade00af430 0x7fade015ded0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade006a980 0x7fade00c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 --2- 172.21.15.153:0/288968623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade00af060 0x7fade00c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 >> 172.21.15.153:0/288968623 conn(0x7fade006b380 msgr2=0x7fade015b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:45.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 shutdown_connections 2023-11-28T13:02:45.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.518+0000 7fade83e8700 1 -- 172.21.15.153:0/288968623 wait complete. 2023-11-28T13:02:45.520 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:02:45.533 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:02:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.660+0000 7f108cb4e700 1 Processor -- start 2023-11-28T13:02:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.660+0000 7f108cb4e700 1 -- start start 2023-11-28T13:02:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.661+0000 7f108cb4e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f1088070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.661+0000 7f108cb4e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f10880af430 0x7f108806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.661+0000 7f108cb4e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10880713c0 con 0x7f108806a270 2023-11-28T13:02:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.661+0000 7f108cb4e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108806dee0 con 0x7f10880af430 2023-11-28T13:02:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.661+0000 7f108cb4e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880716a0 con 0x7f10880af060 2023-11-28T13:02:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f1088070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f1085d9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f10880af430 0x7f108806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f1088070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59470/0 (socket says 172.21.15.153:59470) 2023-11-28T13:02:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 -- 172.21.15.153:0/1288177783 learned_addr learned my addr 172.21.15.153:0/1288177783 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 -- 172.21.15.153:0/1288177783 >> v2:172.21.15.177:3300/0 conn(0x7f10880af430 msgr2=0x7f108806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 --2- 172.21.15.153:0/1288177783 >> v2:172.21.15.177:3300/0 conn(0x7f10880af430 0x7f108806d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 -- 172.21.15.153:0/1288177783 >> v1:172.21.15.177:6789/0 conn(0x7f108806a270 legacy=0x7f1088160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 -- 172.21.15.153:0/1288177783 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088071980 con 0x7f10880af060 2023-11-28T13:02:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108659c700 1 --2- 172.21.15.153:0/1288177783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f1088070d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f107c0098f0 tx=0x7f107c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78cd39a1f00c8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108559a700 1 -- 172.21.15.153:0/1288177783 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c011830 con 0x7f10880af060 2023-11-28T13:02:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108559a700 1 -- 172.21.15.153:0/1288177783 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c011990 con 0x7f10880af060 2023-11-28T13:02:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.662+0000 7f108559a700 1 -- 172.21.15.153:0/1288177783 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c011d20 con 0x7f10880af060 2023-11-28T13:02:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 -- 172.21.15.153:0/1288177783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 msgr2=0x7f1088070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 --2- 172.21.15.153:0/1288177783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f1088070d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f107c0098f0 tx=0x7f107c009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 -- 172.21.15.153:0/1288177783 shutdown_connections 2023-11-28T13:02:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 --2- 172.21.15.153:0/1288177783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f1088070d50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 --2- 172.21.15.153:0/1288177783 >> v2:172.21.15.177:3300/0 conn(0x7f10880af430 0x7f108806d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 -- 172.21.15.153:0/1288177783 >> 172.21.15.153:0/1288177783 conn(0x7f108806b380 msgr2=0x7f108806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 -- 172.21.15.153:0/1288177783 shutdown_connections 2023-11-28T13:02:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.663+0000 7f108cb4e700 1 -- 172.21.15.153:0/1288177783 wait complete. 2023-11-28T13:02:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.664+0000 7f108cb4e700 1 Processor -- start 2023-11-28T13:02:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.664+0000 7f108cb4e700 1 -- start start 2023-11-28T13:02:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108cb4e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f108806a270 0x7f10881f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108cb4e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f10880b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:45.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108cb4e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10880af430 0x7f10881f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:45.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108cb4e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881f2680 con 0x7f108806a270 2023-11-28T13:02:45.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108cb4e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881f66c0 con 0x7f10880af430 2023-11-28T13:02:45.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108cb4e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10881f69a0 con 0x7f10880af060 2023-11-28T13:02:45.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f108806a270 0x7f10881f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:45.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108659c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f108806a270 0x7f10881f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41866/0 (socket says 172.21.15.153:41866) 2023-11-28T13:02:45.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f108659c700 1 -- 172.21.15.153:0/613968763 learned_addr learned my addr 172.21.15.153:0/613968763 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:45.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f1085d9b700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f10880b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:45.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.665+0000 7f1086d9d700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10880af430 0x7f10881f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:45.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108659c700 1 -- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 msgr2=0x7f10880b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:45.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108659c700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f10880b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108659c700 1 -- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10880af430 msgr2=0x7f10881f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108659c700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10880af430 0x7f10881f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108659c700 1 -- 172.21.15.153:0/613968763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c009550 con 0x7f108806a270 2023-11-28T13:02:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f1086d9d700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10880af430 0x7f10881f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108659c700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f108806a270 0x7f10881f3140 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f107c009c40 tx=0x7f107c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f1076ffd700 1 -- 172.21.15.153:0/613968763 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0043c0 con 0x7f108806a270 2023-11-28T13:02:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f1076ffd700 1 -- 172.21.15.153:0/613968763 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c01f070 con 0x7f108806a270 2023-11-28T13:02:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1088071f40 con 0x7f108806a270 2023-11-28T13:02:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f1076ffd700 1 -- 172.21.15.153:0/613968763 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0036a0 con 0x7f108806a270 2023-11-28T13:02:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.666+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1088071c60 con 0x7f108806a270 2023-11-28T13:02:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.667+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10880cb670 con 0x7f108806a270 2023-11-28T13:02:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.667+0000 7f1076ffd700 1 -- 172.21.15.153:0/613968763 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f107c02a020 con 0x7f108806a270 2023-11-28T13:02:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.668+0000 7f1076ffd700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10680682f0 0x7f106806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:45.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.668+0000 7f1076ffd700 1 -- 172.21.15.153:0/613968763 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f107c087130 con 0x7f108806a270 2023-11-28T13:02:45.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.668+0000 7f1085d9b700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10680682f0 0x7f106806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:45.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.668+0000 7f1085d9b700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10680682f0 0x7f106806a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f10880b5a80 tx=0x7f1078009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:45.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.672+0000 7f1076ffd700 1 -- 172.21.15.153:0/613968763 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f107c053a70 con 0x7f108806a270 2023-11-28T13:02:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:45.857+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1088068690 con 0x7f108806a270 2023-11-28T13:02:46.355 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:46.358 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:46.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.521+0000 7f1076ffd700 1 -- 172.21.15.153:0/613968763 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f107c057090 con 0x7f108806a270 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.523+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10680682f0 msgr2=0x7f106806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.523+0000 7f108cb4e700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10680682f0 0x7f106806a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f10880b5a80 tx=0x7f1078009210 comp rx=0 tx=0).stop 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.523+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f108806a270 msgr2=0x7f10881f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.523+0000 7f108cb4e700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f108806a270 0x7f10881f3140 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f107c009c40 tx=0x7f107c004750 comp rx=0 tx=0).stop 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 shutdown_connections 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f10680682f0 0x7f106806a7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f108806a270 0x7f10881f3140 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10880af430 0x7f10881f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 --2- 172.21.15.153:0/613968763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10880af060 0x7f10880b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 >> 172.21.15.153:0/613968763 conn(0x7f108806b380 msgr2=0x7f10880ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 shutdown_connections 2023-11-28T13:02:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.524+0000 7f108cb4e700 1 -- 172.21.15.153:0/613968763 wait complete. 2023-11-28T13:02:46.528 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' removed 2023-11-28T13:02:46.538 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:02:46.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.659+0000 7f2eedfe2700 1 Processor -- start 2023-11-28T13:02:46.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2eedfe2700 1 -- start start 2023-11-28T13:02:46.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2eedfe2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ee80cd8c0 0x7f2ee8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:46.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2eedfe2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2ee80cdc90 0x7f2ee80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:46.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2eedfe2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee813ff00 con 0x7f2ee80b2e70 2023-11-28T13:02:46.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2eedfe2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee81404c0 con 0x7f2ee80cd8c0 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2eedfe2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee815aa80 con 0x7f2ee80cdc90 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2ee77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ee80cd8c0 0x7f2ee8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2ee7fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2ee80b2e70 0x7f2ee806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60040/0 (socket says 172.21.15.153:60040) 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.660+0000 7f2ee7fff700 1 -- 172.21.15.153:0/1145671182 learned_addr learned my addr 172.21.15.153:0/1145671182 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee77fe700 1 -- 172.21.15.153:0/1145671182 >> v2:172.21.15.177:3300/0 conn(0x7f2ee80cdc90 msgr2=0x7f2ee80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee77fe700 1 --2- 172.21.15.153:0/1145671182 >> v2:172.21.15.177:3300/0 conn(0x7f2ee80cdc90 0x7f2ee80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee77fe700 1 -- 172.21.15.153:0/1145671182 >> v1:172.21.15.153:6789/0 conn(0x7f2ee80b2e70 legacy=0x7f2ee806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee77fe700 1 -- 172.21.15.153:0/1145671182 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80b3310 con 0x7f2ee80cd8c0 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee77fe700 1 --2- 172.21.15.153:0/1145671182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80cd8c0 0x7f2ee8140950 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8071c40 tx=0x7f2ed0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239da65977b28a34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee67fc700 1 -- 172.21.15.153:0/1145671182 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0005e30 con 0x7f2ee80cd8c0 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee67fc700 1 -- 172.21.15.153:0/1145671182 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed0006030 con 0x7f2ee80cd8c0 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.661+0000 7f2ee67fc700 1 -- 172.21.15.153:0/1145671182 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00063c0 con 0x7f2ee80cd8c0 2023-11-28T13:02:46.663 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 -- 172.21.15.153:0/1145671182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80cd8c0 msgr2=0x7f2ee8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/1145671182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80cd8c0 0x7f2ee8140950 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8071c40 tx=0x7f2ed0009b00 comp rx=0 tx=0).stop 2023-11-28T13:02:46.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 -- 172.21.15.153:0/1145671182 shutdown_connections 2023-11-28T13:02:46.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/1145671182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80cd8c0 0x7f2ee8140950 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/1145671182 >> v2:172.21.15.177:3300/0 conn(0x7f2ee80cdc90 0x7f2ee80b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 -- 172.21.15.153:0/1145671182 >> 172.21.15.153:0/1145671182 conn(0x7f2ee806aed0 msgr2=0x7f2ee806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:46.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 -- 172.21.15.153:0/1145671182 shutdown_connections 2023-11-28T13:02:46.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.662+0000 7f2eedfe2700 1 -- 172.21.15.153:0/1145671182 wait complete. 2023-11-28T13:02:46.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.663+0000 7f2eedfe2700 1 Processor -- start 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.663+0000 7f2eedfe2700 1 -- start start 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80b2e70 0x7f2ee80ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee80cd8c0 0x7f2ee80c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee80cdc90 0x7f2ee80c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80ca930 con 0x7f2ee80b2e70 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80c76b0 con 0x7f2ee80cdc90 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80c9350 con 0x7f2ee80cd8c0 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80b2e70 0x7f2ee80ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80b2e70 0x7f2ee80ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41880/0 (socket says 172.21.15.153:41880) 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 -- 172.21.15.153:0/725545790 learned_addr learned my addr 172.21.15.153:0/725545790 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee6ffd700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee80cd8c0 0x7f2ee80c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee7fff700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee80cdc90 0x7f2ee80c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:46.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 -- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee80cd8c0 msgr2=0x7f2ee80c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee80cd8c0 0x7f2ee80c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 -- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee80cdc90 msgr2=0x7f2ee80c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee80cdc90 0x7f2ee80c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 -- 172.21.15.153:0/725545790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed0009430 con 0x7f2ee80b2e70 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee7fff700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee80cdc90 0x7f2ee80c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2ee77fe700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80b2e70 0x7f2ee80ca3f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009b40 tx=0x7f2ed0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2edffff700 1 -- 172.21.15.153:0/725545790 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0004450 con 0x7f2ee80b2e70 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2edffff700 1 -- 172.21.15.153:0/725545790 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed00045d0 con 0x7f2ee80b2e70 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee8140150 con 0x7f2ee80b2e70 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2edffff700 1 -- 172.21.15.153:0/725545790 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00038e0 con 0x7f2ee80b2e70 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.664+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee80b35f0 con 0x7f2ee80b2e70 2023-11-28T13:02:46.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.665+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee80cbeb0 con 0x7f2ee80b2e70 2023-11-28T13:02:46.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.666+0000 7f2edffff700 1 -- 172.21.15.153:0/725545790 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ed000f050 con 0x7f2ee80b2e70 2023-11-28T13:02:46.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.666+0000 7f2edffff700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2ec80682f0 0x7f2ec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:46.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.666+0000 7f2edffff700 1 -- 172.21.15.153:0/725545790 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2ed0087630 con 0x7f2ee80b2e70 2023-11-28T13:02:46.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.666+0000 7f2ee6ffd700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2ec80682f0 0x7f2ec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:46.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.667+0000 7f2ee6ffd700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2ec80682f0 0x7f2ec806a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:46.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.671+0000 7f2edffff700 1 -- 172.21.15.153:0/725545790 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ed0054190 con 0x7f2ee80b2e70 2023-11-28T13:02:46.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.865+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2ee80ce0e0 con 0x7f2ee80b2e70 2023-11-28T13:02:46.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.868+0000 7f2edffff700 1 -- 172.21.15.153:0/725545790 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2ed002a0a0 con 0x7f2ee80b2e70 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.870+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2ec80682f0 msgr2=0x7f2ec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.870+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2ec80682f0 0x7f2ec806a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006fd0 tx=0x7f2ed8008040 comp rx=0 tx=0).stop 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.870+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80b2e70 msgr2=0x7f2ee80ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.870+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80b2e70 0x7f2ee80ca3f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009b40 tx=0x7f2ed0004750 comp rx=0 tx=0).stop 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.870+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 shutdown_connections 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.870+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2ec80682f0 0x7f2ec806a7a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.871+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ee80b2e70 0x7f2ee80ca3f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.871+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee80cdc90 0x7f2ee80c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.871+0000 7f2eedfe2700 1 --2- 172.21.15.153:0/725545790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ee80cd8c0 0x7f2ee80c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.871+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 >> 172.21.15.153:0/725545790 conn(0x7f2ee806aed0 msgr2=0x7f2ee813dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:46.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.871+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 shutdown_connections 2023-11-28T13:02:46.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:46.871+0000 7f2eedfe2700 1 -- 172.21.15.153:0/725545790 wait complete. 2023-11-28T13:02:46.884 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T13:02:47.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.003+0000 7fa5b943a700 1 Processor -- start 2023-11-28T13:02:47.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.003+0000 7fa5b943a700 1 -- start start 2023-11-28T13:02:47.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b943a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 0x7fa5b40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b943a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b40cf5e0 con 0x7fa5b415ca50 2023-11-28T13:02:47.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b943a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b40cecb0 con 0x7fa5b40ccc40 2023-11-28T13:02:47.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b943a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40cd470 con 0x7fa5b406ba90 2023-11-28T13:02:47.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b2ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa5b415ca50 0x7fa5b406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60052/0 (socket says 172.21.15.153:60052) 2023-11-28T13:02:47.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b2ffd700 1 -- 172.21.15.153:0/1935627581 learned_addr learned my addr 172.21.15.153:0/1935627581 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:47.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b27fc700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 0x7fa5b40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802183736 0 0) 0x7fa5b40cf5e0 con 0x7fa5b415ca50 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa59c0035b0 con 0x7fa5b415ca50 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093123271 0 0) 0x7fa5b40cecb0 con 0x7fa5b40ccc40 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b40cf5e0 con 0x7fa5b40ccc40 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 356934352 0 0) 0x7fa59c0035b0 con 0x7fa5b415ca50 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 msgr2=0x7fa5b40cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 0x7fa5b40cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 >> v1:172.21.15.177:6789/0 conn(0x7fa5b40ccc40 legacy=0x7fa5b40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40cef90 con 0x7fa5b415ca50 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.004+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fa5a8001670 con 0x7fa5b415ca50 2023-11-28T13:02:47.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 >> v1:172.21.15.153:6789/0 conn(0x7fa5b415ca50 legacy=0x7fa5b406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c003bc0 0x7fa59c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 0x7fa59c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa59c004d90 0x7fa59c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40cd470 con 0x7fa59c004d90 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c00bac0 con 0x7fa59c003bc0 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c00bc80 con 0x7fa5b406ba90 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b37fe700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa59c004d90 0x7fa59c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b2ffd700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c003bc0 0x7fa59c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b27fc700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 0x7fa59c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b37fe700 1 -- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 msgr2=0x7fa59c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b37fe700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b406ba90 0x7fa59c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.010 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b37fe700 1 -- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c003bc0 msgr2=0x7fa59c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b37fe700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c003bc0 0x7fa59c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b37fe700 1 -- 172.21.15.153:0/1935627581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c0035b0 con 0x7fa59c004d90 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b37fe700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa59c004d90 0x7fa59c005200 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac002230 tx=0x7fa59c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96a8db67f541539 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac001f30 con 0x7fa59c004d90 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00ce80 con 0x7fa59c004d90 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.005+0000 7fa5b1ffb700 1 -- 172.21.15.153:0/1935627581 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac003c40 con 0x7fa59c004d90 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.006+0000 7fa5b943a700 1 -- 172.21.15.153:0/1935627581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa59c004d90 msgr2=0x7fa59c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.006+0000 7fa5b943a700 1 --2- 172.21.15.153:0/1935627581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa59c004d90 0x7fa59c005200 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac002230 tx=0x7fa59c005880 comp rx=0 tx=0).stop 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.006+0000 7fa5b37fe700 1 -- 172.21.15.153:0/1935627581 reap_dead start 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.006+0000 7fa5b943a700 1 -- 172.21.15.153:0/1935627581 shutdown_connections 2023-11-28T13:02:47.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.006+0000 7fa5b943a700 1 -- 172.21.15.153:0/1935627581 >> 172.21.15.153:0/1935627581 conn(0x7fa5b409a230 msgr2=0x7fa5b408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:47.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.006+0000 7fa5b943a700 1 -- 172.21.15.153:0/1935627581 shutdown_connections 2023-11-28T13:02:47.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.006+0000 7fa5b943a700 1 -- 172.21.15.153:0/1935627581 wait complete. 2023-11-28T13:02:47.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.007+0000 7fa5b943a700 1 Processor -- start 2023-11-28T13:02:47.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.007+0000 7fa5b943a700 1 -- start start 2023-11-28T13:02:47.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b943a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5b414ec20 0x7fa5b40aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b943a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b414eff0 0x7fa5b40af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b943a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b40c6660 0x7fa5b40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b943a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b415ae40 con 0x7fa5b40c6660 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b943a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40c73b0 con 0x7fa5b414ec20 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b943a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4159e30 con 0x7fa5b414eff0 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b40c6660 0x7fa5b40c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b40c6660 0x7fa5b40c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41904/0 (socket says 172.21.15.153:41904) 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 -- 172.21.15.153:0/3033585373 learned_addr learned my addr 172.21.15.153:0/3033585373 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 -- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b414eff0 msgr2=0x7fa5b40af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b414eff0 0x7fa5b40af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.015 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 -- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5b414ec20 msgr2=0x7fa5b40aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b2ffd700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5b414ec20 0x7fa5b40aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5b414ec20 0x7fa5b40aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b37fe700 1 -- 172.21.15.153:0/3033585373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40cd0e0 con 0x7fa5b40c6660 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.008+0000 7fa5b2ffd700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5b414ec20 0x7fa5b40aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.009+0000 7fa5b37fe700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b40c6660 0x7fa5b40c6b00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac015040 tx=0x7fa5ac001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.009+0000 7fa59b7fe700 1 -- 172.21.15.153:0/3033585373 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac003e90 con 0x7fa5b40c6660 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.009+0000 7fa59b7fe700 1 -- 172.21.15.153:0/3033585373 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac0050d0 con 0x7fa5b40c6660 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.009+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b40cf270 con 0x7fa5b40c6660 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.009+0000 7fa59b7fe700 1 -- 172.21.15.153:0/3033585373 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac005460 con 0x7fa5b40c6660 2023-11-28T13:02:47.016 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.009+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b415ce20 con 0x7fa5b40c6660 2023-11-28T13:02:47.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.010+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b40745c0 con 0x7fa5b40c6660 2023-11-28T13:02:47.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.010+0000 7fa59b7fe700 1 -- 172.21.15.153:0/3033585373 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa5ac010050 con 0x7fa5b40c6660 2023-11-28T13:02:47.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.010+0000 7fa59b7fe700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa5a00682f0 0x7fa5a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.011+0000 7fa5b2ffd700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa5a00682f0 0x7fa5a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.011+0000 7fa59b7fe700 1 -- 172.21.15.153:0/3033585373 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa5ac086950 con 0x7fa5b40c6660 2023-11-28T13:02:47.017 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.011+0000 7fa5b2ffd700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa5a00682f0 0x7fa5a006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8001eb0 tx=0x7fa5a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:47.018 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.015+0000 7fa59b7fe700 1 -- 172.21.15.153:0/3033585373 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa5ac0534b0 con 0x7fa5b40c6660 2023-11-28T13:02:47.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.271+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fa5b41e5cb0 con 0x7fa5b40c6660 2023-11-28T13:02:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.519+0000 7fa59b7fe700 1 -- 172.21.15.153:0/3033585373 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v146) v1 ==== 149+0+0 (secure 0 0 0) 0x7fa5ac002090 con 0x7fa5b40c6660 2023-11-28T13:02:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa5a00682f0 msgr2=0x7fa5a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa5a00682f0 0x7fa5a006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8001eb0 tx=0x7fa5a8009040 comp rx=0 tx=0).stop 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b40c6660 msgr2=0x7fa5b40c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b40c6660 0x7fa5b40c6b00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac015040 tx=0x7fa5ac001650 comp rx=0 tx=0).stop 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 shutdown_connections 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa5a00682f0 0x7fa5a006a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5b40c6660 0x7fa5b40c6b00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5b414ec20 0x7fa5b40aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 --2- 172.21.15.153:0/3033585373 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5b414eff0 0x7fa5b40af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 >> 172.21.15.153:0/3033585373 conn(0x7fa5b409a230 msgr2=0x7fa5b40cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.521+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 shutdown_connections 2023-11-28T13:02:47.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.522+0000 7fa5b943a700 1 -- 172.21.15.153:0/3033585373 wait complete. 2023-11-28T13:02:47.535 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T13:02:47.535 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T13:02:47.535 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:02:47.535 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T13:02:47.656 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 Processor -- start 2023-11-28T13:02:47.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 -- start start 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19880cd8c0 0x7f1988140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880cdc90 0x7f19880b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19880b2e70 0x7f198806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198813ff00 con 0x7f19880cd8c0 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19881404c0 con 0x7f19880b2e70 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.656+0000 7f198e41e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198815aa80 con 0x7f19880cdc90 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19880cd8c0 0x7f1988140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19880cd8c0 0x7f1988140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41908/0 (socket says 172.21.15.153:41908) 2023-11-28T13:02:47.659 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 -- 172.21.15.153:0/2511155291 learned_addr learned my addr 172.21.15.153:0/2511155291 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f198c9bb700 1 --2- 172.21.15.153:0/2511155291 >> v2:172.21.15.177:3300/0 conn(0x7f19880b2e70 0x7f198806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f19877fe700 1 --2- 172.21.15.153:0/2511155291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880cdc90 0x7f19880b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 -- 172.21.15.153:0/2511155291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880cdc90 msgr2=0x7f19880b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 --2- 172.21.15.153:0/2511155291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880cdc90 0x7f19880b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 -- 172.21.15.153:0/2511155291 >> v2:172.21.15.177:3300/0 conn(0x7f19880b2e70 msgr2=0x7f198806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 --2- 172.21.15.153:0/2511155291 >> v2:172.21.15.177:3300/0 conn(0x7f19880b2e70 0x7f198806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 -- 172.21.15.153:0/2511155291 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880b3310 con 0x7f19880cd8c0 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1987fff700 1 --2- 172.21.15.153:0/2511155291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f1988140950 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f19881444c0 tx=0x7f1978009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f267e00739ae6374 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1986ffd700 1 -- 172.21.15.153:0/2511155291 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978005e30 con 0x7f19880cd8c0 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1986ffd700 1 -- 172.21.15.153:0/2511155291 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978004030 con 0x7f19880cd8c0 2023-11-28T13:02:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.657+0000 7f1986ffd700 1 -- 172.21.15.153:0/2511155291 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780043e0 con 0x7f19880cd8c0 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 -- 172.21.15.153:0/2511155291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 msgr2=0x7f1988140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 --2- 172.21.15.153:0/2511155291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f1988140950 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f19881444c0 tx=0x7f1978009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 -- 172.21.15.153:0/2511155291 shutdown_connections 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 --2- 172.21.15.153:0/2511155291 >> v2:172.21.15.177:3300/0 conn(0x7f19880b2e70 0x7f198806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 --2- 172.21.15.153:0/2511155291 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f1988140950 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 --2- 172.21.15.153:0/2511155291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880cdc90 0x7f19880b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 -- 172.21.15.153:0/2511155291 >> 172.21.15.153:0/2511155291 conn(0x7f198806aed0 msgr2=0x7f198806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 -- 172.21.15.153:0/2511155291 shutdown_connections 2023-11-28T13:02:47.661 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.658+0000 7f198e41e700 1 -- 172.21.15.153:0/2511155291 wait complete. 2023-11-28T13:02:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.659+0000 7f198e41e700 1 Processor -- start 2023-11-28T13:02:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.660+0000 7f198e41e700 1 -- start start 2023-11-28T13:02:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.660+0000 7f198e41e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880b2e70 0x7f19880b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.660+0000 7f198e41e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f19880ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198e41e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19880cdc90 0x7f19880b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198e41e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988141590 con 0x7f19880cd8c0 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198e41e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880b65c0 con 0x7f19880cdc90 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198e41e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880b68a0 con 0x7f19880b2e70 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f19880ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f19880ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41916/0 (socket says 172.21.15.153:41916) 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 -- 172.21.15.153:0/438485013 learned_addr learned my addr 172.21.15.153:0/438485013 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198c9bb700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19880cdc90 0x7f19880b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f1987fff700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880b2e70 0x7f19880b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 -- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880b2e70 msgr2=0x7f19880b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880b2e70 0x7f19880b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 -- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19880cdc90 msgr2=0x7f19880b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19880cdc90 0x7f19880b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 -- 172.21.15.153:0/438485013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978009500 con 0x7f19880cd8c0 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198c9bb700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19880cdc90 0x7f19880b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f1987fff700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880b2e70 0x7f19880b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f19877fe700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f19880ca760 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f197c00e7e0 tx=0x7f197c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f1984ff9700 1 -- 172.21.15.153:0/438485013 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c013610 con 0x7f19880cd8c0 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f1984ff9700 1 -- 172.21.15.153:0/438485013 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c013770 con 0x7f19880cd8c0 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1988140150 con 0x7f19880cd8c0 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f1984ff9700 1 -- 172.21.15.153:0/438485013 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c013ae0 con 0x7f19880cd8c0 2023-11-28T13:02:47.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.661+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19880b35f0 con 0x7f19880cd8c0 2023-11-28T13:02:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.662+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19880ce100 con 0x7f19880cd8c0 2023-11-28T13:02:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.663+0000 7f1984ff9700 1 -- 172.21.15.153:0/438485013 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f197c023070 con 0x7f19880cd8c0 2023-11-28T13:02:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.663+0000 7f1984ff9700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19740682f0 0x7f197406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.663+0000 7f1984ff9700 1 -- 172.21.15.153:0/438485013 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f197c085580 con 0x7f19880cd8c0 2023-11-28T13:02:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.663+0000 7f1987fff700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19740682f0 0x7f197406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.664+0000 7f1987fff700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19740682f0 0x7f197406a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f19881444c0 tx=0x7f1978008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:47.668 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.667+0000 7f1984ff9700 1 -- 172.21.15.153:0/438485013 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f197c0520e0 con 0x7f19880cd8c0 2023-11-28T13:02:47.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:47.924+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f1988068690 con 0x7f19880cd8c0 2023-11-28T13:02:48.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.524+0000 7f1984ff9700 1 -- 172.21.15.153:0/438485013 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v147) v1 ==== 172+0+0 (secure 0 0 0) 0x7f197c055700 con 0x7f19880cd8c0 2023-11-28T13:02:48.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19740682f0 msgr2=0x7f197406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:48.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19740682f0 0x7f197406a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f19881444c0 tx=0x7f1978008070 comp rx=0 tx=0).stop 2023-11-28T13:02:48.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 msgr2=0x7f19880ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f19880ca760 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f197c00e7e0 tx=0x7f197c00eba0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 shutdown_connections 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19880cdc90 0x7f19880b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19880b2e70 0x7f19880b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19740682f0 0x7f197406a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 --2- 172.21.15.153:0/438485013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19880cd8c0 0x7f19880ca760 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 >> 172.21.15.153:0/438485013 conn(0x7f198806aed0 msgr2=0x7f198813dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 shutdown_connections 2023-11-28T13:02:48.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.526+0000 7f198e41e700 1 -- 172.21.15.153:0/438485013 wait complete. 2023-11-28T13:02:48.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:02:48.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.669+0000 7fbf18437700 1 Processor -- start 2023-11-28T13:02:48.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.669+0000 7fbf18437700 1 -- start start 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf18437700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbf1006ba90 0x7fbf100af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf18437700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbf100cbdf0 0x7fbf100ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf18437700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf100cc5f0 con 0x7fbf1015cef0 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf18437700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1015d470 con 0x7fbf100cbdf0 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf18437700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1015d750 con 0x7fbf1006ba90 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf169d4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbf100cbdf0 0x7fbf100ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf169d4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbf100cbdf0 0x7fbf100ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41926/0 (socket says 172.21.15.153:41926) 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf169d4700 1 -- 172.21.15.153:0/3874887619 learned_addr learned my addr 172.21.15.153:0/3874887619 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf159d2700 1 --2- 172.21.15.153:0/3874887619 >> v2:172.21.15.177:3300/0 conn(0x7fbf1006ba90 0x7fbf100af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:48.677 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf169d4700 1 -- 172.21.15.153:0/3874887619 >> v2:172.21.15.177:3300/0 conn(0x7fbf1006ba90 msgr2=0x7fbf100af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf169d4700 1 --2- 172.21.15.153:0/3874887619 >> v2:172.21.15.177:3300/0 conn(0x7fbf1006ba90 0x7fbf100af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf169d4700 1 -- 172.21.15.153:0/3874887619 >> v1:172.21.15.177:6789/0 conn(0x7fbf1015cef0 legacy=0x7fbf1006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf169d4700 1 -- 172.21.15.153:0/3874887619 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1015da30 con 0x7fbf100cbdf0 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.670+0000 7fbf159d2700 1 --2- 172.21.15.153:0/3874887619 >> v2:172.21.15.177:3300/0 conn(0x7fbf1006ba90 0x7fbf100af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf169d4700 1 --2- 172.21.15.153:0/3874887619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf100cbdf0 0x7fbf100ca650 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf1006fbb0 tx=0x7fbf0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5142649c76e0863 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf151d1700 1 -- 172.21.15.153:0/3874887619 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800e070 con 0x7fbf100cbdf0 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf151d1700 1 -- 172.21.15.153:0/3874887619 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf08003a20 con 0x7fbf100cbdf0 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf151d1700 1 -- 172.21.15.153:0/3874887619 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08003dd0 con 0x7fbf100cbdf0 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 -- 172.21.15.153:0/3874887619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf100cbdf0 msgr2=0x7fbf100ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:48.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 --2- 172.21.15.153:0/3874887619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf100cbdf0 0x7fbf100ca650 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf1006fbb0 tx=0x7fbf0800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 -- 172.21.15.153:0/3874887619 shutdown_connections 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 --2- 172.21.15.153:0/3874887619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf100cbdf0 0x7fbf100ca650 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 --2- 172.21.15.153:0/3874887619 >> v2:172.21.15.177:3300/0 conn(0x7fbf1006ba90 0x7fbf100af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 -- 172.21.15.153:0/3874887619 >> 172.21.15.153:0/3874887619 conn(0x7fbf100d0000 msgr2=0x7fbf100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 -- 172.21.15.153:0/3874887619 shutdown_connections 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.671+0000 7fbf18437700 1 -- 172.21.15.153:0/3874887619 wait complete. 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.673+0000 7fbf18437700 1 Processor -- start 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.673+0000 7fbf18437700 1 -- start start 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf18437700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf1006ba90 0x7fbf1014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf18437700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf100cbdf0 0x7fbf100afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf18437700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf1015cef0 0x7fbf100b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:48.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf18437700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1014a080 con 0x7fbf1006ba90 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf18437700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c99d0 con 0x7fbf100cbdf0 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf18437700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c9cb0 con 0x7fbf1015cef0 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf1006ba90 0x7fbf1014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf1006ba90 0x7fbf1014cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41928/0 (socket says 172.21.15.153:41928) 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 -- 172.21.15.153:0/2722981212 learned_addr learned my addr 172.21.15.153:0/2722981212 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf159d2700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf100cbdf0 0x7fbf100afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf169d4700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf1015cef0 0x7fbf100b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 -- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf1015cef0 msgr2=0x7fbf100b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf1015cef0 0x7fbf100b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 -- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf100cbdf0 msgr2=0x7fbf100afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf100cbdf0 0x7fbf100afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf161d3700 1 -- 172.21.15.153:0/2722981212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf08009b30 con 0x7fbf1006ba90 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.674+0000 7fbf169d4700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf1015cef0 0x7fbf100b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.675+0000 7fbf161d3700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf1006ba90 0x7fbf1014cb80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000b4a0 tx=0x7fbf0000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.675+0000 7fbf06ffd700 1 -- 172.21.15.153:0/2722981212 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00011830 con 0x7fbf1006ba90 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.675+0000 7fbf06ffd700 1 -- 172.21.15.153:0/2722981212 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00011990 con 0x7fbf1006ba90 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.675+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1015dff0 con 0x7fbf1006ba90 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.675+0000 7fbf06ffd700 1 -- 172.21.15.153:0/2722981212 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00011d40 con 0x7fbf1006ba90 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.675+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1015dd10 con 0x7fbf1006ba90 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.676+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf100cab90 con 0x7fbf1006ba90 2023-11-28T13:02:48.681 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.676+0000 7fbf06ffd700 1 -- 172.21.15.153:0/2722981212 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf0000f050 con 0x7fbf1006ba90 2023-11-28T13:02:48.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.677+0000 7fbf06ffd700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbef40682f0 0x7fbef406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:48.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.677+0000 7fbf06ffd700 1 -- 172.21.15.153:0/2722981212 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbf00085470 con 0x7fbf1006ba90 2023-11-28T13:02:48.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.677+0000 7fbf159d2700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbef40682f0 0x7fbef406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:48.682 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.677+0000 7fbf159d2700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbef40682f0 0x7fbef406a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbf1015aca0 tx=0x7fbf0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:48.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.681+0000 7fbf06ffd700 1 -- 172.21.15.153:0/2722981212 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf00051f70 con 0x7fbf1006ba90 2023-11-28T13:02:48.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.864+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbf100d0e60 con 0x7fbf1006ba90 2023-11-28T13:02:48.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.868+0000 7fbf06ffd700 1 -- 172.21.15.153:0/2722981212 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbf00055590 con 0x7fbf1006ba90 2023-11-28T13:02:48.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbef40682f0 msgr2=0x7fbef406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:48.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbef40682f0 0x7fbef406a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbf1015aca0 tx=0x7fbf0c008040 comp rx=0 tx=0).stop 2023-11-28T13:02:48.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf1006ba90 msgr2=0x7fbf1014cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:48.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf1006ba90 0x7fbf1014cb80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000b4a0 tx=0x7fbf0000b7b0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 shutdown_connections 2023-11-28T13:02:48.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbef40682f0 0x7fbef406a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf1006ba90 0x7fbf1014cb80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf1015cef0 0x7fbf100b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 --2- 172.21.15.153:0/2722981212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf100cbdf0 0x7fbf100afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:48.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 >> 172.21.15.153:0/2722981212 conn(0x7fbf100d0000 msgr2=0x7fbf10071880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:48.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 shutdown_connections 2023-11-28T13:02:48.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:48.870+0000 7fbf18437700 1 -- 172.21.15.153:0/2722981212 wait complete. 2023-11-28T13:02:48.883 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:02:48.883 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:48.928 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:48.982 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:02:48.982 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:02:48.983 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:49.002 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:49.003 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:02:49.003 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:49.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:49.023 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:02:49.023 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:49.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:49.060 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:02:49.060 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:49.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:49.082 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:02:49.082 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:49.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.220+0000 7f27ec879700 1 Processor -- start 2023-11-28T13:02:49.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.220+0000 7f27ec879700 1 -- start start 2023-11-28T13:02:49.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27ec879700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e806dee0 0x7f27e806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27ec879700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f27e8160540 0x7f27e81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27ec879700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e815d060 con 0x7f27e8071040 2023-11-28T13:02:49.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27ec879700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e815d340 con 0x7f27e8160540 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27ec879700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e815d620 con 0x7f27e806dee0 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27e5d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e806dee0 0x7f27e806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27e6d9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f27e8160540 0x7f27e81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27e5d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e806dee0 0x7f27e806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59548/0 (socket says 172.21.15.153:59548) 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27e5d9b700 1 -- 172.21.15.153:0/556389570 learned_addr learned my addr 172.21.15.153:0/556389570 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27e6d9d700 1 -- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e806dee0 msgr2=0x7f27e806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27e6d9d700 1 --2- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e806dee0 0x7f27e806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.221+0000 7f27e6d9d700 1 -- 172.21.15.153:0/556389570 >> v1:172.21.15.177:6789/0 conn(0x7f27e8071040 legacy=0x7f27e8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27e6d9d700 1 -- 172.21.15.153:0/556389570 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e815d900 con 0x7f27e8160540 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27e5d9b700 1 --2- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e806dee0 0x7f27e806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27e6d9d700 1 --2- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e8160540 0x7f27e81609c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009d90 tx=0x7f27e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307c2fb7105124ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:49.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27e559a700 1 -- 172.21.15.153:0/556389570 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e000e070 con 0x7f27e8160540 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27e559a700 1 -- 172.21.15.153:0/556389570 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e000bb20 con 0x7f27e8160540 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27e559a700 1 -- 172.21.15.153:0/556389570 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0012430 con 0x7f27e8160540 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27ec879700 1 -- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e8160540 msgr2=0x7f27e81609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.222+0000 7f27ec879700 1 --2- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e8160540 0x7f27e81609c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009d90 tx=0x7f27e000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.223+0000 7f27ec879700 1 -- 172.21.15.153:0/556389570 shutdown_connections 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.223+0000 7f27ec879700 1 --2- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e8160540 0x7f27e81609c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.223+0000 7f27ec879700 1 --2- 172.21.15.153:0/556389570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e806dee0 0x7f27e806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.223+0000 7f27ec879700 1 -- 172.21.15.153:0/556389570 >> 172.21.15.153:0/556389570 conn(0x7f27e80d0000 msgr2=0x7f27e80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.223+0000 7f27ec879700 1 -- 172.21.15.153:0/556389570 shutdown_connections 2023-11-28T13:02:49.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.223+0000 7f27ec879700 1 -- 172.21.15.153:0/556389570 wait complete. 2023-11-28T13:02:49.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.224+0000 7f27ec879700 1 Processor -- start 2023-11-28T13:02:49.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.224+0000 7f27ec879700 1 -- start start 2023-11-28T13:02:49.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27ec879700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e806dee0 0x7f27e81f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27ec879700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e8071040 0x7f27e80c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27ec879700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8160540 0x7f27e80b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27ec879700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81f3510 con 0x7f27e8160540 2023-11-28T13:02:49.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27ec879700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80c6320 con 0x7f27e806dee0 2023-11-28T13:02:49.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27ec879700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80b3260 con 0x7f27e8071040 2023-11-28T13:02:49.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27e6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8160540 0x7f27e80b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27e6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8160540 0x7f27e80b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41946/0 (socket says 172.21.15.153:41946) 2023-11-28T13:02:49.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.225+0000 7f27e6d9d700 1 -- 172.21.15.153:0/3280881242 learned_addr learned my addr 172.21.15.153:0/3280881242 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:49.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e5d9b700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e8071040 0x7f27e80c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e659c700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e806dee0 0x7f27e81f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e6d9d700 1 -- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e8071040 msgr2=0x7f27e80c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e6d9d700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e8071040 0x7f27e80c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e6d9d700 1 -- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e806dee0 msgr2=0x7f27e81f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e6d9d700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e806dee0 0x7f27e81f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e6d9d700 1 -- 172.21.15.153:0/3280881242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc009550 con 0x7f27e8160540 2023-11-28T13:02:49.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e659c700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e806dee0 0x7f27e81f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:49.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27e6d9d700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8160540 0x7f27e80b4c30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b370 tx=0x7f27e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:49.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27d6ffd700 1 -- 172.21.15.153:0/3280881242 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0012430 con 0x7f27e8160540 2023-11-28T13:02:49.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27d6ffd700 1 -- 172.21.15.153:0/3280881242 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0020070 con 0x7f27e8160540 2023-11-28T13:02:49.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27d6ffd700 1 -- 172.21.15.153:0/3280881242 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0004280 con 0x7f27e8160540 2023-11-28T13:02:49.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e0009a70 con 0x7f27e8160540 2023-11-28T13:02:49.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.226+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e815ccd0 con 0x7f27e8160540 2023-11-28T13:02:49.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.227+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e806e870 con 0x7f27e8160540 2023-11-28T13:02:49.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.228+0000 7f27d6ffd700 1 -- 172.21.15.153:0/3280881242 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f27e002e070 con 0x7f27e8160540 2023-11-28T13:02:49.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.228+0000 7f27d6ffd700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f27c80682f0 0x7f27c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.228+0000 7f27d6ffd700 1 -- 172.21.15.153:0/3280881242 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f27e0022070 con 0x7f27e8160540 2023-11-28T13:02:49.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.228+0000 7f27e659c700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f27c80682f0 0x7f27c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.228+0000 7f27e659c700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f27c80682f0 0x7f27c806a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000f60 tx=0x7f27d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:49.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.232+0000 7f27d6ffd700 1 -- 172.21.15.153:0/3280881242 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27e00541a0 con 0x7f27e8160540 2023-11-28T13:02:49.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.458+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27e806a270 con 0x7f27e8160540 2023-11-28T13:02:49.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.458+0000 7f27d6ffd700 1 -- 172.21.15.153:0/3280881242 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f27e002cdf0 con 0x7f27e8160540 2023-11-28T13:02:49.460 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:49.460 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3366780869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3366780869},{"type":"v1","addr":"172.21.15.177:6834","nonce":3366780869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/4194601284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4194601284},{"type":"v1","addr":"172.21.15.177:6835","nonce":4194601284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5532,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1976427639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1976427639},{"type":"v1","addr":"172.21.15.153:6836","nonce":1976427639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/3324771082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3324771082},{"type":"v1","addr":"172.21.15.153:6837","nonce":3324771082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-11-28T13:02:49.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f27c80682f0 msgr2=0x7f27c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f27c80682f0 0x7f27c806a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000f60 tx=0x7f27d8009040 comp rx=0 tx=0).stop 2023-11-28T13:02:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8160540 msgr2=0x7f27e80b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8160540 0x7f27e80b4c30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b370 tx=0x7f27e0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 shutdown_connections 2023-11-28T13:02:49.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f27c80682f0 0x7f27c806a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f27e8160540 0x7f27e80b4c30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27e806dee0 0x7f27e81f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 --2- 172.21.15.153:0/3280881242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27e8071040 0x7f27e80c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 >> 172.21.15.153:0/3280881242 conn(0x7f27e80d0000 msgr2=0x7f27e815e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:49.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 shutdown_connections 2023-11-28T13:02:49.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.460+0000 7f27ec879700 1 -- 172.21.15.153:0/3280881242 wait complete. 2023-11-28T13:02:49.464 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 25 2023-11-28T13:02:49.473 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:02:49.474 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:02:49.474 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:02:49.474 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:49.474 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:02:49.474 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:02:49.474 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:02:49.474 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:49.475 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:02:49.475 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:02:49.475 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:02:49.475 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:49.475 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:02:49.475 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:02:49.476 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:02:49.476 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:49.483 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:02:49.483 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:02:49.485 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:02:49.485 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:02:49.487 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:02:49.487 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:02:49.488 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:02:49.488 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:02:49.526 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:02:49.526 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:49.527 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:02:49.527 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:49.531 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:02:49.531 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:49.531 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:02:49.532 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:49.532 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:02:49.606 INFO:tasks.ceph.mds.a.smithi153.stdout:starting mds.a at 2023-11-28T13:02:49.606 INFO:tasks.ceph.mds.c.smithi153.stdout:starting mds.c at 2023-11-28T13:02:49.619 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T13:02:49.620 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T13:02:49.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.661+0000 7f6745e6e700 1 Processor -- start 2023-11-28T13:02:49.662 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.661+0000 7f6745e6e700 1 -- start start 2023-11-28T13:02:49.664 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f6745e6e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6740140950 0x7f67400b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f6745e6e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f67400b2e70 0x7f674006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f6745e6e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674013ff00 con 0x7f67400cd8c0 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f6745e6e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67401404c0 con 0x7f6740140950 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f6745e6e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6740140dc0 con 0x7f67400b2e70 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6740140950 0x7f67400b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673f7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f67400cd8c0 0x7f67400cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60138/0 (socket says 172.21.15.153:60138) 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673f7fe700 1 -- 172.21.15.153:0/3431289562 learned_addr learned my addr 172.21.15.153:0/3431289562 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673ffff700 1 --2- 172.21.15.153:0/3431289562 >> v2:172.21.15.177:3300/0 conn(0x7f67400b2e70 0x7f674006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673effd700 1 -- 172.21.15.153:0/3431289562 >> v2:172.21.15.177:3300/0 conn(0x7f67400b2e70 msgr2=0x7f674006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:49.665 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673effd700 1 --2- 172.21.15.153:0/3431289562 >> v2:172.21.15.177:3300/0 conn(0x7f67400b2e70 0x7f674006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673effd700 1 -- 172.21.15.153:0/3431289562 >> v1:172.21.15.153:6789/0 conn(0x7f67400cd8c0 legacy=0x7f67400cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673e7fc700 1 -- 172.21.15.153:0/3431289562 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677595862 0 0) 0x7f674013ff00 con 0x7f67400cd8c0 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.662+0000 7f673effd700 1 -- 172.21.15.153:0/3431289562 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674015aa80 con 0x7f6740140950 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f673effd700 1 --2- 172.21.15.153:0/3431289562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6740140950 0x7f67400b2930 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6740144350 tx=0x7f6728009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f76190c4712a37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f673e7fc700 1 -- 172.21.15.153:0/3431289562 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728011830 con 0x7f6740140950 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f673e7fc700 1 -- 172.21.15.153:0/3431289562 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728011990 con 0x7f6740140950 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f673e7fc700 1 -- 172.21.15.153:0/3431289562 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728011d20 con 0x7f6740140950 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 -- 172.21.15.153:0/3431289562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6740140950 msgr2=0x7f67400b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3431289562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6740140950 0x7f67400b2930 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6740144350 tx=0x7f6728009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 -- 172.21.15.153:0/3431289562 shutdown_connections 2023-11-28T13:02:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3431289562 >> v2:172.21.15.177:3300/0 conn(0x7f67400b2e70 0x7f674006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3431289562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6740140950 0x7f67400b2930 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 -- 172.21.15.153:0/3431289562 >> 172.21.15.153:0/3431289562 conn(0x7f674006aed0 msgr2=0x7f674006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:49.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 -- 172.21.15.153:0/3431289562 shutdown_connections 2023-11-28T13:02:49.667 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.663+0000 7f6745e6e700 1 -- 172.21.15.153:0/3431289562 wait complete. 2023-11-28T13:02:49.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.664+0000 7f6745e6e700 1 Processor -- start 2023-11-28T13:02:49.669 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.665+0000 7f6745e6e700 1 -- start start 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.665+0000 7f6745e6e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67400b2e70 0x7f67400c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.665+0000 7f6745e6e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67400cd8c0 0x7f67401efb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.665+0000 7f6745e6e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6740140950 0x7f67400ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.665+0000 7f6745e6e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400c8fd0 con 0x7f67400cd8c0 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.665+0000 7f6745e6e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674015c2f0 con 0x7f67400b2e70 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.665+0000 7f6745e6e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67400c9270 con 0x7f6740140950 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67400cd8c0 0x7f67401efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.670 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67400cd8c0 0x7f67401efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41982/0 (socket says 172.21.15.153:41982) 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 -- 172.21.15.153:0/3662285561 learned_addr learned my addr 172.21.15.153:0/3662285561 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673f7fe700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67400b2e70 0x7f67400c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673ffff700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6740140950 0x7f67400ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 -- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6740140950 msgr2=0x7f67400ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6740140950 0x7f67400ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 -- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67400b2e70 msgr2=0x7f67400c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67400b2e70 0x7f67400c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 -- 172.21.15.153:0/3662285561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67280094a0 con 0x7f67400cd8c0 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f673effd700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67400cd8c0 0x7f67401efb60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6728009b60 tx=0x7f6728004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f6737fff700 1 -- 172.21.15.153:0/3662285561 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67280036a0 con 0x7f67400cd8c0 2023-11-28T13:02:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f6737fff700 1 -- 172.21.15.153:0/3662285561 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672801e070 con 0x7f67400cd8c0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f6737fff700 1 -- 172.21.15.153:0/3662285561 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728003a20 con 0x7f67400cd8c0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67400b35f0 con 0x7f67400cd8c0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.666+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67400b3310 con 0x7f67400cd8c0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.667+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6740140150 con 0x7f67400cd8c0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.668+0000 7f6737fff700 1 -- 172.21.15.153:0/3662285561 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f672800f050 con 0x7f67400cd8c0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.668+0000 7f6737fff700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f67200682f0 0x7f672006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.668+0000 7f6737fff700 1 -- 172.21.15.153:0/3662285561 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f67280868a0 con 0x7f67400cd8c0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.668+0000 7f673f7fe700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f67200682f0 0x7f672006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:49.672 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.668+0000 7f673f7fe700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f67200682f0 0x7f672006a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6730007fd0 tx=0x7f6730009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:49.673 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.672+0000 7f6737fff700 1 -- 172.21.15.153:0/3662285561 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6728053400 con 0x7f67400cd8c0 2023-11-28T13:02:49.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:49.872+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f67400c9470 con 0x7f67400cd8c0 2023-11-28T13:02:50.655 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.654+0000 7f6737fff700 1 -- 172.21.15.153:0/3662285561 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6728020020 con 0x7f67400cd8c0 2023-11-28T13:02:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f67200682f0 msgr2=0x7f672006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f67200682f0 0x7f672006a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6730007fd0 tx=0x7f6730009040 comp rx=0 tx=0).stop 2023-11-28T13:02:50.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67400cd8c0 msgr2=0x7f67401efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67400cd8c0 0x7f67401efb60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6728009b60 tx=0x7f6728004750 comp rx=0 tx=0).stop 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 shutdown_connections 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6740140950 0x7f67400ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67400b2e70 0x7f67400c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f67200682f0 0x7f672006a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 --2- 172.21.15.153:0/3662285561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f67400cd8c0 0x7f67401efb60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.656+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 >> 172.21.15.153:0/3662285561 conn(0x7f674006aed0 msgr2=0x7f674013ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.657+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 shutdown_connections 2023-11-28T13:02:50.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.657+0000 7f6745e6e700 1 -- 172.21.15.153:0/3662285561 wait complete. 2023-11-28T13:02:50.659 INFO:teuthology.orchestra.run.smithi153.stderr: removed all blocklist entries 2023-11-28T13:02:50.671 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:02:50.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.795+0000 7fc379143700 1 Processor -- start 2023-11-28T13:02:50.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.795+0000 7fc379143700 1 -- start start 2023-11-28T13:02:50.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc379143700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc3740ce040 0x7fc37415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:50.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc379143700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc37406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:50.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc379143700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37406b420 con 0x7fc3740b2930 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc379143700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37406b700 con 0x7fc3740ce040 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc379143700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406b9e0 con 0x7fc3740d0a00 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc3740ce040 0x7fc37415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc3740ce040 0x7fc37415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41992/0 (socket says 172.21.15.153:41992) 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 -- 172.21.15.153:0/1305994383 learned_addr learned my addr 172.21.15.153:0/1305994383 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc37259c700 1 --2- 172.21.15.153:0/1305994383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc37406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 -- 172.21.15.153:0/1305994383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 msgr2=0x7fc37406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 --2- 172.21.15.153:0/1305994383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc37406f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 -- 172.21.15.153:0/1305994383 >> v1:172.21.15.177:6789/0 conn(0x7fc3740b2930 legacy=0x7fc37406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:50.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 -- 172.21.15.153:0/1305994383 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37406bcc0 con 0x7fc3740ce040 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc372d9d700 1 --2- 172.21.15.153:0/1305994383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc37415df10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc364006fd0 tx=0x7fc364009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f6c247aac06df3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc371d9b700 1 -- 172.21.15.153:0/1305994383 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364005e30 con 0x7fc3740ce040 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc371d9b700 1 -- 172.21.15.153:0/1305994383 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364006030 con 0x7fc3740ce040 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.796+0000 7fc371d9b700 1 -- 172.21.15.153:0/1305994383 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640063c0 con 0x7fc3740ce040 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 -- 172.21.15.153:0/1305994383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 msgr2=0x7fc37415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 --2- 172.21.15.153:0/1305994383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc37415df10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc364006fd0 tx=0x7fc364009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 -- 172.21.15.153:0/1305994383 shutdown_connections 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 --2- 172.21.15.153:0/1305994383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc37415df10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 --2- 172.21.15.153:0/1305994383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc37406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 -- 172.21.15.153:0/1305994383 >> 172.21.15.153:0/1305994383 conn(0x7fc37406aed0 msgr2=0x7fc37406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:50.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 -- 172.21.15.153:0/1305994383 shutdown_connections 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.797+0000 7fc379143700 1 -- 172.21.15.153:0/1305994383 wait complete. 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.798+0000 7fc379143700 1 Processor -- start 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.798+0000 7fc379143700 1 -- start start 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc379143700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3740b2930 0x7fc3740c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc379143700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc3741f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc379143700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc374141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc379143700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374152820 con 0x7fc3740ce040 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc379143700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c69d0 con 0x7fc3740b2930 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc379143700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c6cb0 con 0x7fc3740d0a00 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc37259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc3741f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc37259c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc3741f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42000/0 (socket says 172.21.15.153:42000) 2023-11-28T13:02:50.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.799+0000 7fc37259c700 1 -- 172.21.15.153:0/3094923338 learned_addr learned my addr 172.21.15.153:0/3094923338 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37359e700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc374141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc372d9d700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3740b2930 0x7fc3740c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37259c700 1 -- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 msgr2=0x7fc374141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37259c700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc374141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37259c700 1 -- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3740b2930 msgr2=0x7fc3740c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37259c700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3740b2930 0x7fc3740c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37259c700 1 -- 172.21.15.153:0/3094923338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364009500 con 0x7fc3740ce040 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37359e700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc374141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc372d9d700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3740b2930 0x7fc3740c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:50.805 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc37259c700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc3741f2320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc36800e810 tx=0x7fc36800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:50.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc35b7fe700 1 -- 172.21.15.153:0/3094923338 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368013610 con 0x7fc3740ce040 2023-11-28T13:02:50.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc35b7fe700 1 -- 172.21.15.153:0/3094923338 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc368013770 con 0x7fc3740ce040 2023-11-28T13:02:50.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37406fd90 con 0x7fc3740ce040 2023-11-28T13:02:50.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc35b7fe700 1 -- 172.21.15.153:0/3094923338 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368013b20 con 0x7fc3740ce040 2023-11-28T13:02:50.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.800+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37406bfa0 con 0x7fc3740ce040 2023-11-28T13:02:50.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.801+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37406ff20 con 0x7fc3740ce040 2023-11-28T13:02:50.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.801+0000 7fc35b7fe700 1 -- 172.21.15.153:0/3094923338 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc368023070 con 0x7fc3740ce040 2023-11-28T13:02:50.808 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.802+0000 7fc35b7fe700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc360068340 0x7fc36006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:50.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.802+0000 7fc35b7fe700 1 -- 172.21.15.153:0/3094923338 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc3680855f0 con 0x7fc3740ce040 2023-11-28T13:02:50.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.802+0000 7fc372d9d700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc360068340 0x7fc36006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:50.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.802+0000 7fc372d9d700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc360068340 0x7fc36006a7f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc364013040 tx=0x7fc364008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:50.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:50.806+0000 7fc35b7fe700 1 -- 172.21.15.153:0/3094923338 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc3680521a0 con 0x7fc3740ce040 2023-11-28T13:02:51.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.060+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc374068690 con 0x7fc3740ce040 2023-11-28T13:02:51.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.060+0000 7fc35b7fe700 1 -- 172.21.15.153:0/3094923338 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fc3680557c0 con 0x7fc3740ce040 2023-11-28T13:02:51.061 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "auth_dump": [ 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.a", 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlYBAHIBAA4jaMav4CAatojxXY6AkGPw==", 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.062 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.b", 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVl9vBaLBAAD84m+iSSScY6ETZRBpBUEg==", 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.063 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.c", 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlG5hJJxAAtFV9HSiLJClNn3nmB7ORSg==", 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mds.d", 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlvAC6MxAAX0N4lN+NKADdjjjROw+Zbw==", 2023-11-28T13:02:51.064 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.0", 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAq5GVlatZbJhAA2RQenckwVd7P0kBpzW08Lg==", 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.065 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.1", 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAt5GVlf73BJRAA51B4aDKKKVO+yo0+jQ20Yw==", 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.066 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.2", 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAw5GVl3M9qJRAAa2Jhr7Cp6vN/fRKJpMzCOg==", 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.067 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.3", 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAz5GVlWVgHJRAAlP6Gc1QZkzufchXRrkLkbA==", 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.4", 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA55GVlLb5hJRAAMMymIqHNw8oeVW8s3BrtIg==", 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.068 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.5", 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA85GVlwsLjJBAAb/1bWARS8ILpFdBGemlcoA==", 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.069 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.6", 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQA/5GVlwXx5JBAAXOS8JqPaXyQ7sj+2t9tsPQ==", 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.070 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "osd.7", 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQBC5GVl5q+qJBAAMc7ZUInNCMJmf0lC22s1QQ==", 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile osd", 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.071 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.0", 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlR2IBOBAAJBEAQ3PnTN6KaU44zoHW7Q==", 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow r", 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.072 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.1", 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAn5GVlTvYJAxAA68QZff0NcXXeMV7aGZoBfQ==", 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.2", 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlsT5YOxAAZ6ep21AlkI2znaE9wyskxA==", 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.073 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.3", 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAn5GVlxW63CBAApiPmiN4tN3S2dZH/bCKS/Q==", 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow", 2023-11-28T13:02:51.074 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow r", 2023-11-28T13:02:51.075 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow rw", 2023-11-28T13:02:51.075 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow rwx" 2023-11-28T13:02:51.075 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.075 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "client.admin", 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlE1RYARAA5UHDpYgXbOEuWdAhlYnmbA==", 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": "allow *", 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow *", 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.077 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mgr.x", 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVlKLmuHBAANErUDgd4g3Yysqk/sPAbrA==", 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile mgr", 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:51.078 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: "entity": "mgr.y", 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: "key": "AQAm5GVld7eyFhAAafZalFXR/4jRd5fUct7T2Q==", 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": { 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": "allow *", 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": "allow profile mgr", 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": "allow *" 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:51.079 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2023-11-28T13:02:51.080 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:02:51.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc360068340 msgr2=0x7fc36006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc360068340 0x7fc36006a7f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc364013040 tx=0x7fc364008070 comp rx=0 tx=0).stop 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 msgr2=0x7fc3741f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc3741f2320 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc36800e810 tx=0x7fc36800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 shutdown_connections 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3740d0a00 0x7fc374141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3740b2930 0x7fc3740c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc360068340 0x7fc36006a7f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 --2- 172.21.15.153:0/3094923338 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3740ce040 0x7fc3741f2320 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 >> 172.21.15.153:0/3094923338 conn(0x7fc37406aed0 msgr2=0x7fc3740cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 shutdown_connections 2023-11-28T13:02:51.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.063+0000 7fc379143700 1 -- 172.21.15.153:0/3094923338 wait complete. 2023-11-28T13:02:51.083 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:51.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.204+0000 7fca2b33b700 1 Processor -- start 2023-11-28T13:02:51.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.204+0000 7fca2b33b700 1 -- start start 2023-11-28T13:02:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca2b33b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 0x7fca240af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca2b33b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fca2406ba90 0x7fca240ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca2b33b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca240cc5f0 con 0x7fca240cbdf0 2023-11-28T13:02:51.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca2b33b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2415d470 con 0x7fca2406ba90 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca2b33b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415d750 con 0x7fca2415cef0 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca288d6700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fca240cbdf0 0x7fca2406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60170/0 (socket says 172.21.15.153:60170) 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca288d6700 1 -- 172.21.15.153:0/2677881763 learned_addr learned my addr 172.21.15.153:0/2677881763 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886696776 0 0) 0x7fca240cc5f0 con 0x7fca240cbdf0 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca100035b0 con 0x7fca240cbdf0 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca298d8700 1 --2- 172.21.15.153:0/2677881763 >> v2:172.21.15.177:3300/0 conn(0x7fca2406ba90 0x7fca240ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca290d7700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 0x7fca240af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1886942693 0 0) 0x7fca100035b0 con 0x7fca240cbdf0 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 msgr2=0x7fca240af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:51.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 0x7fca240af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 >> v2:172.21.15.177:3300/0 conn(0x7fca2406ba90 msgr2=0x7fca240ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 --2- 172.21.15.153:0/2677881763 >> v2:172.21.15.177:3300/0 conn(0x7fca2406ba90 0x7fca240ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2415da30 con 0x7fca240cbdf0 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fca14001670 con 0x7fca240cbdf0 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 >> v1:172.21.15.153:6789/0 conn(0x7fca240cbdf0 legacy=0x7fca2406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.205+0000 7fca1bfff700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 0x7fca10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca10005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415d470 con 0x7fca10004d90 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415d750 con 0x7fca10003a00 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1000dc20 con 0x7fca2415cef0 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca298d8700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca10005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca288d6700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 0x7fca10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca290d7700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca298d8700 1 -- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 msgr2=0x7fca10004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca298d8700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca2415cef0 0x7fca10004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca298d8700 1 -- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca10003a00 msgr2=0x7fca10003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca298d8700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca298d8700 1 -- 172.21.15.153:0/2677881763 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100035b0 con 0x7fca10004d90 2023-11-28T13:02:51.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca290d7700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca10003a00 0x7fca10003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:51.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca298d8700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca10005200 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009fc0 tx=0x7fca10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5796da1a8337fb9a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00ebd0 con 0x7fca10004d90 2023-11-28T13:02:51.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c00f040 con 0x7fca10004d90 2023-11-28T13:02:51.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.206+0000 7fca1bfff700 1 -- 172.21.15.153:0/2677881763 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c014480 con 0x7fca10004d90 2023-11-28T13:02:51.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.207+0000 7fca2b33b700 1 -- 172.21.15.153:0/2677881763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 msgr2=0x7fca10005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.207+0000 7fca2b33b700 1 --2- 172.21.15.153:0/2677881763 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca10005200 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009fc0 tx=0x7fca10005880 comp rx=0 tx=0).stop 2023-11-28T13:02:51.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.207+0000 7fca298d8700 1 -- 172.21.15.153:0/2677881763 reap_dead start 2023-11-28T13:02:51.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.207+0000 7fca2b33b700 1 -- 172.21.15.153:0/2677881763 shutdown_connections 2023-11-28T13:02:51.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.207+0000 7fca2b33b700 1 -- 172.21.15.153:0/2677881763 >> 172.21.15.153:0/2677881763 conn(0x7fca240d0000 msgr2=0x7fca240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:51.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.207+0000 7fca2b33b700 1 -- 172.21.15.153:0/2677881763 shutdown_connections 2023-11-28T13:02:51.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.207+0000 7fca2b33b700 1 -- 172.21.15.153:0/2677881763 wait complete. 2023-11-28T13:02:51.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.208+0000 7fca2b33b700 1 Processor -- start 2023-11-28T13:02:51.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.208+0000 7fca2b33b700 1 -- start start 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca2b33b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca241595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca2b33b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca240b5940 0x7fca240c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca2b33b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca240b4a90 0x7fca240cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca2b33b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca24159b00 con 0x7fca10004d90 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca2b33b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca240c8d70 con 0x7fca240b4a90 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca2b33b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca240b3fb0 con 0x7fca240b5940 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca241595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca241595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42020/0 (socket says 172.21.15.153:42020) 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 -- 172.21.15.153:0/1617794144 learned_addr learned my addr 172.21.15.153:0/1617794144 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca298d8700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca240b4a90 0x7fca240cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 -- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca240b5940 msgr2=0x7fca240c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca240b5940 0x7fca240c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 -- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca240b4a90 msgr2=0x7fca240cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca240b4a90 0x7fca240cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 -- 172.21.15.153:0/1617794144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2415dff0 con 0x7fca10004d90 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca298d8700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca240b4a90 0x7fca240cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca290d7700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca241595c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fca20010850 tx=0x7fca20010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.209+0000 7fca19ffb700 1 -- 172.21.15.153:0/1617794144 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20010e50 con 0x7fca10004d90 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.210+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2415dd10 con 0x7fca10004d90 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.210+0000 7fca19ffb700 1 -- 172.21.15.153:0/1617794144 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20004020 con 0x7fca10004d90 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.210+0000 7fca19ffb700 1 -- 172.21.15.153:0/1617794144 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca200043b0 con 0x7fca10004d90 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.210+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca240af400 con 0x7fca10004d90 2023-11-28T13:02:51.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.211+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca240b48e0 con 0x7fca10004d90 2023-11-28T13:02:51.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.212+0000 7fca19ffb700 1 -- 172.21.15.153:0/1617794144 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fca20025070 con 0x7fca10004d90 2023-11-28T13:02:51.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.212+0000 7fca19ffb700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca080682f0 0x7fca0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.212+0000 7fca19ffb700 1 -- 172.21.15.153:0/1617794144 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fca20085f90 con 0x7fca10004d90 2023-11-28T13:02:51.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.212+0000 7fca288d6700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca080682f0 0x7fca0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.213+0000 7fca288d6700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca080682f0 0x7fca0806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca240cb800 tx=0x7fca14000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.215+0000 7fca19ffb700 1 -- 172.21.15.153:0/1617794144 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fca20052b40 con 0x7fca10004d90 2023-11-28T13:02:51.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.399+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca240d0e60 con 0x7fca10004d90 2023-11-28T13:02:51.401 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.400+0000 7fca19ffb700 1 -- 172.21.15.153:0/1617794144 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+14315 (secure 0 0 0) 0x7fca20056160 con 0x7fca10004d90 2023-11-28T13:02:51.401 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:51.401 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":59,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:02:50.648436+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca080682f0 msgr2=0x7fca0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca080682f0 0x7fca0806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca240cb800 tx=0x7fca14000ed0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 msgr2=0x7fca241595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca241595c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fca20010850 tx=0x7fca20010c10 comp rx=0 tx=0).stop 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 shutdown_connections 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fca080682f0 0x7fca0806a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca10004d90 0x7fca241595c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca240b4a90 0x7fca240cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 --2- 172.21.15.153:0/1617794144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca240b5940 0x7fca240c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 >> 172.21.15.153:0/1617794144 conn(0x7fca240d0000 msgr2=0x7fca240afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:51.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 shutdown_connections 2023-11-28T13:02:51.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.403+0000 7fca2b33b700 1 -- 172.21.15.153:0/1617794144 wait complete. 2023-11-28T13:02:51.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:02:51.416 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:02:51.540 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.539+0000 7fcc151be700 1 Processor -- start 2023-11-28T13:02:51.540 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.539+0000 7fcc151be700 1 -- start start 2023-11-28T13:02:51.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc151be700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcc1006dee0 0x7fcc1006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc151be700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10160540 0x7fcc101609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc151be700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1015d060 con 0x7fcc10071040 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc151be700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1015d340 con 0x7fcc1006dee0 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc151be700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1015d620 con 0x7fcc10160540 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0e59c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcc1006dee0 0x7fcc1006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0ed9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcc10071040 0x7fcc10071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60190/0 (socket says 172.21.15.153:60190) 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0ed9d700 1 -- 172.21.15.153:0/3811865250 learned_addr learned my addr 172.21.15.153:0/3811865250 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0e59c700 1 -- 172.21.15.153:0/3811865250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10160540 msgr2=0x7fcc101609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0e59c700 1 --2- 172.21.15.153:0/3811865250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10160540 0x7fcc101609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0e59c700 1 -- 172.21.15.153:0/3811865250 >> v1:172.21.15.153:6789/0 conn(0x7fcc10071040 legacy=0x7fcc10071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0e59c700 1 -- 172.21.15.153:0/3811865250 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1015d900 con 0x7fcc1006dee0 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.540+0000 7fcc0e59c700 1 --2- 172.21.15.153:0/3811865250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1006e330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009840 tx=0x7fcc00009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ec416b59c7b673 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc0dd9b700 1 -- 172.21.15.153:0/3811865250 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00011830 con 0x7fcc1006dee0 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc0dd9b700 1 -- 172.21.15.153:0/3811865250 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00011990 con 0x7fcc1006dee0 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc0dd9b700 1 -- 172.21.15.153:0/3811865250 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00011d20 con 0x7fcc1006dee0 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 -- 172.21.15.153:0/3811865250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 msgr2=0x7fcc1006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 --2- 172.21.15.153:0/3811865250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1006e330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009840 tx=0x7fcc00009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 -- 172.21.15.153:0/3811865250 shutdown_connections 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 --2- 172.21.15.153:0/3811865250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10160540 0x7fcc101609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 --2- 172.21.15.153:0/3811865250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1006e330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 -- 172.21.15.153:0/3811865250 >> 172.21.15.153:0/3811865250 conn(0x7fcc100d0000 msgr2=0x7fcc100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:51.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 -- 172.21.15.153:0/3811865250 shutdown_connections 2023-11-28T13:02:51.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.541+0000 7fcc151be700 1 -- 172.21.15.153:0/3811865250 wait complete. 2023-11-28T13:02:51.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.543+0000 7fcc151be700 1 Processor -- start 2023-11-28T13:02:51.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.543+0000 7fcc151be700 1 -- start start 2023-11-28T13:02:51.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc151be700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc151be700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10071040 0x7fcc100c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc151be700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc10160540 0x7fcc1015e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc151be700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1015bc50 con 0x7fcc1006dee0 2023-11-28T13:02:51.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc151be700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100c6db0 con 0x7fcc10160540 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc151be700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc101f6e20 con 0x7fcc10071040 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1015b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42032/0 (socket says 172.21.15.153:42032) 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 -- 172.21.15.153:0/2316366783 learned_addr learned my addr 172.21.15.153:0/2316366783 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0e59c700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10071040 0x7fcc100c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0f59e700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc10160540 0x7fcc1015e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 -- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10071040 msgr2=0x7fcc100c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10071040 0x7fcc100c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 -- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc10160540 msgr2=0x7fcc1015e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc10160540 0x7fcc1015e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0ed9d700 1 -- 172.21.15.153:0/2316366783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc000094a0 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0e59c700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10071040 0x7fcc100c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.544+0000 7fcc0f59e700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc10160540 0x7fcc1015e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.545+0000 7fcc0ed9d700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1015b710 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c480 tx=0x7fcc0400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.545+0000 7fcbf77fe700 1 -- 172.21.15.153:0/2316366783 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04005d70 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.545+0000 7fcbf77fe700 1 -- 172.21.15.153:0/2316366783 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc040070d0 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.545+0000 7fcbf77fe700 1 -- 172.21.15.153:0/2316366783 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04007440 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.545+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1015ccd0 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.545+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1006e870 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.546+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1006a270 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.546+0000 7fcbf77fe700 1 -- 172.21.15.153:0/2316366783 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcc04019030 con 0x7fcc1006dee0 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.546+0000 7fcbf77fe700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbfc0682f0 0x7fcbfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.547+0000 7fcbf77fe700 1 -- 172.21.15.153:0/2316366783 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcc04084c10 con 0x7fcc1006dee0 2023-11-28T13:02:51.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.547+0000 7fcc0e59c700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbfc0682f0 0x7fcbfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.547+0000 7fcc0e59c700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbfc0682f0 0x7fcbfc06a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcc000090b0 tx=0x7fcc00008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.551+0000 7fcbf77fe700 1 -- 172.21.15.153:0/2316366783 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcc0404b5e0 con 0x7fcc1006dee0 2023-11-28T13:02:51.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.732+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc1004e720 con 0x7fcc1006dee0 2023-11-28T13:02:51.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.733+0000 7fcbf77fe700 1 -- 172.21.15.153:0/2316366783 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc040170a0 con 0x7fcc1006dee0 2023-11-28T13:02:51.734 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbfc0682f0 msgr2=0x7fcbfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbfc0682f0 0x7fcbfc06a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcc000090b0 tx=0x7fcc00008030 comp rx=0 tx=0).stop 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 msgr2=0x7fcc1015b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1015b710 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400c480 tx=0x7fcc0400c840 comp rx=0 tx=0).stop 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 shutdown_connections 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcbfc0682f0 0x7fcbfc06a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc1006dee0 0x7fcc1015b710 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc10160540 0x7fcc1015e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 --2- 172.21.15.153:0/2316366783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc10071040 0x7fcc100c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 >> 172.21.15.153:0/2316366783 conn(0x7fcc100d0000 msgr2=0x7fcc1015f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:51.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 shutdown_connections 2023-11-28T13:02:51.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.735+0000 7fcc151be700 1 -- 172.21.15.153:0/2316366783 wait complete. 2023-11-28T13:02:51.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:02:51.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.870+0000 7f5e25505700 1 Processor -- start 2023-11-28T13:02:51.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.870+0000 7f5e25505700 1 -- start start 2023-11-28T13:02:51.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.870+0000 7f5e25505700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5e200d0a80 0x7f5e200cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.870+0000 7f5e25505700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e200cf5e0 con 0x7f5e2015ca50 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.870+0000 7f5e25505700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e200cecb0 con 0x7f5e2006ba90 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.870+0000 7f5e25505700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e200cd470 con 0x7f5e200d0a80 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1f7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5e200d0a80 0x7f5e200cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1effd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5e2015ca50 0x7f5e2006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60204/0 (socket says 172.21.15.153:60204) 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1effd700 1 -- 172.21.15.153:0/3667115561 learned_addr learned my addr 172.21.15.153:0/3667115561 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1f7fe700 1 -- 172.21.15.153:0/3667115561 >> v1:172.21.15.177:6789/0 conn(0x7f5e2006ba90 legacy=0x7f5e200cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1f7fe700 1 -- 172.21.15.153:0/3667115561 >> v1:172.21.15.153:6789/0 conn(0x7f5e2015ca50 legacy=0x7f5e2006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1f7fe700 1 -- 172.21.15.153:0/3667115561 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e200cef90 con 0x7f5e200d0a80 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1dffb700 1 -- 172.21.15.153:0/3667115561 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177919424 0 0) 0x7f5e200cf5e0 con 0x7f5e2015ca50 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1f7fe700 1 --2- 172.21.15.153:0/3667115561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e200d0a80 0x7f5e200cce10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009d90 tx=0x7f5e1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218075fbeb8272dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1dffb700 1 -- 172.21.15.153:0/3667115561 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800e070 con 0x7f5e200d0a80 2023-11-28T13:02:51.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1dffb700 1 -- 172.21.15.153:0/3667115561 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1800bbd0 con 0x7f5e200d0a80 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.871+0000 7f5e1dffb700 1 -- 172.21.15.153:0/3667115561 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18012430 con 0x7f5e200d0a80 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.872+0000 7f5e25505700 1 -- 172.21.15.153:0/3667115561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e200d0a80 msgr2=0x7f5e200cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.872+0000 7f5e25505700 1 --2- 172.21.15.153:0/3667115561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e200d0a80 0x7f5e200cce10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009d90 tx=0x7f5e1800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.872+0000 7f5e25505700 1 -- 172.21.15.153:0/3667115561 shutdown_connections 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.872+0000 7f5e25505700 1 --2- 172.21.15.153:0/3667115561 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e200d0a80 0x7f5e200cce10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.872+0000 7f5e25505700 1 -- 172.21.15.153:0/3667115561 >> 172.21.15.153:0/3667115561 conn(0x7f5e2009a230 msgr2=0x7f5e2008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.872+0000 7f5e25505700 1 -- 172.21.15.153:0/3667115561 shutdown_connections 2023-11-28T13:02:51.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.872+0000 7f5e25505700 1 -- 172.21.15.153:0/3667115561 wait complete. 2023-11-28T13:02:51.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.873+0000 7f5e25505700 1 Processor -- start 2023-11-28T13:02:51.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.873+0000 7f5e25505700 1 -- start start 2023-11-28T13:02:51.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e25505700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e2006ba90 0x7f5e200c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e25505700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e200d0a80 0x7f5e2014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e25505700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2015ca50 0x7f5e200b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e25505700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200c5190 con 0x7f5e2006ba90 2023-11-28T13:02:51.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e25505700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2014f9b0 con 0x7f5e200d0a80 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e25505700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200b2bf0 con 0x7f5e2015ca50 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e2006ba90 0x7f5e200c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e2006ba90 0x7f5e200c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42048/0 (socket says 172.21.15.153:42048) 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 -- 172.21.15.153:0/1517385693 learned_addr learned my addr 172.21.15.153:0/1517385693 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1f7fe700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2015ca50 0x7f5e200b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1e7fc700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e200d0a80 0x7f5e2014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 -- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2015ca50 msgr2=0x7f5e200b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2015ca50 0x7f5e200b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 -- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e200d0a80 msgr2=0x7f5e2014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e200d0a80 0x7f5e2014f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 -- 172.21.15.153:0/1517385693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18009a70 con 0x7f5e2006ba90 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e1effd700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e2006ba90 0x7f5e200c4a90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c520 tx=0x7f5e1000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e077fe700 1 -- 172.21.15.153:0/1517385693 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004c80 con 0x7f5e2006ba90 2023-11-28T13:02:51.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.874+0000 7f5e077fe700 1 -- 172.21.15.153:0/1517385693 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10004e00 con 0x7f5e2006ba90 2023-11-28T13:02:51.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.875+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2015ce20 con 0x7f5e2006ba90 2023-11-28T13:02:51.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.875+0000 7f5e077fe700 1 -- 172.21.15.153:0/1517385693 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10007310 con 0x7f5e2006ba90 2023-11-28T13:02:51.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.875+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e200cf270 con 0x7f5e2006ba90 2023-11-28T13:02:51.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.875+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2015cfb0 con 0x7f5e2006ba90 2023-11-28T13:02:51.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.876+0000 7f5e077fe700 1 -- 172.21.15.153:0/1517385693 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e1001a020 con 0x7f5e2006ba90 2023-11-28T13:02:51.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.876+0000 7f5e077fe700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:51.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.876+0000 7f5e077fe700 1 -- 172.21.15.153:0/1517385693 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5e100855c0 con 0x7f5e2006ba90 2023-11-28T13:02:51.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.876+0000 7f5e1e7fc700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:51.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.877+0000 7f5e1e7fc700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e200b6fd0 tx=0x7f5e14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:51.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:51.880+0000 7f5e077fe700 1 -- 172.21.15.153:0/1517385693 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5e10052170 con 0x7f5e2006ba90 2023-11-28T13:02:52.062 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.061+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5e200745c0 con 0x7f5e2006ba90 2023-11-28T13:02:52.062 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.062+0000 7f5e077fe700 1 -- 172.21.15.153:0/1517385693 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5e10055790 con 0x7f5e2006ba90 2023-11-28T13:02:52.063 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:02:52.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.063+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5e0c0682f0 msgr2=0x7f5e0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.063+0000 7f5e25505700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e200b6fd0 tx=0x7f5e14009040 comp rx=0 tx=0).stop 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.063+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e2006ba90 msgr2=0x7f5e200c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.063+0000 7f5e25505700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e2006ba90 0x7f5e200c4a90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c520 tx=0x7f5e1000c8e0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 shutdown_connections 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e2006ba90 0x7f5e200c4a90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2015ca50 0x7f5e200b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 --2- 172.21.15.153:0/1517385693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e200d0a80 0x7f5e2014f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 >> 172.21.15.153:0/1517385693 conn(0x7f5e2009a230 msgr2=0x7f5e2015bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 shutdown_connections 2023-11-28T13:02:52.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.064+0000 7f5e25505700 1 -- 172.21.15.153:0/1517385693 wait complete. 2023-11-28T13:02:52.076 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:02:52.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.196+0000 7fbc2a105700 1 Processor -- start 2023-11-28T13:02:52.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.196+0000 7fbc2a105700 1 -- start start 2023-11-28T13:02:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc2a105700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbc24071040 0x7fbc24071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc2a105700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbc24160540 0x7fbc24160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc2a105700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc24071950 con 0x7fbc2406dee0 2023-11-28T13:02:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc2a105700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2415d060 con 0x7fbc24160540 2023-11-28T13:02:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc2a105700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2415d340 con 0x7fbc24071040 2023-11-28T13:02:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbc24160540 0x7fbc24160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fbc24160540 0x7fbc24160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42058/0 (socket says 172.21.15.153:42058) 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 -- 172.21.15.153:0/1717064578 learned_addr learned my addr 172.21.15.153:0/1717064578 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc237fe700 1 --2- 172.21.15.153:0/1717064578 >> v2:172.21.15.177:3300/0 conn(0x7fbc24071040 0x7fbc24071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 -- 172.21.15.153:0/1717064578 >> v2:172.21.15.177:3300/0 conn(0x7fbc24071040 msgr2=0x7fbc24071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 --2- 172.21.15.153:0/1717064578 >> v2:172.21.15.177:3300/0 conn(0x7fbc24071040 0x7fbc24071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 -- 172.21.15.153:0/1717064578 >> v1:172.21.15.177:6789/0 conn(0x7fbc2406dee0 legacy=0x7fbc2406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 -- 172.21.15.153:0/1717064578 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2415d620 con 0x7fbc24160540 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc23fff700 1 --2- 172.21.15.153:0/1717064578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24160540 0x7fbc24160980 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbc240d0f50 tx=0x7fbc1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50ed38a4c8b7a63 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.197+0000 7fbc227fc700 1 -- 172.21.15.153:0/1717064578 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00b870 con 0x7fbc24160540 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc227fc700 1 -- 172.21.15.153:0/1717064578 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c00b9d0 con 0x7fbc24160540 2023-11-28T13:02:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc227fc700 1 -- 172.21.15.153:0/1717064578 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00bd60 con 0x7fbc24160540 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 -- 172.21.15.153:0/1717064578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24160540 msgr2=0x7fbc24160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 --2- 172.21.15.153:0/1717064578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24160540 0x7fbc24160980 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbc240d0f50 tx=0x7fbc1c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 -- 172.21.15.153:0/1717064578 shutdown_connections 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 --2- 172.21.15.153:0/1717064578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24160540 0x7fbc24160980 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 --2- 172.21.15.153:0/1717064578 >> v2:172.21.15.177:3300/0 conn(0x7fbc24071040 0x7fbc24071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 -- 172.21.15.153:0/1717064578 >> 172.21.15.153:0/1717064578 conn(0x7fbc240d0000 msgr2=0x7fbc240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 -- 172.21.15.153:0/1717064578 shutdown_connections 2023-11-28T13:02:52.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.198+0000 7fbc2a105700 1 -- 172.21.15.153:0/1717064578 wait complete. 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.199+0000 7fbc2a105700 1 Processor -- start 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc2a105700 1 -- start start 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc2a105700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2406dee0 0x7fbc24153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc2a105700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24071040 0x7fbc240b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc2a105700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc24160540 0x7fbc2415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc2a105700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c5c50 con 0x7fbc24071040 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc2a105700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c5040 con 0x7fbc2406dee0 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc2a105700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c5320 con 0x7fbc24160540 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc22ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24071040 0x7fbc240b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc22ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24071040 0x7fbc240b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42062/0 (socket says 172.21.15.153:42062) 2023-11-28T13:02:52.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.200+0000 7fbc22ffd700 1 -- 172.21.15.153:0/2481925942 learned_addr learned my addr 172.21.15.153:0/2481925942 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc237fe700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2406dee0 0x7fbc24153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc23fff700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc24160540 0x7fbc2415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc22ffd700 1 -- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc24160540 msgr2=0x7fbc2415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc22ffd700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc24160540 0x7fbc2415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc22ffd700 1 -- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2406dee0 msgr2=0x7fbc24153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc22ffd700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2406dee0 0x7fbc24153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc22ffd700 1 -- 172.21.15.153:0/2481925942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c009550 con 0x7fbc24071040 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc22ffd700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24071040 0x7fbc240b2e30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c450 tx=0x7fbc0c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc1bfff700 1 -- 172.21.15.153:0/2481925942 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0052d0 con 0x7fbc24071040 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc1bfff700 1 -- 172.21.15.153:0/2481925942 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c005430 con 0x7fbc24071040 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2415ccd0 con 0x7fbc24071040 2023-11-28T13:02:52.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc1bfff700 1 -- 172.21.15.153:0/2481925942 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0038c0 con 0x7fbc24071040 2023-11-28T13:02:52.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.201+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2415d900 con 0x7fbc24071040 2023-11-28T13:02:52.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.202+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2406a270 con 0x7fbc24071040 2023-11-28T13:02:52.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.202+0000 7fbc1bfff700 1 -- 172.21.15.153:0/2481925942 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbc0c019030 con 0x7fbc24071040 2023-11-28T13:02:52.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.203+0000 7fbc1bfff700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc04068320 0x7fbc0406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.203+0000 7fbc1bfff700 1 -- 172.21.15.153:0/2481925942 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbc0c084b70 con 0x7fbc24071040 2023-11-28T13:02:52.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.203+0000 7fbc237fe700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc04068320 0x7fbc0406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.203+0000 7fbc237fe700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc04068320 0x7fbc0406a7d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009a60 tx=0x7fbc14003a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:52.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.207+0000 7fbc1bfff700 1 -- 172.21.15.153:0/2481925942 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbc0c0516c0 con 0x7fbc24071040 2023-11-28T13:02:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.393+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbc240720a0 con 0x7fbc24071040 2023-11-28T13:02:52.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.393+0000 7fbc1bfff700 1 -- 172.21.15.153:0/2481925942 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbc0c054ce0 con 0x7fbc24071040 2023-11-28T13:02:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:52.394 INFO:teuthology.orchestra.run.smithi153.stdout:[ 2023-11-28T13:02:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 1, 2023-11-28T13:02:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": ".mgr" 2023-11-28T13:02:52.394 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:02:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:02:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "poolnum": 2, 2023-11-28T13:02:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: "poolname": "rbd" 2023-11-28T13:02:52.395 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:02:52.395 INFO:teuthology.orchestra.run.smithi153.stdout:] 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc04068320 msgr2=0x7fbc0406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc04068320 0x7fbc0406a7d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009a60 tx=0x7fbc14003a20 comp rx=0 tx=0).stop 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24071040 msgr2=0x7fbc240b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24071040 0x7fbc240b2e30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c450 tx=0x7fbc0c00c810 comp rx=0 tx=0).stop 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 shutdown_connections 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc24160540 0x7fbc2415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc2406dee0 0x7fbc24153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fbc04068320 0x7fbc0406a7d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 --2- 172.21.15.153:0/2481925942 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbc24071040 0x7fbc240b2e30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 >> 172.21.15.153:0/2481925942 conn(0x7fbc240d0000 msgr2=0x7fbc240cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:52.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 shutdown_connections 2023-11-28T13:02:52.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.395+0000 7fbc2a105700 1 -- 172.21.15.153:0/2481925942 wait complete. 2023-11-28T13:02:52.408 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:02:52.408 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:02:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.532+0000 7f36c4f5b700 1 Processor -- start 2023-11-28T13:02:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.532+0000 7f36c4f5b700 1 -- start start 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.532+0000 7f36c4f5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c015aba0 0x7f36c00b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36c4f5b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c015af70 con 0x7f36c00b6ae0 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36c4f5b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c00cc870 con 0x7f36c00d0140 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36c4f5b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00cfec0 con 0x7f36c015aba0 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36bdd9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f36c00b6ae0 0x7f36c00cf4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60234/0 (socket says 172.21.15.153:60234) 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36bdd9b700 1 -- 172.21.15.153:0/950511810 learned_addr learned my addr 172.21.15.153:0/950511810 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36be59c700 1 --2- 172.21.15.153:0/950511810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c015aba0 0x7f36c00b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36be59c700 1 -- 172.21.15.153:0/950511810 >> v1:172.21.15.177:6789/0 conn(0x7f36c00d0140 legacy=0x7f36c00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36be59c700 1 -- 172.21.15.153:0/950511810 >> v1:172.21.15.153:6789/0 conn(0x7f36c00b6ae0 legacy=0x7f36c00cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36be59c700 1 -- 172.21.15.153:0/950511810 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c00b4290 con 0x7f36c015aba0 2023-11-28T13:02:52.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.533+0000 7f36be59c700 1 --2- 172.21.15.153:0/950511810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c015aba0 0x7f36c00b3d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009870 tx=0x7f36b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924f9149025abbb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:52.538 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36bd59a700 1 -- 172.21.15.153:0/950511810 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0011830 con 0x7f36c015aba0 2023-11-28T13:02:52.538 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36bd59a700 1 -- 172.21.15.153:0/950511810 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b0011990 con 0x7f36c015aba0 2023-11-28T13:02:52.538 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36bd59a700 1 -- 172.21.15.153:0/950511810 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0011d20 con 0x7f36c015aba0 2023-11-28T13:02:52.539 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36c4f5b700 1 -- 172.21.15.153:0/950511810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c015aba0 msgr2=0x7f36c00b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.539 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/950511810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c015aba0 0x7f36c00b3d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009870 tx=0x7f36b0009b80 comp rx=0 tx=0).stop 2023-11-28T13:02:52.540 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36c4f5b700 1 -- 172.21.15.153:0/950511810 shutdown_connections 2023-11-28T13:02:52.540 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/950511810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c015aba0 0x7f36c00b3d50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.541 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36c4f5b700 1 -- 172.21.15.153:0/950511810 >> 172.21.15.153:0/950511810 conn(0x7f36c006df70 msgr2=0x7f36c006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:52.541 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36c4f5b700 1 -- 172.21.15.153:0/950511810 shutdown_connections 2023-11-28T13:02:52.541 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.538+0000 7f36c4f5b700 1 -- 172.21.15.153:0/950511810 wait complete. 2023-11-28T13:02:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 Processor -- start 2023-11-28T13:02:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 -- start start 2023-11-28T13:02:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c00d0140 0x7f36c00c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c015aba0 0x7f36c015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.543 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c00b5470 0x7f36c00b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00b5e50 con 0x7f36c015aba0 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c01f35a0 con 0x7f36c00b5470 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.540+0000 7f36c4f5b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c01f2730 con 0x7f36c00d0140 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c015aba0 0x7f36c015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c015aba0 0x7f36c015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42076/0 (socket says 172.21.15.153:42076) 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 -- 172.21.15.153:0/2310276239 learned_addr learned my addr 172.21.15.153:0/2310276239 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36be59c700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c00d0140 0x7f36c00c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bed9d700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c00b5470 0x7f36c00b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 -- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c00d0140 msgr2=0x7f36c00c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c00d0140 0x7f36c00c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 -- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c00b5470 msgr2=0x7f36c00b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:52.544 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c00b5470 0x7f36c00b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 -- 172.21.15.153:0/2310276239 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b00094a0 con 0x7f36c015aba0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bed9d700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c00b5470 0x7f36c00b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36bdd9b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c015aba0 0x7f36c015d2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c480 tx=0x7f36b400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36aeffd700 1 -- 172.21.15.153:0/2310276239 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4005d70 con 0x7f36c015aba0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c00ccaa0 con 0x7f36c015aba0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36aeffd700 1 -- 172.21.15.153:0/2310276239 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b40070d0 con 0x7f36c015aba0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36aeffd700 1 -- 172.21.15.153:0/2310276239 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4007440 con 0x7f36c015aba0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.541+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c00cfbf0 con 0x7f36c015aba0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.542+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c00ccd80 con 0x7f36c015aba0 2023-11-28T13:02:52.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.543+0000 7f36aeffd700 1 -- 172.21.15.153:0/2310276239 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f36b4019030 con 0x7f36c015aba0 2023-11-28T13:02:52.546 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.543+0000 7f36aeffd700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f36a00682f0 0x7f36a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:52.546 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.543+0000 7f36aeffd700 1 -- 172.21.15.153:0/2310276239 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f36b4084b20 con 0x7f36c015aba0 2023-11-28T13:02:52.546 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.543+0000 7f36be59c700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f36a00682f0 0x7f36a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:52.546 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.543+0000 7f36be59c700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f36a00682f0 0x7f36a006a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36b00097c0 tx=0x7f36b0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:52.548 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.547+0000 7f36aeffd700 1 -- 172.21.15.153:0/2310276239 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f36b404b5e0 con 0x7f36c015aba0 2023-11-28T13:02:52.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:52.729+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f36c01f36e0 con 0x7f36c015aba0 2023-11-28T13:02:53.547 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.547+0000 7f36aeffd700 1 -- 172.21.15.153:0/2310276239 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f36c01f36e0 con 0x7f36c015aba0 2023-11-28T13:02:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f36a00682f0 msgr2=0x7f36a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f36a00682f0 0x7f36a006a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36b00097c0 tx=0x7f36b0008030 comp rx=0 tx=0).stop 2023-11-28T13:02:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c015aba0 msgr2=0x7f36c015d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c015aba0 0x7f36c015d2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c480 tx=0x7f36b400c790 comp rx=0 tx=0).stop 2023-11-28T13:02:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 shutdown_connections 2023-11-28T13:02:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c00b5470 0x7f36c00b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36c00d0140 0x7f36c00c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f36a00682f0 0x7f36a006a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 --2- 172.21.15.153:0/2310276239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36c015aba0 0x7f36c015d2f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.550+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 >> 172.21.15.153:0/2310276239 conn(0x7f36c006df70 msgr2=0x7f36c0157a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.551+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 shutdown_connections 2023-11-28T13:02:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.551+0000 7f36c4f5b700 1 -- 172.21.15.153:0/2310276239 wait complete. 2023-11-28T13:02:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' created 2023-11-28T13:02:53.567 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:02:53.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.695+0000 7f99d7167700 1 Processor -- start 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.695+0000 7f99d7167700 1 -- start start 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.696+0000 7f99d7167700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 0x7f99d006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.696+0000 7f99d7167700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f99d006d0a0 0x7f99d00ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.696+0000 7f99d7167700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d006ae60 con 0x7f99d006b830 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.696+0000 7f99d7167700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d006a270 con 0x7f99d006d0a0 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.696+0000 7f99d7167700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006a530 con 0x7f99d006e9f0 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.696+0000 7f99cffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f99d006b830 0x7f99d006c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60246/0 (socket says 172.21.15.153:60246) 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.696+0000 7f99cffff700 1 -- 172.21.15.153:0/3610370858 learned_addr learned my addr 172.21.15.153:0/3610370858 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670827962 0 0) 0x7f99d006ae60 con 0x7f99d006b830 2023-11-28T13:02:53.707 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c00035b0 con 0x7f99d006b830 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99d4f03700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 0x7f99d006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99d5704700 1 --2- 172.21.15.153:0/3610370858 >> v2:172.21.15.177:3300/0 conn(0x7f99d006d0a0 0x7f99d00ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3367290858 0 0) 0x7f99c00035b0 con 0x7f99d006b830 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 msgr2=0x7f99d006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 0x7f99d006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 >> v2:172.21.15.177:3300/0 conn(0x7f99d006d0a0 msgr2=0x7f99d00ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 --2- 172.21.15.153:0/3610370858 >> v2:172.21.15.177:3300/0 conn(0x7f99d006d0a0 0x7f99d00ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d006a810 con 0x7f99d006b830 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f99bc001670 con 0x7f99d006b830 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 >> v1:172.21.15.153:6789/0 conn(0x7f99d006b830 legacy=0x7f99d006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:53.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99c0003a00 0x7f99c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c0004430 0x7f99c00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 0x7f99c0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006a270 con 0x7f99c0004430 2023-11-28T13:02:53.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006a530 con 0x7f99c0003a00 2023-11-28T13:02:53.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c000ba10 con 0x7f99d006e9f0 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cffff700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c0004430 0x7f99c00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99d4f03700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99c0003a00 0x7f99c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99d5704700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 0x7f99c0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cffff700 1 -- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 msgr2=0x7f99c0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cffff700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99d006e9f0 0x7f99c0004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cffff700 1 -- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99c0003a00 msgr2=0x7f99c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cffff700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99c0003a00 0x7f99c0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99cffff700 1 -- 172.21.15.153:0/3610370858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00035b0 con 0x7f99c0004430 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.697+0000 7f99d4f03700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99c0003a00 0x7f99c0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99cffff700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c0004430 0x7f99c00048f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c2f0 tx=0x7f99c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96083f7196e176 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007c40 con 0x7f99c0004430 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc00ce30 con 0x7f99c0004430 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99cf7fe700 1 -- 172.21.15.153:0/3610370858 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc005ba0 con 0x7f99c0004430 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99d7167700 1 -- 172.21.15.153:0/3610370858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c0004430 msgr2=0x7f99c00048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99d7167700 1 --2- 172.21.15.153:0/3610370858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99c0004430 0x7f99c00048f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00c2f0 tx=0x7f99c0005880 comp rx=0 tx=0).stop 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99d5704700 1 -- 172.21.15.153:0/3610370858 reap_dead start 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99d7167700 1 -- 172.21.15.153:0/3610370858 shutdown_connections 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99d7167700 1 -- 172.21.15.153:0/3610370858 >> 172.21.15.153:0/3610370858 conn(0x7f99d00b12b0 msgr2=0x7f99d00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99d7167700 1 -- 172.21.15.153:0/3610370858 shutdown_connections 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.698+0000 7f99d7167700 1 -- 172.21.15.153:0/3610370858 wait complete. 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.700+0000 7f99d7167700 1 Processor -- start 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.700+0000 7f99d7167700 1 -- start start 2023-11-28T13:02:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d7167700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99c0004430 0x7f99d00c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d7167700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d01528b0 0x7f99d00c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d7167700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99d00b0110 0x7f99d00b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d7167700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00b0ac0 con 0x7f99d01528b0 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d7167700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00c56c0 con 0x7f99d00b0110 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d7167700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00c6100 con 0x7f99c0004430 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99cffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d01528b0 0x7f99d00c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99cffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d01528b0 0x7f99d00c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42094/0 (socket says 172.21.15.153:42094) 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99cffff700 1 -- 172.21.15.153:0/2867716789 learned_addr learned my addr 172.21.15.153:0/2867716789 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d5704700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99d00b0110 0x7f99d00b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99d4f03700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99c0004430 0x7f99d00c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99cffff700 1 -- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99c0004430 msgr2=0x7f99d00c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99cffff700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99c0004430 0x7f99d00c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99cffff700 1 -- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99d00b0110 msgr2=0x7f99d00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.701+0000 7f99cffff700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99d00b0110 0x7f99d00b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99cffff700 1 -- 172.21.15.153:0/2867716789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d006edc0 con 0x7f99d01528b0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99d5704700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99d00b0110 0x7f99d00b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99cffff700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d01528b0 0x7f99d00c42b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f99bc001930 tx=0x7f99d0159580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99cd7fa700 1 -- 172.21.15.153:0/2867716789 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0070d0 con 0x7f99d01528b0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99cd7fa700 1 -- 172.21.15.153:0/2867716789 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc007250 con 0x7f99d01528b0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d006aaf0 con 0x7f99d01528b0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99cd7fa700 1 -- 172.21.15.153:0/2867716789 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007600 con 0x7f99d01528b0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.702+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d006ef50 con 0x7f99d01528b0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.703+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d01f0280 con 0x7f99d01528b0 2023-11-28T13:02:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.704+0000 7f99cd7fa700 1 -- 172.21.15.153:0/2867716789 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f99bc024030 con 0x7f99d01528b0 2023-11-28T13:02:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.704+0000 7f99cd7fa700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f99b40682f0 0x7f99b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.704+0000 7f99cd7fa700 1 -- 172.21.15.153:0/2867716789 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f99bc086cb0 con 0x7f99d01528b0 2023-11-28T13:02:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.704+0000 7f99d4f03700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f99b40682f0 0x7f99b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.704+0000 7f99d4f03700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f99b40682f0 0x7f99b406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99c40099b0 tx=0x7f99c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.708+0000 7f99cd7fa700 1 -- 172.21.15.153:0/2867716789 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99bc0537b0 con 0x7f99d01528b0 2023-11-28T13:02:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:53.891+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f99d00cd5b0 con 0x7f99d01528b0 2023-11-28T13:02:54.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.561+0000 7f99cd7fa700 1 -- 172.21.15.153:0/2867716789 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7f99bc007da0 con 0x7f99d01528b0 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.569+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f99b40682f0 msgr2=0x7f99b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.569+0000 7f99d7167700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f99b40682f0 0x7f99b406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99c40099b0 tx=0x7f99c4008040 comp rx=0 tx=0).stop 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.569+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d01528b0 msgr2=0x7f99d00c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.569+0000 7f99d7167700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d01528b0 0x7f99d00c42b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f99bc001930 tx=0x7f99d0159580 comp rx=0 tx=0).stop 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.570+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 shutdown_connections 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.570+0000 7f99d7167700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99d00b0110 0x7f99d00b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.570+0000 7f99d7167700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99c0004430 0x7f99d00c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.571 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.570+0000 7f99d7167700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f99b40682f0 0x7f99b406a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.570+0000 7f99d7167700 1 --2- 172.21.15.153:0/2867716789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99d01528b0 0x7f99d00c42b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.570+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 >> 172.21.15.153:0/2867716789 conn(0x7f99d00b12b0 msgr2=0x7f99d00ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:54.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.570+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 shutdown_connections 2023-11-28T13:02:54.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.571+0000 7f99d7167700 1 -- 172.21.15.153:0/2867716789 wait complete. 2023-11-28T13:02:54.574 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' created 2023-11-28T13:02:54.589 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:02:54.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.716+0000 7efec158a700 1 Processor -- start 2023-11-28T13:02:54.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.716+0000 7efec158a700 1 -- start start 2023-11-28T13:02:54.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efec158a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efebc0cd8c0 0x7efebc140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efec158a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efebc0cdc90 0x7efebc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efec158a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efec158a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efebc13ff00 con 0x7efebc0cd8c0 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efec158a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efebc1404c0 con 0x7efebc0cdc90 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efec158a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efebc15aa80 con 0x7efebc0b2e70 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efebaffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efebc0cd8c0 0x7efebc140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efebaffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7efebc0cd8c0 0x7efebc140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42100/0 (socket says 172.21.15.153:42100) 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.717+0000 7efebaffd700 1 -- 172.21.15.153:0/2344834970 learned_addr learned my addr 172.21.15.153:0/2344834970 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efeba7fc700 1 --2- 172.21.15.153:0/2344834970 >> v2:172.21.15.177:3300/0 conn(0x7efebc0cdc90 0x7efebc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efebb7fe700 1 --2- 172.21.15.153:0/2344834970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efebaffd700 1 -- 172.21.15.153:0/2344834970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 msgr2=0x7efebc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efebaffd700 1 --2- 172.21.15.153:0/2344834970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efebaffd700 1 -- 172.21.15.153:0/2344834970 >> v2:172.21.15.177:3300/0 conn(0x7efebc0cdc90 msgr2=0x7efebc0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efebaffd700 1 --2- 172.21.15.153:0/2344834970 >> v2:172.21.15.177:3300/0 conn(0x7efebc0cdc90 0x7efebc0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efebaffd700 1 -- 172.21.15.153:0/2344834970 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc0b3310 con 0x7efebc0cd8c0 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efebaffd700 1 --2- 172.21.15.153:0/2344834970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc140950 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efebc071c40 tx=0x7efeb0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cfc1a63627676 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efeb9ffb700 1 -- 172.21.15.153:0/2344834970 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0005e30 con 0x7efebc0cd8c0 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efeb9ffb700 1 -- 172.21.15.153:0/2344834970 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0004030 con 0x7efebc0cd8c0 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.718+0000 7efeb9ffb700 1 -- 172.21.15.153:0/2344834970 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00043e0 con 0x7efebc0cd8c0 2023-11-28T13:02:54.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 -- 172.21.15.153:0/2344834970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 msgr2=0x7efebc140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 --2- 172.21.15.153:0/2344834970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc140950 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efebc071c40 tx=0x7efeb0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 -- 172.21.15.153:0/2344834970 shutdown_connections 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 --2- 172.21.15.153:0/2344834970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 --2- 172.21.15.153:0/2344834970 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc140950 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 --2- 172.21.15.153:0/2344834970 >> v2:172.21.15.177:3300/0 conn(0x7efebc0cdc90 0x7efebc0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 -- 172.21.15.153:0/2344834970 >> 172.21.15.153:0/2344834970 conn(0x7efebc06aed0 msgr2=0x7efebc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 -- 172.21.15.153:0/2344834970 shutdown_connections 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.719+0000 7efec158a700 1 -- 172.21.15.153:0/2344834970 wait complete. 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.720+0000 7efec158a700 1 Processor -- start 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.720+0000 7efec158a700 1 -- start start 2023-11-28T13:02:54.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efec158a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc0c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efec158a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efec158a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efebc0cdc90 0x7efebc13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efec158a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc1f2f00 con 0x7efebc0cd8c0 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efec158a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc06b9f0 con 0x7efebc0cdc90 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efec158a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efebc1f28b0 con 0x7efebc0b2e70 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc1541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42106/0 (socket says 172.21.15.153:42106) 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 -- 172.21.15.153:0/3724198586 learned_addr learned my addr 172.21.15.153:0/3724198586 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efebaffd700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc0c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efebb7fe700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efebc0cdc90 0x7efebc13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 -- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 msgr2=0x7efebc0c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:54.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc0c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 -- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efebc0cdc90 msgr2=0x7efebc13ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efebc0cdc90 0x7efebc13ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 -- 172.21.15.153:0/3724198586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0009500 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efeba7fc700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc1541f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e7e0 tx=0x7efeac00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efea37fe700 1 -- 172.21.15.153:0/3724198586 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac00c070 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efea37fe700 1 -- 172.21.15.153:0/3724198586 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac013610 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.721+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc140150 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.722+0000 7efea37fe700 1 -- 172.21.15.153:0/3724198586 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac013980 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.722+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc0b35f0 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.722+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc0ce100 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.723+0000 7efea37fe700 1 -- 172.21.15.153:0/3724198586 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efeac023070 con 0x7efebc0cd8c0 2023-11-28T13:02:54.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.723+0000 7efea37fe700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efea80682f0 0x7efea806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:54.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.724+0000 7efebaffd700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efea80682f0 0x7efea806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:54.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.724+0000 7efea37fe700 1 -- 172.21.15.153:0/3724198586 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5962+0+0 (secure 0 0 0) 0x7efeac085990 con 0x7efebc0cd8c0 2023-11-28T13:02:54.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.724+0000 7efebaffd700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efea80682f0 0x7efea806a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efebc071c40 tx=0x7efeb0008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:54.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.728+0000 7efea37fe700 1 -- 172.21.15.153:0/3724198586 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efeac0521d0 con 0x7efebc0cd8c0 2023-11-28T13:02:54.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.952+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efebc0cfb10 con 0x7efebc0cd8c0 2023-11-28T13:02:54.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.957+0000 7efea37fe700 1 -- 172.21.15.153:0/3724198586 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v27) v1 ==== 173+0+0 (secure 0 0 0) 0x7efeac0557f0 con 0x7efebc0cd8c0 2023-11-28T13:02:54.963 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efea80682f0 msgr2=0x7efea806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efea80682f0 0x7efea806a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efebc071c40 tx=0x7efeb0008070 comp rx=0 tx=0).stop 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 msgr2=0x7efebc1541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc1541f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e7e0 tx=0x7efeac00eaf0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 shutdown_connections 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efebc0cdc90 0x7efebc13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efebc0b2e70 0x7efebc0c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7efea80682f0 0x7efea806a7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 --2- 172.21.15.153:0/3724198586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efebc0cd8c0 0x7efebc1541f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:54.964 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.963+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 >> 172.21.15.153:0/3724198586 conn(0x7efebc06aed0 msgr2=0x7efebc0cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:54.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.968+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 shutdown_connections 2023-11-28T13:02:54.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:54.969+0000 7efec158a700 1 -- 172.21.15.153:0/3724198586 wait complete. 2023-11-28T13:02:54.978 INFO:teuthology.orchestra.run.smithi153.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T13:02:54.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:55.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.107+0000 7fb3e3a80700 1 Processor -- start 2023-11-28T13:02:55.108 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.107+0000 7fb3e3a80700 1 -- start start 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.108+0000 7fb3e3a80700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb3dc160740 0x7fb3dc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.108+0000 7fb3e3a80700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3dc06bbb0 con 0x7fb3dc06b060 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.108+0000 7fb3e3a80700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3dc0cc030 con 0x7fb3dc160b10 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.108+0000 7fb3e3a80700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3dc0ca720 con 0x7fb3dc160740 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.108+0000 7fb3e181c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb3dc160740 0x7fb3dc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.108+0000 7fb3e201d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb3dc06b060 0x7fb3dc06b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60292/0 (socket says 172.21.15.153:60292) 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.108+0000 7fb3e201d700 1 -- 172.21.15.153:0/2161612892 learned_addr learned my addr 172.21.15.153:0/2161612892 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e181c700 1 -- 172.21.15.153:0/2161612892 >> v1:172.21.15.177:6789/0 conn(0x7fb3dc160b10 legacy=0x7fb3dc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e181c700 1 -- 172.21.15.153:0/2161612892 >> v1:172.21.15.153:6789/0 conn(0x7fb3dc06b060 legacy=0x7fb3dc06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.111 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e181c700 1 -- 172.21.15.153:0/2161612892 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc0c9e10 con 0x7fb3dc160740 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e181c700 1 --2- 172.21.15.153:0/2161612892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0cbaf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009840 tx=0x7fb3cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842d05753f07e86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e081a700 1 -- 172.21.15.153:0/2161612892 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc004230 con 0x7fb3dc160740 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e081a700 1 -- 172.21.15.153:0/2161612892 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc004390 con 0x7fb3dc160740 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e081a700 1 -- 172.21.15.153:0/2161612892 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0036a0 con 0x7fb3dc160740 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e3a80700 1 -- 172.21.15.153:0/2161612892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 msgr2=0x7fb3dc0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.109+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/2161612892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0cbaf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009840 tx=0x7fb3cc009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.110+0000 7fb3e3a80700 1 -- 172.21.15.153:0/2161612892 shutdown_connections 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.110+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/2161612892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0cbaf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.110+0000 7fb3e3a80700 1 -- 172.21.15.153:0/2161612892 >> 172.21.15.153:0/2161612892 conn(0x7fb3dc0b12b0 msgr2=0x7fb3dc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.110+0000 7fb3e3a80700 1 -- 172.21.15.153:0/2161612892 shutdown_connections 2023-11-28T13:02:55.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.110+0000 7fb3e3a80700 1 -- 172.21.15.153:0/2161612892 wait complete. 2023-11-28T13:02:55.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.111+0000 7fb3e3a80700 1 Processor -- start 2023-11-28T13:02:55.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.111+0000 7fb3e3a80700 1 -- start start 2023-11-28T13:02:55.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e3a80700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3dc06b060 0x7fb3dc15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e3a80700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e3a80700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3dc160b10 0x7fb3dc0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e3a80700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc0b3da0 con 0x7fb3dc160740 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e3a80700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc0b3ac0 con 0x7fb3dc160b10 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e3a80700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc0c3490 con 0x7fb3dc06b060 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42132/0 (socket says 172.21.15.153:42132) 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 -- 172.21.15.153:0/3025673503 learned_addr learned my addr 172.21.15.153:0/3025673503 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e201d700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3dc160b10 0x7fb3dc0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e181c700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3dc06b060 0x7fb3dc15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 -- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3dc06b060 msgr2=0x7fb3dc15bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3dc06b060 0x7fb3dc15bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 -- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3dc160b10 msgr2=0x7fb3dc0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3dc160b10 0x7fb3dc0b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.112+0000 7fb3e101b700 1 -- 172.21.15.153:0/3025673503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0094a0 con 0x7fb3dc160740 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3e201d700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3dc160b10 0x7fb3dc0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3e181c700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3dc06b060 0x7fb3dc15bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3e101b700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0b3000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c4d0 tx=0x7fb3d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3d27fc700 1 -- 172.21.15.153:0/3025673503 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8011790 con 0x7fb3dc160740 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3d27fc700 1 -- 172.21.15.153:0/3025673503 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d80118f0 con 0x7fb3dc160740 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3dc0ca0d0 con 0x7fb3dc160740 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3d27fc700 1 -- 172.21.15.153:0/3025673503 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8011ca0 con 0x7fb3dc160740 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.113+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3dc06efc0 con 0x7fb3dc160740 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.114+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3dc0ca3b0 con 0x7fb3dc160740 2023-11-28T13:02:55.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.114+0000 7fb3d27fc700 1 -- 172.21.15.153:0/3025673503 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb3d8010050 con 0x7fb3dc160740 2023-11-28T13:02:55.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.115+0000 7fb3d27fc700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb3c00682f0 0x7fb3c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.115+0000 7fb3d27fc700 1 -- 172.21.15.153:0/3025673503 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb3d8085a10 con 0x7fb3dc160740 2023-11-28T13:02:55.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.115+0000 7fb3e181c700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb3c00682f0 0x7fb3c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.115+0000 7fb3e181c700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb3c00682f0 0x7fb3c006a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0c8610 tx=0x7fb3cc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.119 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.119+0000 7fb3d27fc700 1 -- 172.21.15.153:0/3025673503 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3d8052270 con 0x7fb3dc160740 2023-11-28T13:02:55.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.302+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3dc066110 con 0x7fb3dc160740 2023-11-28T13:02:55.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.303+0000 7fb3d27fc700 1 -- 172.21.15.153:0/3025673503 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17982 (secure 0 0 0) 0x7fb3d8055890 con 0x7fb3dc160740 2023-11-28T13:02:55.304 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:55.304 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":62,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:02:54.953071+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:52.747638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:53.909181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:55.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb3c00682f0 msgr2=0x7fb3c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb3c00682f0 0x7fb3c006a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0c8610 tx=0x7fb3cc008030 comp rx=0 tx=0).stop 2023-11-28T13:02:55.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 msgr2=0x7fb3dc0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0b3000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800c4d0 tx=0x7fb3d800c890 comp rx=0 tx=0).stop 2023-11-28T13:02:55.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 shutdown_connections 2023-11-28T13:02:55.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3dc160b10 0x7fb3dc0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3dc06b060 0x7fb3dc15bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb3c00682f0 0x7fb3c006a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 --2- 172.21.15.153:0/3025673503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb3dc160740 0x7fb3dc0b3000 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 >> 172.21.15.153:0/3025673503 conn(0x7fb3dc0b12b0 msgr2=0x7fb3dc0c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:55.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 shutdown_connections 2023-11-28T13:02:55.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.305+0000 7fb3e3a80700 1 -- 172.21.15.153:0/3025673503 wait complete. 2023-11-28T13:02:55.318 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:55.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.438+0000 7fcb22eca700 1 Processor -- start 2023-11-28T13:02:55.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.438+0000 7fcb22eca700 1 -- start start 2023-11-28T13:02:55.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb22eca700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb1c140950 0x7fcb1c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb22eca700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcb1c0b2e70 0x7fcb1c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb22eca700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1c13ff00 con 0x7fcb1c0cd8c0 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb22eca700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1c1404c0 con 0x7fcb1c140950 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb22eca700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1c140dc0 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb1bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fcb1c140950 0x7fcb1c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb20c66700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcb1c0cd8c0 0x7fcb1c0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60302/0 (socket says 172.21.15.153:60302) 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb20c66700 1 -- 172.21.15.153:0/3850995265 learned_addr learned my addr 172.21.15.153:0/3850995265 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb1b7fe700 1 -- 172.21.15.153:0/3850995265 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111311566 0 0) 0x7fcb1c13ff00 con 0x7fcb1c0cd8c0 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb1b7fe700 1 -- 172.21.15.153:0/3850995265 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb0c0035b0 con 0x7fcb1c0cd8c0 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb21467700 1 --2- 172.21.15.153:0/3850995265 >> v2:172.21.15.177:3300/0 conn(0x7fcb1c0b2e70 0x7fcb1c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb1bfff700 1 -- 172.21.15.153:0/3850995265 >> v2:172.21.15.177:3300/0 conn(0x7fcb1c0b2e70 msgr2=0x7fcb1c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb1bfff700 1 --2- 172.21.15.153:0/3850995265 >> v2:172.21.15.177:3300/0 conn(0x7fcb1c0b2e70 0x7fcb1c06f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb1bfff700 1 -- 172.21.15.153:0/3850995265 >> v1:172.21.15.153:6789/0 conn(0x7fcb1c0cd8c0 legacy=0x7fcb1c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.439+0000 7fcb1bfff700 1 -- 172.21.15.153:0/3850995265 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c15aa80 con 0x7fcb1c140950 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb1bfff700 1 --2- 172.21.15.153:0/3850995265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c140950 0x7fcb1c0b2930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c144350 tx=0x7fcb08009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0745bb50cc1d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb1b7fe700 1 -- 172.21.15.153:0/3850995265 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08011830 con 0x7fcb1c140950 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb1b7fe700 1 -- 172.21.15.153:0/3850995265 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb08011990 con 0x7fcb1c140950 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb1b7fe700 1 -- 172.21.15.153:0/3850995265 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08011d00 con 0x7fcb1c140950 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 -- 172.21.15.153:0/3850995265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c140950 msgr2=0x7fcb1c0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3850995265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c140950 0x7fcb1c0b2930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c144350 tx=0x7fcb08009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 -- 172.21.15.153:0/3850995265 shutdown_connections 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3850995265 >> v2:172.21.15.177:3300/0 conn(0x7fcb1c0b2e70 0x7fcb1c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3850995265 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c140950 0x7fcb1c0b2930 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 -- 172.21.15.153:0/3850995265 >> 172.21.15.153:0/3850995265 conn(0x7fcb1c06aed0 msgr2=0x7fcb1c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:55.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 -- 172.21.15.153:0/3850995265 shutdown_connections 2023-11-28T13:02:55.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.440+0000 7fcb22eca700 1 -- 172.21.15.153:0/3850995265 wait complete. 2023-11-28T13:02:55.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.441+0000 7fcb22eca700 1 Processor -- start 2023-11-28T13:02:55.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.441+0000 7fcb22eca700 1 -- start start 2023-11-28T13:02:55.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb22eca700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c0b2e70 0x7fcb1c142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb22eca700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb1c0cd8c0 0x7fcb1c1ef810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.446 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb22eca700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb1c140950 0x7fcb1c1f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb22eca700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c1efd50 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb22eca700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c142ba0 con 0x7fcb1c0cd8c0 2023-11-28T13:02:55.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb22eca700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c142e80 con 0x7fcb1c140950 2023-11-28T13:02:55.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb20c66700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c0b2e70 0x7fcb1c142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb20c66700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c0b2e70 0x7fcb1c142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42144/0 (socket says 172.21.15.153:42144) 2023-11-28T13:02:55.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.442+0000 7fcb20c66700 1 -- 172.21.15.153:0/3683055571 learned_addr learned my addr 172.21.15.153:0/3683055571 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:55.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb20c66700 1 -- 172.21.15.153:0/3683055571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb1c140950 msgr2=0x7fcb1c1f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb20c66700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb1c140950 0x7fcb1c1f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb20c66700 1 -- 172.21.15.153:0/3683055571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb1c0cd8c0 msgr2=0x7fcb1c1ef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb20c66700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb1c0cd8c0 0x7fcb1c1ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb20c66700 1 -- 172.21.15.153:0/3683055571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080094a0 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb20c66700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c0b2e70 0x7fcb1c142530 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c700 tx=0x7fcb1000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb197fa700 1 -- 172.21.15.153:0/3683055571 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100074e0 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb197fa700 1 -- 172.21.15.153:0/3683055571 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10007640 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c0b35f0 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb197fa700 1 -- 172.21.15.153:0/3683055571 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10005a70 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.443+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c0b3310 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.444+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c140150 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.444+0000 7fcb197fa700 1 -- 172.21.15.153:0/3683055571 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb10019030 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.445+0000 7fcb197fa700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb000682f0 0x7fcb0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.445+0000 7fcb197fa700 1 -- 172.21.15.153:0/3683055571 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb10085130 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.445+0000 7fcb1bfff700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb000682f0 0x7fcb0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.445+0000 7fcb1bfff700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb000682f0 0x7fcb0006a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c144350 tx=0x7fcb08010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.449+0000 7fcb197fa700 1 -- 172.21.15.153:0/3683055571 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcb10051910 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.633 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.633+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb1c0c7dd0 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.634+0000 7fcb197fa700 1 -- 172.21.15.153:0/3683055571 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17982 (secure 0 0 0) 0x7fcb10054f30 con 0x7fcb1c0b2e70 2023-11-28T13:02:55.635 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:55.635 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":62,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:02:54.953071+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:52.747638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:53.909181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:55.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb000682f0 msgr2=0x7fcb0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb000682f0 0x7fcb0006a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c144350 tx=0x7fcb08010000 comp rx=0 tx=0).stop 2023-11-28T13:02:55.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c0b2e70 msgr2=0x7fcb1c142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c0b2e70 0x7fcb1c142530 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c700 tx=0x7fcb1000cac0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 shutdown_connections 2023-11-28T13:02:55.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcb000682f0 0x7fcb0006a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb1c0b2e70 0x7fcb1c142530 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb1c140950 0x7fcb1c1f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 --2- 172.21.15.153:0/3683055571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb1c0cd8c0 0x7fcb1c1ef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 >> 172.21.15.153:0/3683055571 conn(0x7fcb1c06aed0 msgr2=0x7fcb1c0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:55.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 shutdown_connections 2023-11-28T13:02:55.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.636+0000 7fcb22eca700 1 -- 172.21.15.153:0/3683055571 wait complete. 2023-11-28T13:02:55.649 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:02:55.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.772+0000 7f0f55b7b700 1 Processor -- start 2023-11-28T13:02:55.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.772+0000 7f0f55b7b700 1 -- start start 2023-11-28T13:02:55.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.773+0000 7f0f55b7b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0f50160b10 0x7f0f500cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.773+0000 7f0f55b7b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f5006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.773+0000 7f0f55b7b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f500c9e10 con 0x7f0f50160740 2023-11-28T13:02:55.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.773+0000 7f0f55b7b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f500ca720 con 0x7f0f50160b10 2023-11-28T13:02:55.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.773+0000 7f0f55b7b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5006efc0 con 0x7f0f5006b060 2023-11-28T13:02:55.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4effd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0f50160b10 0x7f0f500cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f5006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4f7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0f50160740 0x7f0f500cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:36930/0 (socket says 172.21.15.153:36930) 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4effd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0f50160b10 0x7f0f500cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:51620/0 (socket says 172.21.15.153:51620) 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4f7fe700 1 -- 172.21.15.153:0/1692100324 learned_addr learned my addr 172.21.15.153:0/1692100324 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4effd700 1 -- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 msgr2=0x7f0f5006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4effd700 1 --2- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f5006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4effd700 1 -- 172.21.15.153:0/1692100324 >> v1:172.21.15.177:6789/0 conn(0x7f0f50160740 legacy=0x7f0f500cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4effd700 1 -- 172.21.15.153:0/1692100324 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f500ca0d0 con 0x7f0f50160b10 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4e7fc700 1 -- 172.21.15.153:0/1692100324 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046089994 0 0) 0x7f0f500c9e10 con 0x7f0f50160740 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4ffff700 1 --2- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f5006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4effd700 1 --2- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160b10 0x7f0f500cd390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009840 tx=0x7f0f40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cc74480e9cd6ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4e7fc700 1 -- 172.21.15.153:0/1692100324 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40011830 con 0x7f0f50160b10 2023-11-28T13:02:55.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4e7fc700 1 -- 172.21.15.153:0/1692100324 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40011990 con 0x7f0f50160b10 2023-11-28T13:02:55.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.774+0000 7f0f4e7fc700 1 -- 172.21.15.153:0/1692100324 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40011d20 con 0x7f0f50160b10 2023-11-28T13:02:55.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 -- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160b10 msgr2=0x7f0f500cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160b10 0x7f0f500cd390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009840 tx=0x7f0f40009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 -- 172.21.15.153:0/1692100324 shutdown_connections 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f5006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/1692100324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160b10 0x7f0f500cd390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 -- 172.21.15.153:0/1692100324 >> 172.21.15.153:0/1692100324 conn(0x7f0f500b12b0 msgr2=0x7f0f500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 -- 172.21.15.153:0/1692100324 shutdown_connections 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.775+0000 7f0f55b7b700 1 -- 172.21.15.153:0/1692100324 wait complete. 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.776+0000 7f0f55b7b700 1 Processor -- start 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.776+0000 7f0f55b7b700 1 -- start start 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f55b7b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f500c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f55b7b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160740 0x7f0f501599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f55b7b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f50160b10 0x7f0f5006d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f55b7b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500c8e20 con 0x7f0f50160b10 2023-11-28T13:02:55.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f55b7b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500c81a0 con 0x7f0f50160740 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f55b7b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500c8480 con 0x7f0f5006b060 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f4ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f50160b10 0x7f0f5006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f4ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f50160b10 0x7f0f5006d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42162/0 (socket says 172.21.15.153:42162) 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f4ffff700 1 -- 172.21.15.153:0/4179619289 learned_addr learned my addr 172.21.15.153:0/4179619289 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f4effd700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160740 0x7f0f501599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.777+0000 7f0f4f7fe700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f500c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f4ffff700 1 -- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 msgr2=0x7f0f500c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f4ffff700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f500c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f4ffff700 1 -- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160740 msgr2=0x7f0f501599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f4ffff700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160740 0x7f0f501599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:55.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f4ffff700 1 -- 172.21.15.153:0/4179619289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48009a70 con 0x7f0f50160b10 2023-11-28T13:02:55.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f4ffff700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f50160b10 0x7f0f5006d800 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800efc0 tx=0x7f0f4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f37fff700 1 -- 172.21.15.153:0/4179619289 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800ca90 con 0x7f0f50160b10 2023-11-28T13:02:55.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f37fff700 1 -- 172.21.15.153:0/4179619289 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4800f040 con 0x7f0f50160b10 2023-11-28T13:02:55.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f37fff700 1 -- 172.21.15.153:0/4179619289 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800ce10 con 0x7f0f50160b10 2023-11-28T13:02:55.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f400094a0 con 0x7f0f50160b10 2023-11-28T13:02:55.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.778+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f500cd8d0 con 0x7f0f50160b10 2023-11-28T13:02:55.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.779+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f500ca3b0 con 0x7f0f50160b10 2023-11-28T13:02:55.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.779+0000 7f0f37fff700 1 -- 172.21.15.153:0/4179619289 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0f48014480 con 0x7f0f50160b10 2023-11-28T13:02:55.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.780+0000 7f0f37fff700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0f3c0682f0 0x7f0f3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:55.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.780+0000 7f0f37fff700 1 -- 172.21.15.153:0/4179619289 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f48086400 con 0x7f0f50160b10 2023-11-28T13:02:55.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.780+0000 7f0f4f7fe700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0f3c0682f0 0x7f0f3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:55.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.780+0000 7f0f4f7fe700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0f3c0682f0 0x7f0f3c06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f44007fd0 tx=0x7f0f44007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:55.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:55.784+0000 7f0f37fff700 1 -- 172.21.15.153:0/4179619289 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0f48052c00 con 0x7f0f50160b10 2023-11-28T13:02:56.014 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.013+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0f50160f60 con 0x7f0f50160b10 2023-11-28T13:02:56.972 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.972+0000 7f0f37fff700 1 -- 172.21.15.153:0/4179619289 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0f48009d70 con 0x7f0f50160b10 2023-11-28T13:02:56.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.974+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0f3c0682f0 msgr2=0x7f0f3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.979 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.974+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0f3c0682f0 0x7f0f3c06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f44007fd0 tx=0x7f0f44007de0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f50160b10 msgr2=0x7f0f5006d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f50160b10 0x7f0f5006d800 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800efc0 tx=0x7f0f4800c3a0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 shutdown_connections 2023-11-28T13:02:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0f3c0682f0 0x7f0f3c06a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f50160b10 0x7f0f5006d800 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0f5006b060 0x7f0f500c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 --2- 172.21.15.153:0/4179619289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f50160740 0x7f0f501599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 >> 172.21.15.153:0/4179619289 conn(0x7f0f500b12b0 msgr2=0x7f0f5015d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:56.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 shutdown_connections 2023-11-28T13:02:56.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:56.975+0000 7f0f55b7b700 1 -- 172.21.15.153:0/4179619289 wait complete. 2023-11-28T13:02:56.990 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:02:56.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:02:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.118+0000 7f00d10ea700 1 Processor -- start 2023-11-28T13:02:57.119 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.119+0000 7f00d10ea700 1 -- start start 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.119+0000 7f00d10ea700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f00cc160b10 0x7f00cc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.119+0000 7f00d10ea700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f00cc06b060 0x7f00cc06b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.119+0000 7f00d10ea700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc0c9e10 con 0x7f00cc160740 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.119+0000 7f00d10ea700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00cc0ca720 con 0x7f00cc06b060 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.119+0000 7f00d10ea700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00cc06efc0 con 0x7f00cc160b10 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.119+0000 7f00cb59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f00cc06b060 0x7f00cc06b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00cb59e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f00cc06b060 0x7f00cc06b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42170/0 (socket says 172.21.15.153:42170) 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00cb59e700 1 -- 172.21.15.153:0/3162964332 learned_addr learned my addr 172.21.15.153:0/3162964332 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:57.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00ca59c700 1 --2- 172.21.15.153:0/3162964332 >> v2:172.21.15.177:3300/0 conn(0x7f00cc160b10 0x7f00cc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00cb59e700 1 -- 172.21.15.153:0/3162964332 >> v2:172.21.15.177:3300/0 conn(0x7f00cc160b10 msgr2=0x7f00cc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00cb59e700 1 --2- 172.21.15.153:0/3162964332 >> v2:172.21.15.177:3300/0 conn(0x7f00cc160b10 0x7f00cc0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00cb59e700 1 -- 172.21.15.153:0/3162964332 >> v1:172.21.15.177:6789/0 conn(0x7f00cc160740 legacy=0x7f00cc0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00cb59e700 1 -- 172.21.15.153:0/3162964332 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0ca0d0 con 0x7f00cc06b060 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00cb59e700 1 --2- 172.21.15.153:0/3162964332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc06b4c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0aeb10 tx=0x7f00c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a17862dd477122f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00c9d9b700 1 -- 172.21.15.153:0/3162964332 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c400e070 con 0x7f00cc06b060 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00c9d9b700 1 -- 172.21.15.153:0/3162964332 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c400b9c0 con 0x7f00cc06b060 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.120+0000 7f00c9d9b700 1 -- 172.21.15.153:0/3162964332 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c400bd70 con 0x7f00cc06b060 2023-11-28T13:02:57.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 -- 172.21.15.153:0/3162964332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 msgr2=0x7f00cc06b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 --2- 172.21.15.153:0/3162964332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc06b4c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0aeb10 tx=0x7f00c400b330 comp rx=0 tx=0).stop 2023-11-28T13:02:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 -- 172.21.15.153:0/3162964332 shutdown_connections 2023-11-28T13:02:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 --2- 172.21.15.153:0/3162964332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc06b4c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 --2- 172.21.15.153:0/3162964332 >> v2:172.21.15.177:3300/0 conn(0x7f00cc160b10 0x7f00cc0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 -- 172.21.15.153:0/3162964332 >> 172.21.15.153:0/3162964332 conn(0x7f00cc0b12b0 msgr2=0x7f00cc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 -- 172.21.15.153:0/3162964332 shutdown_connections 2023-11-28T13:02:57.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.121+0000 7f00d10ea700 1 -- 172.21.15.153:0/3162964332 wait complete. 2023-11-28T13:02:57.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.122+0000 7f00d10ea700 1 Processor -- start 2023-11-28T13:02:57.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.122+0000 7f00d10ea700 1 -- start start 2023-11-28T13:02:57.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00d10ea700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00d10ea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00cc160740 0x7f00cc06da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00d10ea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00cc160b10 0x7f00cc1585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00d10ea700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0c7ed0 con 0x7f00cc06b060 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00d10ea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0c81b0 con 0x7f00cc160b10 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00d10ea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0c8490 con 0x7f00cc160740 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42172/0 (socket says 172.21.15.153:42172) 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 -- 172.21.15.153:0/844775481 learned_addr learned my addr 172.21.15.153:0/844775481 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cb59e700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00cc160b10 0x7f00cc1585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00ca59c700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00cc160740 0x7f00cc06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 -- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00cc160740 msgr2=0x7f00cc06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00cc160740 0x7f00cc06da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 -- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00cc160b10 msgr2=0x7f00cc1585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00cc160b10 0x7f00cc1585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 -- 172.21.15.153:0/844775481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c4009b30 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00ca59c700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00cc160740 0x7f00cc06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.123+0000 7f00cad9d700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc159fd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c4e0 tx=0x7f00c000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.124+0000 7f00b37fe700 1 -- 172.21.15.153:0/844775481 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00052d0 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.124+0000 7f00b37fe700 1 -- 172.21.15.153:0/844775481 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c0005430 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.124+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc0cd8d0 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.124+0000 7f00b37fe700 1 -- 172.21.15.153:0/844775481 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00038c0 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.124+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc0ca3b0 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.125+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc160f60 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.125+0000 7f00b37fe700 1 -- 172.21.15.153:0/844775481 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f00c0019030 con 0x7f00cc06b060 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.125+0000 7f00b37fe700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00b80682f0 0x7f00b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.125+0000 7f00b37fe700 1 -- 172.21.15.153:0/844775481 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00c0084f40 con 0x7f00cc06b060 2023-11-28T13:02:57.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.126+0000 7f00ca59c700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00b80682f0 0x7f00b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.126+0000 7f00ca59c700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00b80682f0 0x7f00b806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f00cc15c1f0 tx=0x7f00bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.130+0000 7f00b37fe700 1 -- 172.21.15.153:0/844775481 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00c00517a0 con 0x7f00cc06b060 2023-11-28T13:02:57.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.358+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f00cc066110 con 0x7f00cc06b060 2023-11-28T13:02:57.974 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.973+0000 7f00b37fe700 1 -- 172.21.15.153:0/844775481 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f00c00170c0 con 0x7f00cc06b060 2023-11-28T13:02:57.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00b80682f0 msgr2=0x7f00b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00b80682f0 0x7f00b806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f00cc15c1f0 tx=0x7f00bc008040 comp rx=0 tx=0).stop 2023-11-28T13:02:57.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 msgr2=0x7f00cc159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc159fd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c4e0 tx=0x7f00c000c7f0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.977 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 shutdown_connections 2023-11-28T13:02:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f00b80682f0 0x7f00b806a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00cc06b060 0x7f00cc159fd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00cc160b10 0x7f00cc1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 --2- 172.21.15.153:0/844775481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00cc160740 0x7f00cc06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 >> 172.21.15.153:0/844775481 conn(0x7f00cc0b12b0 msgr2=0x7f00cc0ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 shutdown_connections 2023-11-28T13:02:57.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:57.976+0000 7f00d10ea700 1 -- 172.21.15.153:0/844775481 wait complete. 2023-11-28T13:02:57.991 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:02:57.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:58.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.116+0000 7f471bfa9700 1 Processor -- start 2023-11-28T13:02:58.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.116+0000 7f471bfa9700 1 -- start start 2023-11-28T13:02:58.120 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f471bfa9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 0x7f47140af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f471bfa9700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140cc5f0 con 0x7f47140cbdf0 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f471bfa9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471415d470 con 0x7f471406ba90 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f471bfa9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471415d750 con 0x7f471415cef0 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4719544700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f47140cbdf0 0x7f471406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60346/0 (socket says 172.21.15.153:60346) 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4719544700 1 -- 172.21.15.153:0/1008370280 learned_addr learned my addr 172.21.15.153:0/1008370280 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865713338 0 0) 0x7f47140cc5f0 con 0x7f47140cbdf0 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47000035b0 con 0x7f47140cbdf0 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4719d45700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 0x7f47140af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 388023299 0 0) 0x7f47000035b0 con 0x7f47140cbdf0 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 msgr2=0x7f47140af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 0x7f47140af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 >> v1:172.21.15.177:6789/0 conn(0x7f471406ba90 legacy=0x7f47140ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471415da30 con 0x7f47140cbdf0 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f4704001670 con 0x7f47140cbdf0 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 >> v1:172.21.15.153:6789/0 conn(0x7f47140cbdf0 legacy=0x7f471406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.117+0000 7f4718d43700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4700003a30 0x7f4700003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4700004850 0x7f4700004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 0x7f4700005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471415d750 con 0x7f4700004850 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470000bac0 con 0x7f4700003a30 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470000bc50 con 0x7f471415cef0 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719544700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4700004850 0x7f4700004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719d45700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4700003a30 0x7f4700003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f471a546700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 0x7f4700005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719544700 1 -- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 msgr2=0x7f4700005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719544700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 0x7f4700005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719544700 1 -- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4700003a30 msgr2=0x7f4700003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719544700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4700003a30 0x7f4700003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719544700 1 -- 172.21.15.153:0/1008370280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000035b0 con 0x7f4700004850 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719d45700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4700003a30 0x7f4700003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f471a546700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f471415cef0 0x7f4700005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4719544700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4700004850 0x7f4700004cc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f470400c240 tx=0x7f4700005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ad64e854c59fc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704007c10 con 0x7f4700004850 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470400cde0 con 0x7f4700004850 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.118+0000 7f4718d43700 1 -- 172.21.15.153:0/1008370280 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704005ba0 con 0x7f4700004850 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.119+0000 7f471bfa9700 1 -- 172.21.15.153:0/1008370280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4700004850 msgr2=0x7f4700004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.119+0000 7f471bfa9700 1 --2- 172.21.15.153:0/1008370280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4700004850 0x7f4700004cc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f470400c240 tx=0x7f4700005880 comp rx=0 tx=0).stop 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.119+0000 7f471a546700 1 -- 172.21.15.153:0/1008370280 reap_dead start 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.119+0000 7f471bfa9700 1 -- 172.21.15.153:0/1008370280 shutdown_connections 2023-11-28T13:02:58.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.119+0000 7f471bfa9700 1 -- 172.21.15.153:0/1008370280 >> 172.21.15.153:0/1008370280 conn(0x7f47140d0000 msgr2=0x7f47140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.119+0000 7f471bfa9700 1 -- 172.21.15.153:0/1008370280 shutdown_connections 2023-11-28T13:02:58.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.119+0000 7f471bfa9700 1 -- 172.21.15.153:0/1008370280 wait complete. 2023-11-28T13:02:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.120+0000 7f471bfa9700 1 Processor -- start 2023-11-28T13:02:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.120+0000 7f471bfa9700 1 -- start start 2023-11-28T13:02:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4700004850 0x7f47141edf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.127 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47141592e0 0x7f4714159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4714159e50 0x7f471415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471415a800 con 0x7f4714159e50 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47141ee4b0 con 0x7f47141592e0 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47140c3e60 con 0x7f4700004850 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4714159e50 0x7f471415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4714159e50 0x7f471415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42196/0 (socket says 172.21.15.153:42196) 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 -- 172.21.15.153:0/3721205547 learned_addr learned my addr 172.21.15.153:0/3721205547 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f4719544700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47141592e0 0x7f4714159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f4719d45700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4700004850 0x7f47141edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 -- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4700004850 msgr2=0x7f47141edf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4700004850 0x7f47141edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 -- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47141592e0 msgr2=0x7f4714159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47141592e0 0x7f4714159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 -- 172.21.15.153:0/3721205547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471415dff0 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471a546700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4714159e50 0x7f471415a2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f471406fad0 tx=0x7f470c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f470a7fc700 1 -- 172.21.15.153:0/3721205547 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c003c10 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f470a7fc700 1 -- 172.21.15.153:0/3721205547 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c003d90 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471415dd10 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f470a7fc700 1 -- 172.21.15.153:0/3721205547 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c005310 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.121+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47140af400 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.122+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47140c4250 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.123+0000 7f470a7fc700 1 -- 172.21.15.153:0/3721205547 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f470c005470 con 0x7f4714159e50 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.123+0000 7f470a7fc700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f8068280 0x7f46f806a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.123+0000 7f470a7fc700 1 -- 172.21.15.153:0/3721205547 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f470c085dc0 con 0x7f4714159e50 2023-11-28T13:02:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.123+0000 7f4719d45700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f8068280 0x7f46f806a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.123+0000 7f4719d45700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f8068280 0x7f46f806a730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4710009a50 tx=0x7f4710008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.127+0000 7f470a7fc700 1 -- 172.21.15.153:0/3721205547 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f470c052620 con 0x7f4714159e50 2023-11-28T13:02:58.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.354+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47140d0e60 con 0x7f4714159e50 2023-11-28T13:02:58.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.355+0000 7f470a7fc700 1 -- 172.21.15.153:0/3721205547 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f470c001b50 con 0x7f4714159e50 2023-11-28T13:02:58.355 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:02:57.970702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:02:58.357 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f8068280 msgr2=0x7f46f806a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f8068280 0x7f46f806a730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4710009a50 tx=0x7f4710008040 comp rx=0 tx=0).stop 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4714159e50 msgr2=0x7f471415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4714159e50 0x7f471415a2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f471406fad0 tx=0x7f470c00ee70 comp rx=0 tx=0).stop 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 shutdown_connections 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46f8068280 0x7f46f806a730 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4714159e50 0x7f471415a2c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4700004850 0x7f47141edf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 --2- 172.21.15.153:0/3721205547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47141592e0 0x7f4714159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 >> 172.21.15.153:0/3721205547 conn(0x7f47140d0000 msgr2=0x7f47140cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.358 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 shutdown_connections 2023-11-28T13:02:58.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.357+0000 7f471bfa9700 1 -- 172.21.15.153:0/3721205547 wait complete. 2023-11-28T13:02:58.359 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 31 2023-11-28T13:02:58.370 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.489+0000 7fb2dd1e7700 1 Processor -- start 2023-11-28T13:02:58.490 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.489+0000 7fb2dd1e7700 1 -- start start 2023-11-28T13:02:58.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.489+0000 7fb2dd1e7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb2d80cbdf0 0x7fb2d80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.489+0000 7fb2dd1e7700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d80cc5f0 con 0x7fb2d815cef0 2023-11-28T13:02:58.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.489+0000 7fb2dd1e7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d815d470 con 0x7fb2d806ba90 2023-11-28T13:02:58.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.489+0000 7fb2dd1e7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d815d750 con 0x7fb2d80cbdf0 2023-11-28T13:02:58.492 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d6d9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb2d815cef0 0x7fb2d806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60364/0 (socket says 172.21.15.153:60364) 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d6d9d700 1 -- 172.21.15.153:0/961373266 learned_addr learned my addr 172.21.15.153:0/961373266 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d759e700 1 --2- 172.21.15.153:0/961373266 >> v2:172.21.15.177:3300/0 conn(0x7fb2d80cbdf0 0x7fb2d80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d5d9b700 1 -- 172.21.15.153:0/961373266 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500027059 0 0) 0x7fb2d80cc5f0 con 0x7fb2d815cef0 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d5d9b700 1 -- 172.21.15.153:0/961373266 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c00035b0 con 0x7fb2d815cef0 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d5d9b700 1 -- 172.21.15.153:0/961373266 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163578297 0 0) 0x7fb2d815d470 con 0x7fb2d806ba90 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d5d9b700 1 -- 172.21.15.153:0/961373266 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d80cc5f0 con 0x7fb2d806ba90 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d759e700 1 -- 172.21.15.153:0/961373266 >> v1:172.21.15.177:6789/0 conn(0x7fb2d806ba90 legacy=0x7fb2d80ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d759e700 1 -- 172.21.15.153:0/961373266 >> v1:172.21.15.153:6789/0 conn(0x7fb2d815cef0 legacy=0x7fb2d806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d759e700 1 -- 172.21.15.153:0/961373266 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d815da30 con 0x7fb2d80cbdf0 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d759e700 1 --2- 172.21.15.153:0/961373266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d80cbdf0 0x7fb2d80af400 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009d90 tx=0x7fb2d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46b3c6436e6d2922 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.493 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d5d9b700 1 -- 172.21.15.153:0/961373266 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000e070 con 0x7fb2d80cbdf0 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d5d9b700 1 -- 172.21.15.153:0/961373266 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d000bb20 con 0x7fb2d80cbdf0 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.490+0000 7fb2d5d9b700 1 -- 172.21.15.153:0/961373266 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000be90 con 0x7fb2d80cbdf0 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.491+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/961373266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d80cbdf0 msgr2=0x7fb2d80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.491+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/961373266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d80cbdf0 0x7fb2d80af400 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009d90 tx=0x7fb2d000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.491+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/961373266 shutdown_connections 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.491+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/961373266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d80cbdf0 0x7fb2d80af400 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.491+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/961373266 >> 172.21.15.153:0/961373266 conn(0x7fb2d80d0000 msgr2=0x7fb2d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.491+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/961373266 shutdown_connections 2023-11-28T13:02:58.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.491+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/961373266 wait complete. 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 Processor -- start 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 -- start start 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d806ba90 0x7fb2d80c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d80cbdf0 0x7fb2d81534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d815cef0 0x7fb2d80b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c5f90 con 0x7fb2d806ba90 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8153a10 con 0x7fb2d815cef0 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2dd1e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80b5c70 con 0x7fb2d80cbdf0 2023-11-28T13:02:58.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2d6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d806ba90 0x7fb2d80c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2d6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d806ba90 0x7fb2d80c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42206/0 (socket says 172.21.15.153:42206) 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.493+0000 7fb2d6d9d700 1 -- 172.21.15.153:0/4068216085 learned_addr learned my addr 172.21.15.153:0/4068216085 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d759e700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d815cef0 0x7fb2d80b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d659c700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d80cbdf0 0x7fb2d81534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d6d9d700 1 -- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d80cbdf0 msgr2=0x7fb2d81534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d6d9d700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d80cbdf0 0x7fb2d81534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d6d9d700 1 -- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d815cef0 msgr2=0x7fb2d80b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d6d9d700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d815cef0 0x7fb2d80b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d6d9d700 1 -- 172.21.15.153:0/4068216085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0009a70 con 0x7fb2d806ba90 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d759e700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d815cef0 0x7fb2d80b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2d6d9d700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d806ba90 0x7fb2d80c4d00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8001eb0 tx=0x7fb2c800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2bf7fe700 1 -- 172.21.15.153:0/4068216085 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8005d70 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2bf7fe700 1 -- 172.21.15.153:0/4068216085 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c80070d0 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d815dff0 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2bf7fe700 1 -- 172.21.15.153:0/4068216085 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8007460 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.494+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d815dd10 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.495+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d80cad60 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.495+0000 7fb2bf7fe700 1 -- 172.21.15.153:0/4068216085 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb2c8019030 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.496+0000 7fb2bf7fe700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb2c40682f0 0x7fb2c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.496+0000 7fb2bf7fe700 1 -- 172.21.15.153:0/4068216085 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2c8085050 con 0x7fb2d806ba90 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.496+0000 7fb2d659c700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb2c40682f0 0x7fb2c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.496+0000 7fb2d659c700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb2c40682f0 0x7fb2c406a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80c5240 tx=0x7fb2cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.501 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.500+0000 7fb2bf7fe700 1 -- 172.21.15.153:0/4068216085 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb2c80518b0 con 0x7fb2d806ba90 2023-11-28T13:02:58.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.685+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2d806b7b0 con 0x7fb2d806ba90 2023-11-28T13:02:58.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.686+0000 7fb2bf7fe700 1 -- 172.21.15.153:0/4068216085 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17969 (secure 0 0 0) 0x7fb2c8054ed0 con 0x7fb2d806ba90 2023-11-28T13:02:58.687 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:58.687 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":63,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:02:55.955303+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:52.747638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:53.909181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb2c40682f0 msgr2=0x7fb2c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb2c40682f0 0x7fb2c406a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80c5240 tx=0x7fb2cc009040 comp rx=0 tx=0).stop 2023-11-28T13:02:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d806ba90 msgr2=0x7fb2d80c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d806ba90 0x7fb2d80c4d00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8001eb0 tx=0x7fb2c800c860 comp rx=0 tx=0).stop 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 shutdown_connections 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb2c40682f0 0x7fb2c406a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d806ba90 0x7fb2d80c4d00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d815cef0 0x7fb2d80b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 --2- 172.21.15.153:0/4068216085 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d80cbdf0 0x7fb2d81534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 >> 172.21.15.153:0/4068216085 conn(0x7fb2d80d0000 msgr2=0x7fb2d80af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 shutdown_connections 2023-11-28T13:02:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.688+0000 7fb2dd1e7700 1 -- 172.21.15.153:0/4068216085 wait complete. 2023-11-28T13:02:58.701 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:02:58.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:02:58.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.820+0000 7f5dc8b44700 1 Processor -- start 2023-11-28T13:02:58.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.820+0000 7f5dc8b44700 1 -- start start 2023-11-28T13:02:58.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.821+0000 7f5dc8b44700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5dc40cd8c0 0x7f5dc4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.821+0000 7f5dc8b44700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5dc406f850 0x7f5dc406fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.821+0000 7f5dc8b44700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc413ff00 con 0x7f5dc40cdc90 2023-11-28T13:02:58.824 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.821+0000 7f5dc8b44700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc41404c0 con 0x7f5dc406f850 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.821+0000 7f5dc8b44700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc415aa80 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc2d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5dc406f850 0x7f5dc406fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc1d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5dc40cdc90 0x7f5dc40b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60378/0 (socket says 172.21.15.153:60378) 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc1d9b700 1 -- 172.21.15.153:0/1895187093 learned_addr learned my addr 172.21.15.153:0/1895187093 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc259c700 1 --2- 172.21.15.153:0/1895187093 >> v2:172.21.15.177:3300/0 conn(0x7f5dc40cd8c0 0x7f5dc4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc2d9d700 1 -- 172.21.15.153:0/1895187093 >> v2:172.21.15.177:3300/0 conn(0x7f5dc40cd8c0 msgr2=0x7f5dc4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc2d9d700 1 --2- 172.21.15.153:0/1895187093 >> v2:172.21.15.177:3300/0 conn(0x7f5dc40cd8c0 0x7f5dc4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc2d9d700 1 -- 172.21.15.153:0/1895187093 >> v1:172.21.15.153:6789/0 conn(0x7f5dc40cdc90 legacy=0x7f5dc40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc2d9d700 1 -- 172.21.15.153:0/1895187093 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40b3040 con 0x7f5dc406f850 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc159a700 1 -- 172.21.15.153:0/1895187093 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441914916 0 0) 0x7f5dc413ff00 con 0x7f5dc40cdc90 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc259c700 1 --2- 172.21.15.153:0/1895187093 >> v2:172.21.15.177:3300/0 conn(0x7f5dc40cd8c0 0x7f5dc4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:58.825 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc2d9d700 1 --2- 172.21.15.153:0/1895187093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc406f850 0x7f5dc406fcf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4070a20 tx=0x7f5dbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1c2f6a226cd6d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc159a700 1 -- 172.21.15.153:0/1895187093 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00e070 con 0x7f5dc406f850 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc159a700 1 -- 172.21.15.153:0/1895187093 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00bbd0 con 0x7f5dc406f850 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.822+0000 7f5dc159a700 1 -- 172.21.15.153:0/1895187093 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc012430 con 0x7f5dc406f850 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 -- 172.21.15.153:0/1895187093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc406f850 msgr2=0x7f5dc406fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/1895187093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc406f850 0x7f5dc406fcf0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4070a20 tx=0x7f5dbc00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 -- 172.21.15.153:0/1895187093 shutdown_connections 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/1895187093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc406f850 0x7f5dc406fcf0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/1895187093 >> v2:172.21.15.177:3300/0 conn(0x7f5dc40cd8c0 0x7f5dc4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 -- 172.21.15.153:0/1895187093 >> 172.21.15.153:0/1895187093 conn(0x7f5dc406aed0 msgr2=0x7f5dc406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 -- 172.21.15.153:0/1895187093 shutdown_connections 2023-11-28T13:02:58.826 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.823+0000 7f5dc8b44700 1 -- 172.21.15.153:0/1895187093 wait complete. 2023-11-28T13:02:58.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.824+0000 7f5dc8b44700 1 Processor -- start 2023-11-28T13:02:58.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.824+0000 7f5dc8b44700 1 -- start start 2023-11-28T13:02:58.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc8b44700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5dc406f850 0x7f5dc41ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc8b44700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc40cd8c0 0x7f5dc40b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.830 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc8b44700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5dc40cdc90 0x7f5dc4146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc8b44700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41efae0 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc8b44700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b1aa0 con 0x7f5dc406f850 2023-11-28T13:02:58.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc8b44700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b2340 con 0x7f5dc40cdc90 2023-11-28T13:02:58.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc40cd8c0 0x7f5dc40b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc40cd8c0 0x7f5dc40b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42222/0 (socket says 172.21.15.153:42222) 2023-11-28T13:02:58.831 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 -- 172.21.15.153:0/3633657375 learned_addr learned my addr 172.21.15.153:0/3633657375 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc259c700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5dc406f850 0x7f5dc41ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc2d9d700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5dc40cdc90 0x7f5dc4146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 -- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5dc40cdc90 msgr2=0x7f5dc4146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5dc40cdc90 0x7f5dc4146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 -- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5dc406f850 msgr2=0x7f5dc41ef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5dc406f850 0x7f5dc41ef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc1d9b700 1 -- 172.21.15.153:0/3633657375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009a70 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.825+0000 7f5dc259c700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5dc406f850 0x7f5dc41ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.826+0000 7f5dc1d9b700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc40cd8c0 0x7f5dc40b3d00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5db400c550 tx=0x7f5db400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.826+0000 7f5db2ffd700 1 -- 172.21.15.153:0/3633657375 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4004c80 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.832 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.826+0000 7f5db2ffd700 1 -- 172.21.15.153:0/3633657375 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4004de0 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.826+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc40b35f0 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.826+0000 7f5db2ffd700 1 -- 172.21.15.153:0/3633657375 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40072d0 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.826+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc40b3310 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.827+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc4140150 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.827+0000 7f5db2ffd700 1 -- 172.21.15.153:0/3633657375 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5db401a020 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.827+0000 7f5db2ffd700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5da40682f0 0x7f5da406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.828+0000 7f5db2ffd700 1 -- 172.21.15.153:0/3633657375 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5db40857f0 con 0x7f5dc40cd8c0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.828+0000 7f5dc259c700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5da40682f0 0x7f5da406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.833 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.828+0000 7f5dc259c700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5da40682f0 0x7f5da406a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5db800aaf0 tx=0x7f5db8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:58.832+0000 7f5db2ffd700 1 -- 172.21.15.153:0/3633657375 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5db4052100 con 0x7f5dc40cd8c0 2023-11-28T13:02:59.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.012+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5dc4068690 con 0x7f5da40682f0 2023-11-28T13:02:59.020 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.020+0000 7f5db2ffd700 1 -- 172.21.15.153:0/3633657375 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300538 (secure 0 0 0) 0x7f5dc4068690 con 0x7f5da40682f0 2023-11-28T13:02:59.024 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:59.026 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5da40682f0 msgr2=0x7f5da406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5da40682f0 0x7f5da406a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5db800aaf0 tx=0x7f5db8008040 comp rx=0 tx=0).stop 2023-11-28T13:02:59.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc40cd8c0 msgr2=0x7f5dc40b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc40cd8c0 0x7f5dc40b3d00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5db400c550 tx=0x7f5db400c910 comp rx=0 tx=0).stop 2023-11-28T13:02:59.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 shutdown_connections 2023-11-28T13:02:59.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5dc40cdc90 0x7f5dc4146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5dc406f850 0x7f5dc41ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f5da40682f0 0x7f5da406a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 --2- 172.21.15.153:0/3633657375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5dc40cd8c0 0x7f5dc40b3d00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 >> 172.21.15.153:0/3633657375 conn(0x7f5dc406aed0 msgr2=0x7f5dc413efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 shutdown_connections 2023-11-28T13:02:59.028 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.023+0000 7f5dc8b44700 1 -- 172.21.15.153:0/3633657375 wait complete. 2023-11-28T13:02:59.028 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:02:59.036 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2023-11-28T13:02:58.182211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":166,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":81208,"kb_used_data":6712,"kb_used_omap":0,"kb_used_meta":74432,"kb_avail":754893512,"statfs":{"total":773094113280,"available":773010956288,"internally_reserved":0,"allocated":6873088,"data_stored":3637600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76218368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.635722"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654061+0000","last_change":"2023-11-28T13:02:54.984256+0000","last_active":"2023-11-28T13:02:56.654061+0000","last_peered":"2023-11-28T13:02:56.654061+0000","last_clean":"2023-11-28T13:02:56.654061+0000","last_became_active":"2023-11-28T13:02:54.984012+0000","last_became_peered":"2023-11-28T13:02:54.984012+0000","last_unstale":"2023-11-28T13:02:56.654061+0000","last_undegraded":"2023-11-28T13:02:56.654061+0000","last_fullsized":"2023-11-28T13:02:56.654061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:58.068458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557187+0000","last_change":"2023-11-28T13:02:54.557187+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557187+0000","last_undegraded":"2023-11-28T13:02:54.557187+0000","last_fullsized":"2023-11-28T13:02:54.557187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:21.818517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557410+0000","last_change":"2023-11-28T13:02:54.557410+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557410+0000","last_undegraded":"2023-11-28T13:02:54.557410+0000","last_fullsized":"2023-11-28T13:02:54.557410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:09.427676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557425+0000","last_change":"2023-11-28T13:02:54.557425+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557425+0000","last_undegraded":"2023-11-28T13:02:54.557425+0000","last_fullsized":"2023-11-28T13:02:54.557425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:28.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653599+0000","last_change":"2023-11-28T13:02:54.973929+0000","last_active":"2023-11-28T13:02:56.653599+0000","last_peered":"2023-11-28T13:02:56.653599+0000","last_clean":"2023-11-28T13:02:56.653599+0000","last_became_active":"2023-11-28T13:02:54.973639+0000","last_became_peered":"2023-11-28T13:02:54.973639+0000","last_unstale":"2023-11-28T13:02:56.653599+0000","last_undegraded":"2023-11-28T13:02:56.653599+0000","last_fullsized":"2023-11-28T13:02:56.653599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:28.472469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.551962+0000","last_change":"2023-11-28T13:02:53.551962+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.551962+0000","last_undegraded":"2023-11-28T13:02:53.551962+0000","last_fullsized":"2023-11-28T13:02:53.551962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654252+0000","last_change":"2023-11-28T13:02:54.576056+0000","last_active":"2023-11-28T13:02:56.654252+0000","last_peered":"2023-11-28T13:02:56.654252+0000","last_clean":"2023-11-28T13:02:56.654252+0000","last_became_active":"2023-11-28T13:02:54.575718+0000","last_became_peered":"2023-11-28T13:02:54.575718+0000","last_unstale":"2023-11-28T13:02:56.654252+0000","last_undegraded":"2023-11-28T13:02:56.654252+0000","last_fullsized":"2023-11-28T13:02:56.654252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:27.738591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654159+0000","last_change":"2023-11-28T13:02:54.969947+0000","last_active":"2023-11-28T13:02:56.654159+0000","last_peered":"2023-11-28T13:02:56.654159+0000","last_clean":"2023-11-28T13:02:56.654159+0000","last_became_active":"2023-11-28T13:02:54.969698+0000","last_became_peered":"2023-11-28T13:02:54.969698+0000","last_unstale":"2023-11-28T13:02:56.654159+0000","last_undegraded":"2023-11-28T13:02:56.654159+0000","last_fullsized":"2023-11-28T13:02:56.654159+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:05.666567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654306+0000","last_change":"2023-11-28T13:02:54.572156+0000","last_active":"2023-11-28T13:02:56.654306+0000","last_peered":"2023-11-28T13:02:56.654306+0000","last_clean":"2023-11-28T13:02:56.654306+0000","last_became_active":"2023-11-28T13:02:54.571876+0000","last_became_peered":"2023-11-28T13:02:54.571876+0000","last_unstale":"2023-11-28T13:02:56.654306+0000","last_undegraded":"2023-11-28T13:02:56.654306+0000","last_fullsized":"2023-11-28T13:02:56.654306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:50.653241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653781+0000","last_change":"2023-11-28T13:02:54.564777+0000","last_active":"2023-11-28T13:02:56.653781+0000","last_peered":"2023-11-28T13:02:56.653781+0000","last_clean":"2023-11-28T13:02:56.653781+0000","last_became_active":"2023-11-28T13:02:54.564503+0000","last_became_peered":"2023-11-28T13:02:54.564503+0000","last_unstale":"2023-11-28T13:02:56.653781+0000","last_undegraded":"2023-11-28T13:02:56.653781+0000","last_fullsized":"2023-11-28T13:02:56.653781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:39:38.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570949+0000","last_change":"2023-11-28T13:02:54.565557+0000","last_active":"2023-11-28T13:02:54.570949+0000","last_peered":"2023-11-28T13:02:54.570949+0000","last_clean":"2023-11-28T13:02:54.570949+0000","last_became_active":"2023-11-28T13:02:54.564651+0000","last_became_peered":"2023-11-28T13:02:54.564651+0000","last_unstale":"2023-11-28T13:02:54.570949+0000","last_undegraded":"2023-11-28T13:02:54.570949+0000","last_fullsized":"2023-11-28T13:02:54.570949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:36.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570862+0000","last_change":"2023-11-28T13:02:54.565910+0000","last_active":"2023-11-28T13:02:54.570862+0000","last_peered":"2023-11-28T13:02:54.570862+0000","last_clean":"2023-11-28T13:02:54.570862+0000","last_became_active":"2023-11-28T13:02:54.565445+0000","last_became_peered":"2023-11-28T13:02:54.565445+0000","last_unstale":"2023-11-28T13:02:54.570862+0000","last_undegraded":"2023-11-28T13:02:54.570862+0000","last_fullsized":"2023-11-28T13:02:54.570862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.321903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570596+0000","last_change":"2023-11-28T13:02:54.563263+0000","last_active":"2023-11-28T13:02:54.570596+0000","last_peered":"2023-11-28T13:02:54.570596+0000","last_clean":"2023-11-28T13:02:54.570596+0000","last_became_active":"2023-11-28T13:02:54.562391+0000","last_became_peered":"2023-11-28T13:02:54.562391+0000","last_unstale":"2023-11-28T13:02:54.570596+0000","last_undegraded":"2023-11-28T13:02:54.570596+0000","last_fullsized":"2023-11-28T13:02:54.570596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:55.820672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.558233+0000","last_change":"2023-11-28T13:02:54.558233+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.558233+0000","last_undegraded":"2023-11-28T13:02:54.558233+0000","last_fullsized":"2023-11-28T13:02:54.558233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:11.238594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557956+0000","last_change":"2023-11-28T13:02:54.557956+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557956+0000","last_undegraded":"2023-11-28T13:02:54.557956+0000","last_fullsized":"2023-11-28T13:02:54.557956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:02.456068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.552504+0000","last_change":"2023-11-28T13:02:53.552504+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.552504+0000","last_undegraded":"2023-11-28T13:02:53.552504+0000","last_fullsized":"2023-11-28T13:02:53.552504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:54.448798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653728+0000","last_change":"2023-11-28T13:02:54.979265+0000","last_active":"2023-11-28T13:02:56.653728+0000","last_peered":"2023-11-28T13:02:56.653728+0000","last_clean":"2023-11-28T13:02:56.653728+0000","last_became_active":"2023-11-28T13:02:54.978935+0000","last_became_peered":"2023-11-28T13:02:54.978935+0000","last_unstale":"2023-11-28T13:02:56.653728+0000","last_undegraded":"2023-11-28T13:02:56.653728+0000","last_fullsized":"2023-11-28T13:02:56.653728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:18.297439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654399+0000","last_change":"2023-11-28T13:02:54.571685+0000","last_active":"2023-11-28T13:02:56.654399+0000","last_peered":"2023-11-28T13:02:56.654399+0000","last_clean":"2023-11-28T13:02:56.654399+0000","last_became_active":"2023-11-28T13:02:54.571375+0000","last_became_peered":"2023-11-28T13:02:54.571375+0000","last_unstale":"2023-11-28T13:02:56.654399+0000","last_undegraded":"2023-11-28T13:02:56.654399+0000","last_fullsized":"2023-11-28T13:02:56.654399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:45.383406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.552099+0000","last_change":"2023-11-28T13:02:53.552099+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.552099+0000","last_undegraded":"2023-11-28T13:02:53.552099+0000","last_fullsized":"2023-11-28T13:02:53.552099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:24.750211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570836+0000","last_change":"2023-11-28T13:02:54.565731+0000","last_active":"2023-11-28T13:02:54.570836+0000","last_peered":"2023-11-28T13:02:54.570836+0000","last_clean":"2023-11-28T13:02:54.570836+0000","last_became_active":"2023-11-28T13:02:54.565059+0000","last_became_peered":"2023-11-28T13:02:54.565059+0000","last_unstale":"2023-11-28T13:02:54.570836+0000","last_undegraded":"2023-11-28T13:02:54.570836+0000","last_fullsized":"2023-11-28T13:02:54.570836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:36.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570628+0000","last_change":"2023-11-28T13:02:54.565314+0000","last_active":"2023-11-28T13:02:54.570628+0000","last_peered":"2023-11-28T13:02:54.570628+0000","last_clean":"2023-11-28T13:02:54.570628+0000","last_became_active":"2023-11-28T13:02:54.563242+0000","last_became_peered":"2023-11-28T13:02:54.563242+0000","last_unstale":"2023-11-28T13:02:54.570628+0000","last_undegraded":"2023-11-28T13:02:54.570628+0000","last_fullsized":"2023-11-28T13:02:54.570628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:36.187142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.551604+0000","last_change":"2023-11-28T13:02:53.551604+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.551604+0000","last_undegraded":"2023-11-28T13:02:53.551604+0000","last_fullsized":"2023-11-28T13:02:53.551604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:34:44.950034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:51.618966+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:02:51.618966+0000","last_peered":"2023-11-28T13:02:51.618966+0000","last_clean":"2023-11-28T13:02:51.618966+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:02:51.618966+0000","last_undegraded":"2023-11-28T13:02:51.618966+0000","last_fullsized":"2023-11-28T13:02:51.618966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653875+0000","last_change":"2023-11-28T13:02:54.572150+0000","last_active":"2023-11-28T13:02:56.653875+0000","last_peered":"2023-11-28T13:02:56.653875+0000","last_clean":"2023-11-28T13:02:56.653875+0000","last_became_active":"2023-11-28T13:02:54.571846+0000","last_became_peered":"2023-11-28T13:02:54.571846+0000","last_unstale":"2023-11-28T13:02:56.653875+0000","last_undegraded":"2023-11-28T13:02:56.653875+0000","last_fullsized":"2023-11-28T13:02:56.653875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.733701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436262+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:02:50.436262+0000","last_peered":"2023-11-28T13:02:50.436262+0000","last_clean":"2023-11-28T13:02:50.436262+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:02:50.436262+0000","last_undegraded":"2023-11-28T13:02:50.436262+0000","last_fullsized":"2023-11-28T13:02:50.436262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.654923+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:02:50.654923+0000","last_peered":"2023-11-28T13:02:50.654923+0000","last_clean":"2023-11-28T13:02:50.654923+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:02:50.654923+0000","last_undegraded":"2023-11-28T13:02:50.654923+0000","last_fullsized":"2023-11-28T13:02:50.654923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570580+0000","last_change":"2023-11-28T13:02:54.564196+0000","last_active":"2023-11-28T13:02:54.570580+0000","last_peered":"2023-11-28T13:02:54.570580+0000","last_clean":"2023-11-28T13:02:54.570580+0000","last_became_active":"2023-11-28T13:02:54.563920+0000","last_became_peered":"2023-11-28T13:02:54.563920+0000","last_unstale":"2023-11-28T13:02:54.570580+0000","last_undegraded":"2023-11-28T13:02:54.570580+0000","last_fullsized":"2023-11-28T13:02:54.570580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:57.576385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436267+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:02:50.436267+0000","last_peered":"2023-11-28T13:02:50.436267+0000","last_clean":"2023-11-28T13:02:50.436267+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:02:50.436267+0000","last_undegraded":"2023-11-28T13:02:50.436267+0000","last_fullsized":"2023-11-28T13:02:50.436267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436375+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:02:50.436375+0000","last_peered":"2023-11-28T13:02:50.436375+0000","last_clean":"2023-11-28T13:02:50.436375+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:02:50.436375+0000","last_undegraded":"2023-11-28T13:02:50.436375+0000","last_fullsized":"2023-11-28T13:02:50.436375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653826+0000","last_change":"2023-11-28T13:02:54.974128+0000","last_active":"2023-11-28T13:02:56.653826+0000","last_peered":"2023-11-28T13:02:56.653826+0000","last_clean":"2023-11-28T13:02:56.653826+0000","last_became_active":"2023-11-28T13:02:54.973776+0000","last_became_peered":"2023-11-28T13:02:54.973776+0000","last_unstale":"2023-11-28T13:02:56.653826+0000","last_undegraded":"2023-11-28T13:02:56.653826+0000","last_fullsized":"2023-11-28T13:02:56.653826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:49.099601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653780+0000","last_change":"2023-11-28T13:02:54.564097+0000","last_active":"2023-11-28T13:02:56.653780+0000","last_peered":"2023-11-28T13:02:56.653780+0000","last_clean":"2023-11-28T13:02:56.653780+0000","last_became_active":"2023-11-28T13:02:54.563816+0000","last_became_peered":"2023-11-28T13:02:54.563816+0000","last_unstale":"2023-11-28T13:02:56.653780+0000","last_undegraded":"2023-11-28T13:02:56.653780+0000","last_fullsized":"2023-11-28T13:02:56.653780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:52.277147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436342+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:02:50.436342+0000","last_peered":"2023-11-28T13:02:50.436342+0000","last_clean":"2023-11-28T13:02:50.436342+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:02:50.436342+0000","last_undegraded":"2023-11-28T13:02:50.436342+0000","last_fullsized":"2023-11-28T13:02:50.436342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436334+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:02:50.436334+0000","last_peered":"2023-11-28T13:02:50.436334+0000","last_clean":"2023-11-28T13:02:50.436334+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:02:50.436334+0000","last_undegraded":"2023-11-28T13:02:50.436334+0000","last_fullsized":"2023-11-28T13:02:50.436334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:51.618962+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:02:51.618962+0000","last_peered":"2023-11-28T13:02:51.618962+0000","last_clean":"2023-11-28T13:02:51.618962+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:02:51.618962+0000","last_undegraded":"2023-11-28T13:02:51.618962+0000","last_fullsized":"2023-11-28T13:02:51.618962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570476+0000","last_change":"2023-11-28T13:02:54.565033+0000","last_active":"2023-11-28T13:02:54.570476+0000","last_peered":"2023-11-28T13:02:54.570476+0000","last_clean":"2023-11-28T13:02:54.570476+0000","last_became_active":"2023-11-28T13:02:54.564748+0000","last_became_peered":"2023-11-28T13:02:54.564748+0000","last_unstale":"2023-11-28T13:02:54.570476+0000","last_undegraded":"2023-11-28T13:02:54.570476+0000","last_fullsized":"2023-11-28T13:02:54.570476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:58.581060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:49.647344+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:02:49.647344+0000","last_peered":"2023-11-28T13:02:49.647344+0000","last_clean":"2023-11-28T13:02:49.647344+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:02:49.647344+0000","last_undegraded":"2023-11-28T13:02:49.647344+0000","last_fullsized":"2023-11-28T13:02:49.647344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":34,"acting":34,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9600,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362240,"statfs":{"total":96636764160,"available":96626933760,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.497,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.63,"15min":0.67800000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.498,"5min":8.218,"15min":8.218},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39700000000000002,"15min":0.40400000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.499,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.40400000000000003,"15min":0.41299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.47599999999999998,"5min":1.355,"15min":1.355},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.871,"15min":0.95199999999999996},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55400000000000005,"5min":13.384,"15min":13.384},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.58999999999999997,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52300000000000002,"5min":5.226,"15min":5.226},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.75900000000000001,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79400000000000004,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.80100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.85399999999999998,"15min":0.89700000000000002},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81899999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.79100000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.51300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93799999999999994,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.88300000000000001,"15min":0.92900000000000005},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.78900000000000003,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.85999999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.34100000000000003,"15min":0.33500000000000002},"min":{"1min":0.216,"5min":0.187,"15min":0.187},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.28599999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.436},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":9332,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362508,"statfs":{"total":96636764160,"available":96627208192,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.496}]},{"osd":1,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.502,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.753,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59799999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14756,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357084,"statfs":{"total":96636764160,"available":96621654016,"internally_reserved":0,"allocated":1339392,"data_stored":917739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.85299999999999998,"15min":0.93999999999999995},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.85499999999999998,"15min":0.93799999999999994},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61399999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.86199999999999999,"15min":0.93899999999999995},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.98799999999999999,"15min":1.091},"min":{"1min":0.247,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58399999999999996,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.69099999999999995,"15min":0.70899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94299999999999995,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.218,"15min":1.3280000000000001},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68999999999999995,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.204,"15min":1.3109999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76100000000000001,"5min":19.02,"15min":19.02},"last":0.443},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.503}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.748,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.83099999999999996,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.2250000000000001,"15min":1.3320000000000001},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.40999999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10488,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361352,"statfs":{"total":96636764160,"available":96626024448,"internally_reserved":0,"allocated":1294336,"data_stored":891937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":1.415,"15min":1.415},"last":0.59099999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":1.867,"15min":1.867},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71399999999999997,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2749999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.8440000000000003,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":0.623}]},{"osd":4,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.51200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.528,"15min":1.528},"last":0.505}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.45000000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9332,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362508,"statfs":{"total":96636764160,"available":96627208192,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":1.419,"15min":1.419},"last":0.505},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.41699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.442},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.626}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.40300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79900000000000004,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.68899999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52900000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47899999999999998},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67100000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73399999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.748,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.70599999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9140,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362700,"statfs":{"total":96636764160,"available":96627404800,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.44,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.66200000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.623},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.497}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.441,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55600000000000005}]},{"osd":4,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.5,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.20999999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.45900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.33600000000000002,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.441,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52600000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.307},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.373}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54900000000000004,"5min":0.752,"15min":0.752},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.51600000000000001,"5min":0.875,"15min":0.875},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":1.044,"15min":1.044},"last":0.44800000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75900000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71299999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68400000000000005,"5min":1.089,"15min":1.089},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72499999999999998,"5min":1.202,"15min":1.202},"last":0.496},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74399999999999999,"5min":1.02,"15min":1.02},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:02:59.042 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:02:59.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.197+0000 7f31acdce700 1 Processor -- start 2023-11-28T13:02:59.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.197+0000 7f31acdce700 1 -- start start 2023-11-28T13:02:59.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31acdce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a815cef0 0x7f31a80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31acdce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f31a806ba90 0x7f31a80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31acdce700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80cc5f0 con 0x7f31a80cbdf0 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31acdce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a815d470 con 0x7f31a806ba90 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31acdce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a815d750 con 0x7f31a815cef0 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f31a806ba90 0x7f31a80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f31a806ba90 0x7f31a80ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42230/0 (socket says 172.21.15.153:42230) 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 -- 172.21.15.153:0/3227271328 learned_addr learned my addr 172.21.15.153:0/3227271328 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a659c700 1 --2- 172.21.15.153:0/3227271328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a815cef0 0x7f31a80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 -- 172.21.15.153:0/3227271328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a815cef0 msgr2=0x7f31a80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:59.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 --2- 172.21.15.153:0/3227271328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a815cef0 0x7f31a80af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 -- 172.21.15.153:0/3227271328 >> v1:172.21.15.177:6789/0 conn(0x7f31a80cbdf0 legacy=0x7f31a806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 -- 172.21.15.153:0/3227271328 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a815da30 con 0x7f31a806ba90 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a6d9d700 1 --2- 172.21.15.153:0/3227271328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a806ba90 0x7f31a80ca650 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009e50 tx=0x7f31a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e2360c0d0e76b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.198+0000 7f31a559a700 1 -- 172.21.15.153:0/3227271328 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a000e070 con 0x7f31a806ba90 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31a559a700 1 -- 172.21.15.153:0/3227271328 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a0003a20 con 0x7f31a806ba90 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31a559a700 1 -- 172.21.15.153:0/3227271328 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a0003dd0 con 0x7f31a806ba90 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 -- 172.21.15.153:0/3227271328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a806ba90 msgr2=0x7f31a80ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 --2- 172.21.15.153:0/3227271328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a806ba90 0x7f31a80ca650 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009e50 tx=0x7f31a000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 -- 172.21.15.153:0/3227271328 shutdown_connections 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 --2- 172.21.15.153:0/3227271328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a806ba90 0x7f31a80ca650 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 --2- 172.21.15.153:0/3227271328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a815cef0 0x7f31a80af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 -- 172.21.15.153:0/3227271328 >> 172.21.15.153:0/3227271328 conn(0x7f31a80d0000 msgr2=0x7f31a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 -- 172.21.15.153:0/3227271328 shutdown_connections 2023-11-28T13:02:59.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.199+0000 7f31acdce700 1 -- 172.21.15.153:0/3227271328 wait complete. 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.201+0000 7f31acdce700 1 Processor -- start 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.201+0000 7f31acdce700 1 -- start start 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31acdce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a806ba90 0x7f31a8099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31acdce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a80cbdf0 0x7f31a80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31acdce700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a815cef0 0x7f31a815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31acdce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80c80c0 con 0x7f31a80cbdf0 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31acdce700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80c7440 con 0x7f31a815cef0 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31acdce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80c7720 con 0x7f31a806ba90 2023-11-28T13:02:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a80cbdf0 0x7f31a80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a80cbdf0 0x7f31a80b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42234/0 (socket says 172.21.15.153:42234) 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 -- 172.21.15.153:0/3348764922 learned_addr learned my addr 172.21.15.153:0/3348764922 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a6d9d700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a815cef0 0x7f31a815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a659c700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a806ba90 0x7f31a8099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 -- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a806ba90 msgr2=0x7f31a8099d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a806ba90 0x7f31a8099d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 -- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a815cef0 msgr2=0x7f31a815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a815cef0 0x7f31a815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 -- 172.21.15.153:0/3348764922 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a0009b30 con 0x7f31a80cbdf0 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a6d9d700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a815cef0 0x7f31a815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:59.207 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f31a5d9b700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a80cbdf0 0x7f31a80b4d50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f319c00c4d0 tx=0x7f319c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.202+0000 7f3196ffd700 1 -- 172.21.15.153:0/3348764922 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c011790 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.203+0000 7f3196ffd700 1 -- 172.21.15.153:0/3348764922 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c0118f0 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.203+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a815dff0 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.203+0000 7f3196ffd700 1 -- 172.21.15.153:0/3348764922 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c011ca0 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.203+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a815dd10 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.203+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a80cab90 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.204+0000 7f3196ffd700 1 -- 172.21.15.153:0/3348764922 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f319c010050 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.204+0000 7f3196ffd700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f31880682f0 0x7f318806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.204+0000 7f3196ffd700 1 -- 172.21.15.153:0/3348764922 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f319c085980 con 0x7f31a80cbdf0 2023-11-28T13:02:59.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.204+0000 7f31a659c700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f31880682f0 0x7f318806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.205+0000 7f31a659c700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f31880682f0 0x7f318806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f31a815c130 tx=0x7f3198008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.208+0000 7f3196ffd700 1 -- 172.21.15.153:0/3348764922 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f319c0521e0 con 0x7f31a80cbdf0 2023-11-28T13:02:59.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.387+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31a80af720 con 0x7f31880682f0 2023-11-28T13:02:59.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.395+0000 7f3196ffd700 1 -- 172.21.15.153:0/3348764922 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300538 (secure 0 0 0) 0x7f31a80af720 con 0x7f31880682f0 2023-11-28T13:02:59.402 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:59.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f31880682f0 msgr2=0x7f318806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f31880682f0 0x7f318806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f31a815c130 tx=0x7f3198008040 comp rx=0 tx=0).stop 2023-11-28T13:02:59.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a80cbdf0 msgr2=0x7f31a80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a80cbdf0 0x7f31a80b4d50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f319c00c4d0 tx=0x7f319c00c890 comp rx=0 tx=0).stop 2023-11-28T13:02:59.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 shutdown_connections 2023-11-28T13:02:59.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31a815cef0 0x7f31a815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31a806ba90 0x7f31a8099d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f31880682f0 0x7f318806a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 --2- 172.21.15.153:0/3348764922 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f31a80cbdf0 0x7f31a80b4d50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 >> 172.21.15.153:0/3348764922 conn(0x7f31a80d0000 msgr2=0x7f31a80cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 shutdown_connections 2023-11-28T13:02:59.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.399+0000 7f31acdce700 1 -- 172.21.15.153:0/3348764922 wait complete. 2023-11-28T13:02:59.406 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:02:59.412 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2023-11-28T13:02:58.182211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":166,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":81208,"kb_used_data":6712,"kb_used_omap":0,"kb_used_meta":74432,"kb_avail":754893512,"statfs":{"total":773094113280,"available":773010956288,"internally_reserved":0,"allocated":6873088,"data_stored":3637600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76218368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.635722"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654061+0000","last_change":"2023-11-28T13:02:54.984256+0000","last_active":"2023-11-28T13:02:56.654061+0000","last_peered":"2023-11-28T13:02:56.654061+0000","last_clean":"2023-11-28T13:02:56.654061+0000","last_became_active":"2023-11-28T13:02:54.984012+0000","last_became_peered":"2023-11-28T13:02:54.984012+0000","last_unstale":"2023-11-28T13:02:56.654061+0000","last_undegraded":"2023-11-28T13:02:56.654061+0000","last_fullsized":"2023-11-28T13:02:56.654061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:58.068458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557187+0000","last_change":"2023-11-28T13:02:54.557187+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557187+0000","last_undegraded":"2023-11-28T13:02:54.557187+0000","last_fullsized":"2023-11-28T13:02:54.557187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:21.818517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557410+0000","last_change":"2023-11-28T13:02:54.557410+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557410+0000","last_undegraded":"2023-11-28T13:02:54.557410+0000","last_fullsized":"2023-11-28T13:02:54.557410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:09.427676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557425+0000","last_change":"2023-11-28T13:02:54.557425+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557425+0000","last_undegraded":"2023-11-28T13:02:54.557425+0000","last_fullsized":"2023-11-28T13:02:54.557425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:28.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653599+0000","last_change":"2023-11-28T13:02:54.973929+0000","last_active":"2023-11-28T13:02:56.653599+0000","last_peered":"2023-11-28T13:02:56.653599+0000","last_clean":"2023-11-28T13:02:56.653599+0000","last_became_active":"2023-11-28T13:02:54.973639+0000","last_became_peered":"2023-11-28T13:02:54.973639+0000","last_unstale":"2023-11-28T13:02:56.653599+0000","last_undegraded":"2023-11-28T13:02:56.653599+0000","last_fullsized":"2023-11-28T13:02:56.653599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:28.472469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.551962+0000","last_change":"2023-11-28T13:02:53.551962+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.551962+0000","last_undegraded":"2023-11-28T13:02:53.551962+0000","last_fullsized":"2023-11-28T13:02:53.551962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654252+0000","last_change":"2023-11-28T13:02:54.576056+0000","last_active":"2023-11-28T13:02:56.654252+0000","last_peered":"2023-11-28T13:02:56.654252+0000","last_clean":"2023-11-28T13:02:56.654252+0000","last_became_active":"2023-11-28T13:02:54.575718+0000","last_became_peered":"2023-11-28T13:02:54.575718+0000","last_unstale":"2023-11-28T13:02:56.654252+0000","last_undegraded":"2023-11-28T13:02:56.654252+0000","last_fullsized":"2023-11-28T13:02:56.654252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:27.738591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654159+0000","last_change":"2023-11-28T13:02:54.969947+0000","last_active":"2023-11-28T13:02:56.654159+0000","last_peered":"2023-11-28T13:02:56.654159+0000","last_clean":"2023-11-28T13:02:56.654159+0000","last_became_active":"2023-11-28T13:02:54.969698+0000","last_became_peered":"2023-11-28T13:02:54.969698+0000","last_unstale":"2023-11-28T13:02:56.654159+0000","last_undegraded":"2023-11-28T13:02:56.654159+0000","last_fullsized":"2023-11-28T13:02:56.654159+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:05.666567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654306+0000","last_change":"2023-11-28T13:02:54.572156+0000","last_active":"2023-11-28T13:02:56.654306+0000","last_peered":"2023-11-28T13:02:56.654306+0000","last_clean":"2023-11-28T13:02:56.654306+0000","last_became_active":"2023-11-28T13:02:54.571876+0000","last_became_peered":"2023-11-28T13:02:54.571876+0000","last_unstale":"2023-11-28T13:02:56.654306+0000","last_undegraded":"2023-11-28T13:02:56.654306+0000","last_fullsized":"2023-11-28T13:02:56.654306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:50.653241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653781+0000","last_change":"2023-11-28T13:02:54.564777+0000","last_active":"2023-11-28T13:02:56.653781+0000","last_peered":"2023-11-28T13:02:56.653781+0000","last_clean":"2023-11-28T13:02:56.653781+0000","last_became_active":"2023-11-28T13:02:54.564503+0000","last_became_peered":"2023-11-28T13:02:54.564503+0000","last_unstale":"2023-11-28T13:02:56.653781+0000","last_undegraded":"2023-11-28T13:02:56.653781+0000","last_fullsized":"2023-11-28T13:02:56.653781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:39:38.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570949+0000","last_change":"2023-11-28T13:02:54.565557+0000","last_active":"2023-11-28T13:02:54.570949+0000","last_peered":"2023-11-28T13:02:54.570949+0000","last_clean":"2023-11-28T13:02:54.570949+0000","last_became_active":"2023-11-28T13:02:54.564651+0000","last_became_peered":"2023-11-28T13:02:54.564651+0000","last_unstale":"2023-11-28T13:02:54.570949+0000","last_undegraded":"2023-11-28T13:02:54.570949+0000","last_fullsized":"2023-11-28T13:02:54.570949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:36.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570862+0000","last_change":"2023-11-28T13:02:54.565910+0000","last_active":"2023-11-28T13:02:54.570862+0000","last_peered":"2023-11-28T13:02:54.570862+0000","last_clean":"2023-11-28T13:02:54.570862+0000","last_became_active":"2023-11-28T13:02:54.565445+0000","last_became_peered":"2023-11-28T13:02:54.565445+0000","last_unstale":"2023-11-28T13:02:54.570862+0000","last_undegraded":"2023-11-28T13:02:54.570862+0000","last_fullsized":"2023-11-28T13:02:54.570862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.321903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570596+0000","last_change":"2023-11-28T13:02:54.563263+0000","last_active":"2023-11-28T13:02:54.570596+0000","last_peered":"2023-11-28T13:02:54.570596+0000","last_clean":"2023-11-28T13:02:54.570596+0000","last_became_active":"2023-11-28T13:02:54.562391+0000","last_became_peered":"2023-11-28T13:02:54.562391+0000","last_unstale":"2023-11-28T13:02:54.570596+0000","last_undegraded":"2023-11-28T13:02:54.570596+0000","last_fullsized":"2023-11-28T13:02:54.570596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:55.820672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.558233+0000","last_change":"2023-11-28T13:02:54.558233+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.558233+0000","last_undegraded":"2023-11-28T13:02:54.558233+0000","last_fullsized":"2023-11-28T13:02:54.558233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:11.238594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557956+0000","last_change":"2023-11-28T13:02:54.557956+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557956+0000","last_undegraded":"2023-11-28T13:02:54.557956+0000","last_fullsized":"2023-11-28T13:02:54.557956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:02.456068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.552504+0000","last_change":"2023-11-28T13:02:53.552504+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.552504+0000","last_undegraded":"2023-11-28T13:02:53.552504+0000","last_fullsized":"2023-11-28T13:02:53.552504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:54.448798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653728+0000","last_change":"2023-11-28T13:02:54.979265+0000","last_active":"2023-11-28T13:02:56.653728+0000","last_peered":"2023-11-28T13:02:56.653728+0000","last_clean":"2023-11-28T13:02:56.653728+0000","last_became_active":"2023-11-28T13:02:54.978935+0000","last_became_peered":"2023-11-28T13:02:54.978935+0000","last_unstale":"2023-11-28T13:02:56.653728+0000","last_undegraded":"2023-11-28T13:02:56.653728+0000","last_fullsized":"2023-11-28T13:02:56.653728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:18.297439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654399+0000","last_change":"2023-11-28T13:02:54.571685+0000","last_active":"2023-11-28T13:02:56.654399+0000","last_peered":"2023-11-28T13:02:56.654399+0000","last_clean":"2023-11-28T13:02:56.654399+0000","last_became_active":"2023-11-28T13:02:54.571375+0000","last_became_peered":"2023-11-28T13:02:54.571375+0000","last_unstale":"2023-11-28T13:02:56.654399+0000","last_undegraded":"2023-11-28T13:02:56.654399+0000","last_fullsized":"2023-11-28T13:02:56.654399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:45.383406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.552099+0000","last_change":"2023-11-28T13:02:53.552099+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.552099+0000","last_undegraded":"2023-11-28T13:02:53.552099+0000","last_fullsized":"2023-11-28T13:02:53.552099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:24.750211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570836+0000","last_change":"2023-11-28T13:02:54.565731+0000","last_active":"2023-11-28T13:02:54.570836+0000","last_peered":"2023-11-28T13:02:54.570836+0000","last_clean":"2023-11-28T13:02:54.570836+0000","last_became_active":"2023-11-28T13:02:54.565059+0000","last_became_peered":"2023-11-28T13:02:54.565059+0000","last_unstale":"2023-11-28T13:02:54.570836+0000","last_undegraded":"2023-11-28T13:02:54.570836+0000","last_fullsized":"2023-11-28T13:02:54.570836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:36.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570628+0000","last_change":"2023-11-28T13:02:54.565314+0000","last_active":"2023-11-28T13:02:54.570628+0000","last_peered":"2023-11-28T13:02:54.570628+0000","last_clean":"2023-11-28T13:02:54.570628+0000","last_became_active":"2023-11-28T13:02:54.563242+0000","last_became_peered":"2023-11-28T13:02:54.563242+0000","last_unstale":"2023-11-28T13:02:54.570628+0000","last_undegraded":"2023-11-28T13:02:54.570628+0000","last_fullsized":"2023-11-28T13:02:54.570628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:36.187142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.551604+0000","last_change":"2023-11-28T13:02:53.551604+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.551604+0000","last_undegraded":"2023-11-28T13:02:53.551604+0000","last_fullsized":"2023-11-28T13:02:53.551604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:34:44.950034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:51.618966+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:02:51.618966+0000","last_peered":"2023-11-28T13:02:51.618966+0000","last_clean":"2023-11-28T13:02:51.618966+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:02:51.618966+0000","last_undegraded":"2023-11-28T13:02:51.618966+0000","last_fullsized":"2023-11-28T13:02:51.618966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653875+0000","last_change":"2023-11-28T13:02:54.572150+0000","last_active":"2023-11-28T13:02:56.653875+0000","last_peered":"2023-11-28T13:02:56.653875+0000","last_clean":"2023-11-28T13:02:56.653875+0000","last_became_active":"2023-11-28T13:02:54.571846+0000","last_became_peered":"2023-11-28T13:02:54.571846+0000","last_unstale":"2023-11-28T13:02:56.653875+0000","last_undegraded":"2023-11-28T13:02:56.653875+0000","last_fullsized":"2023-11-28T13:02:56.653875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.733701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436262+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:02:50.436262+0000","last_peered":"2023-11-28T13:02:50.436262+0000","last_clean":"2023-11-28T13:02:50.436262+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:02:50.436262+0000","last_undegraded":"2023-11-28T13:02:50.436262+0000","last_fullsized":"2023-11-28T13:02:50.436262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.654923+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:02:50.654923+0000","last_peered":"2023-11-28T13:02:50.654923+0000","last_clean":"2023-11-28T13:02:50.654923+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:02:50.654923+0000","last_undegraded":"2023-11-28T13:02:50.654923+0000","last_fullsized":"2023-11-28T13:02:50.654923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570580+0000","last_change":"2023-11-28T13:02:54.564196+0000","last_active":"2023-11-28T13:02:54.570580+0000","last_peered":"2023-11-28T13:02:54.570580+0000","last_clean":"2023-11-28T13:02:54.570580+0000","last_became_active":"2023-11-28T13:02:54.563920+0000","last_became_peered":"2023-11-28T13:02:54.563920+0000","last_unstale":"2023-11-28T13:02:54.570580+0000","last_undegraded":"2023-11-28T13:02:54.570580+0000","last_fullsized":"2023-11-28T13:02:54.570580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:57.576385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436267+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:02:50.436267+0000","last_peered":"2023-11-28T13:02:50.436267+0000","last_clean":"2023-11-28T13:02:50.436267+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:02:50.436267+0000","last_undegraded":"2023-11-28T13:02:50.436267+0000","last_fullsized":"2023-11-28T13:02:50.436267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436375+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:02:50.436375+0000","last_peered":"2023-11-28T13:02:50.436375+0000","last_clean":"2023-11-28T13:02:50.436375+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:02:50.436375+0000","last_undegraded":"2023-11-28T13:02:50.436375+0000","last_fullsized":"2023-11-28T13:02:50.436375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653826+0000","last_change":"2023-11-28T13:02:54.974128+0000","last_active":"2023-11-28T13:02:56.653826+0000","last_peered":"2023-11-28T13:02:56.653826+0000","last_clean":"2023-11-28T13:02:56.653826+0000","last_became_active":"2023-11-28T13:02:54.973776+0000","last_became_peered":"2023-11-28T13:02:54.973776+0000","last_unstale":"2023-11-28T13:02:56.653826+0000","last_undegraded":"2023-11-28T13:02:56.653826+0000","last_fullsized":"2023-11-28T13:02:56.653826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:49.099601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653780+0000","last_change":"2023-11-28T13:02:54.564097+0000","last_active":"2023-11-28T13:02:56.653780+0000","last_peered":"2023-11-28T13:02:56.653780+0000","last_clean":"2023-11-28T13:02:56.653780+0000","last_became_active":"2023-11-28T13:02:54.563816+0000","last_became_peered":"2023-11-28T13:02:54.563816+0000","last_unstale":"2023-11-28T13:02:56.653780+0000","last_undegraded":"2023-11-28T13:02:56.653780+0000","last_fullsized":"2023-11-28T13:02:56.653780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:52.277147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436342+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:02:50.436342+0000","last_peered":"2023-11-28T13:02:50.436342+0000","last_clean":"2023-11-28T13:02:50.436342+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:02:50.436342+0000","last_undegraded":"2023-11-28T13:02:50.436342+0000","last_fullsized":"2023-11-28T13:02:50.436342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436334+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:02:50.436334+0000","last_peered":"2023-11-28T13:02:50.436334+0000","last_clean":"2023-11-28T13:02:50.436334+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:02:50.436334+0000","last_undegraded":"2023-11-28T13:02:50.436334+0000","last_fullsized":"2023-11-28T13:02:50.436334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:51.618962+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:02:51.618962+0000","last_peered":"2023-11-28T13:02:51.618962+0000","last_clean":"2023-11-28T13:02:51.618962+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:02:51.618962+0000","last_undegraded":"2023-11-28T13:02:51.618962+0000","last_fullsized":"2023-11-28T13:02:51.618962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570476+0000","last_change":"2023-11-28T13:02:54.565033+0000","last_active":"2023-11-28T13:02:54.570476+0000","last_peered":"2023-11-28T13:02:54.570476+0000","last_clean":"2023-11-28T13:02:54.570476+0000","last_became_active":"2023-11-28T13:02:54.564748+0000","last_became_peered":"2023-11-28T13:02:54.564748+0000","last_unstale":"2023-11-28T13:02:54.570476+0000","last_undegraded":"2023-11-28T13:02:54.570476+0000","last_fullsized":"2023-11-28T13:02:54.570476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:58.581060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:49.647344+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:02:49.647344+0000","last_peered":"2023-11-28T13:02:49.647344+0000","last_clean":"2023-11-28T13:02:49.647344+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:02:49.647344+0000","last_undegraded":"2023-11-28T13:02:49.647344+0000","last_fullsized":"2023-11-28T13:02:49.647344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":34,"acting":34,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9600,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362240,"statfs":{"total":96636764160,"available":96626933760,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.497,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.63,"15min":0.67800000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.498,"5min":8.218,"15min":8.218},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39700000000000002,"15min":0.40400000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.499,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.40400000000000003,"15min":0.41299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.47599999999999998,"5min":1.355,"15min":1.355},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.871,"15min":0.95199999999999996},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55400000000000005,"5min":13.384,"15min":13.384},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.58999999999999997,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52300000000000002,"5min":5.226,"15min":5.226},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.75900000000000001,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79400000000000004,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.80100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.85399999999999998,"15min":0.89700000000000002},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81899999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.79100000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.51300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93799999999999994,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.88300000000000001,"15min":0.92900000000000005},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.78900000000000003,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.85999999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.34100000000000003,"15min":0.33500000000000002},"min":{"1min":0.216,"5min":0.187,"15min":0.187},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.28599999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.436},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":9332,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362508,"statfs":{"total":96636764160,"available":96627208192,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.496}]},{"osd":1,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.502,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.753,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59799999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14756,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357084,"statfs":{"total":96636764160,"available":96621654016,"internally_reserved":0,"allocated":1339392,"data_stored":917739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.85299999999999998,"15min":0.93999999999999995},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.85499999999999998,"15min":0.93799999999999994},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61399999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.86199999999999999,"15min":0.93899999999999995},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.98799999999999999,"15min":1.091},"min":{"1min":0.247,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58399999999999996,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.69099999999999995,"15min":0.70899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94299999999999995,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.218,"15min":1.3280000000000001},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68999999999999995,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.204,"15min":1.3109999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76100000000000001,"5min":19.02,"15min":19.02},"last":0.443},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.503}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.748,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.83099999999999996,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.2250000000000001,"15min":1.3320000000000001},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.40999999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10488,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361352,"statfs":{"total":96636764160,"available":96626024448,"internally_reserved":0,"allocated":1294336,"data_stored":891937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":1.415,"15min":1.415},"last":0.59099999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":1.867,"15min":1.867},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71399999999999997,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2749999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.8440000000000003,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":0.623}]},{"osd":4,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.51200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.528,"15min":1.528},"last":0.505}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.45000000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9332,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362508,"statfs":{"total":96636764160,"available":96627208192,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":1.419,"15min":1.419},"last":0.505},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.41699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.442},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.626}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.40300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79900000000000004,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.68899999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52900000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47899999999999998},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67100000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73399999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.748,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.70599999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9140,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362700,"statfs":{"total":96636764160,"available":96627404800,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.44,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.66200000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.623},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.497}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.441,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55600000000000005}]},{"osd":4,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.5,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.20999999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.45900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.33600000000000002,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.441,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52600000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.307},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.373}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54900000000000004,"5min":0.752,"15min":0.752},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.51600000000000001,"5min":0.875,"15min":0.875},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":1.044,"15min":1.044},"last":0.44800000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75900000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71299999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68400000000000005,"5min":1.089,"15min":1.089},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72499999999999998,"5min":1.202,"15min":1.202},"last":0.496},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74399999999999999,"5min":1.02,"15min":1.02},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:02:59.417 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:02:59.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.572+0000 7fe11a5eb700 1 Processor -- start 2023-11-28T13:02:59.573 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.572+0000 7fe11a5eb700 1 -- start start 2023-11-28T13:02:59.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe11a5eb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe114154e70 0x7fe1140cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe11a5eb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe114071750 0x7fe11415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe11a5eb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe114155240 con 0x7fe1140cf360 2023-11-28T13:02:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe11a5eb700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11415b100 con 0x7fe114071750 2023-11-28T13:02:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe11a5eb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11415b3e0 con 0x7fe114154e70 2023-11-28T13:02:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe118b88700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe114071750 0x7fe11415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe118b88700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe114071750 0x7fe11415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42244/0 (socket says 172.21.15.153:42244) 2023-11-28T13:02:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.573+0000 7fe118b88700 1 -- 172.21.15.153:0/2433225738 learned_addr learned my addr 172.21.15.153:0/2433225738 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe118b88700 1 -- 172.21.15.153:0/2433225738 >> v2:172.21.15.177:3300/0 conn(0x7fe114154e70 msgr2=0x7fe1140cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe118b88700 1 --2- 172.21.15.153:0/2433225738 >> v2:172.21.15.177:3300/0 conn(0x7fe114154e70 0x7fe1140cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe118b88700 1 -- 172.21.15.153:0/2433225738 >> v1:172.21.15.177:6789/0 conn(0x7fe1140cf360 legacy=0x7fe114071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe118b88700 1 -- 172.21.15.153:0/2433225738 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11415b6c0 con 0x7fe114071750 2023-11-28T13:02:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe118b88700 1 --2- 172.21.15.153:0/2433225738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe114071750 0x7fe11415e680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe114073b80 tx=0x7fe10c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf4cd4a79d43b34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe112ffd700 1 -- 172.21.15.153:0/2433225738 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00e070 con 0x7fe114071750 2023-11-28T13:02:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe112ffd700 1 -- 172.21.15.153:0/2433225738 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c003a20 con 0x7fe114071750 2023-11-28T13:02:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe112ffd700 1 -- 172.21.15.153:0/2433225738 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003dd0 con 0x7fe114071750 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 -- 172.21.15.153:0/2433225738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe114071750 msgr2=0x7fe11415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/2433225738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe114071750 0x7fe11415e680 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe114073b80 tx=0x7fe10c00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 -- 172.21.15.153:0/2433225738 shutdown_connections 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/2433225738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe114071750 0x7fe11415e680 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/2433225738 >> v2:172.21.15.177:3300/0 conn(0x7fe114154e70 0x7fe1140cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 -- 172.21.15.153:0/2433225738 >> 172.21.15.153:0/2433225738 conn(0x7fe1140d0000 msgr2=0x7fe1140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 -- 172.21.15.153:0/2433225738 shutdown_connections 2023-11-28T13:02:59.579 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.574+0000 7fe11a5eb700 1 -- 172.21.15.153:0/2433225738 wait complete. 2023-11-28T13:02:59.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.576+0000 7fe11a5eb700 1 Processor -- start 2023-11-28T13:02:59.580 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.576+0000 7fe11a5eb700 1 -- start start 2023-11-28T13:02:59.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.576+0000 7fe11a5eb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe114071750 0x7fe114152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.582 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.576+0000 7fe11a5eb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1140cf360 0x7fe1140cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.576+0000 7fe11a5eb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe114154e70 0x7fe1140cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.576+0000 7fe11a5eb700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1141538e0 con 0x7fe1140cf360 2023-11-28T13:02:59.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.576+0000 7fe11a5eb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140c3b30 con 0x7fe114071750 2023-11-28T13:02:59.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe11a5eb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe114153300 con 0x7fe114154e70 2023-11-28T13:02:59.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1140cf360 0x7fe1140cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1140cf360 0x7fe1140cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42248/0 (socket says 172.21.15.153:42248) 2023-11-28T13:02:59.583 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 -- 172.21.15.153:0/3098685982 learned_addr learned my addr 172.21.15.153:0/3098685982 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe118b88700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe114154e70 0x7fe1140cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe113fff700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe114071750 0x7fe114152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 -- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe114154e70 msgr2=0x7fe1140cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe114154e70 0x7fe1140cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 -- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe114071750 msgr2=0x7fe114152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe114071750 0x7fe114152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 -- 172.21.15.153:0/3098685982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009b30 con 0x7fe1140cf360 2023-11-28T13:02:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe1137fe700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1140cf360 0x7fe1140cbbc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c4a0 tx=0x7fe10800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe110ff9700 1 -- 172.21.15.153:0/3098685982 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080054e0 con 0x7fe1140cf360 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe110ff9700 1 -- 172.21.15.153:0/3098685982 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108003680 con 0x7fe1140cf360 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1140af520 con 0x7fe1140cf360 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe110ff9700 1 -- 172.21.15.153:0/3098685982 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108003a30 con 0x7fe1140cf360 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.577+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11415b9a0 con 0x7fe1140cf360 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.578+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11415adb0 con 0x7fe1140cf360 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.579+0000 7fe110ff9700 1 -- 172.21.15.153:0/3098685982 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe108019030 con 0x7fe1140cf360 2023-11-28T13:02:59.585 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.579+0000 7fe110ff9700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1000682f0 0x7fe10006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.579+0000 7fe110ff9700 1 -- 172.21.15.153:0/3098685982 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe108084f30 con 0x7fe1140cf360 2023-11-28T13:02:59.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.579+0000 7fe113fff700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1000682f0 0x7fe10006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.579+0000 7fe113fff700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1000682f0 0x7fe10006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ae7b0 tx=0x7fe104009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.583+0000 7fe110ff9700 1 -- 172.21.15.153:0/3098685982 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe108051790 con 0x7fe1140cf360 2023-11-28T13:02:59.763 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.762+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe11409e000 con 0x7fe1000682f0 2023-11-28T13:02:59.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.770+0000 7fe110ff9700 1 -- 172.21.15.153:0/3098685982 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300538 (secure 0 0 0) 0x7fe11409e000 con 0x7fe1000682f0 2023-11-28T13:02:59.774 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1000682f0 msgr2=0x7fe10006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1000682f0 0x7fe10006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ae7b0 tx=0x7fe104009110 comp rx=0 tx=0).stop 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1140cf360 msgr2=0x7fe1140cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1140cf360 0x7fe1140cbbc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c4a0 tx=0x7fe10800c7b0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 shutdown_connections 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe114154e70 0x7fe1140cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe114071750 0x7fe114152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1000682f0 0x7fe10006a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 --2- 172.21.15.153:0/3098685982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1140cf360 0x7fe1140cbbc0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.773+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 >> 172.21.15.153:0/3098685982 conn(0x7fe1140d0000 msgr2=0x7fe1140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.774+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 shutdown_connections 2023-11-28T13:02:59.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.774+0000 7fe11a5eb700 1 -- 172.21.15.153:0/3098685982 wait complete. 2023-11-28T13:02:59.778 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:02:59.787 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2023-11-28T13:02:58.182211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":166,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":81208,"kb_used_data":6712,"kb_used_omap":0,"kb_used_meta":74432,"kb_avail":754893512,"statfs":{"total":773094113280,"available":773010956288,"internally_reserved":0,"allocated":6873088,"data_stored":3637600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76218368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.635722"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654061+0000","last_change":"2023-11-28T13:02:54.984256+0000","last_active":"2023-11-28T13:02:56.654061+0000","last_peered":"2023-11-28T13:02:56.654061+0000","last_clean":"2023-11-28T13:02:56.654061+0000","last_became_active":"2023-11-28T13:02:54.984012+0000","last_became_peered":"2023-11-28T13:02:54.984012+0000","last_unstale":"2023-11-28T13:02:56.654061+0000","last_undegraded":"2023-11-28T13:02:56.654061+0000","last_fullsized":"2023-11-28T13:02:56.654061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:58.068458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557187+0000","last_change":"2023-11-28T13:02:54.557187+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557187+0000","last_undegraded":"2023-11-28T13:02:54.557187+0000","last_fullsized":"2023-11-28T13:02:54.557187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:21.818517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557410+0000","last_change":"2023-11-28T13:02:54.557410+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557410+0000","last_undegraded":"2023-11-28T13:02:54.557410+0000","last_fullsized":"2023-11-28T13:02:54.557410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:09.427676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557425+0000","last_change":"2023-11-28T13:02:54.557425+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557425+0000","last_undegraded":"2023-11-28T13:02:54.557425+0000","last_fullsized":"2023-11-28T13:02:54.557425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:28.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653599+0000","last_change":"2023-11-28T13:02:54.973929+0000","last_active":"2023-11-28T13:02:56.653599+0000","last_peered":"2023-11-28T13:02:56.653599+0000","last_clean":"2023-11-28T13:02:56.653599+0000","last_became_active":"2023-11-28T13:02:54.973639+0000","last_became_peered":"2023-11-28T13:02:54.973639+0000","last_unstale":"2023-11-28T13:02:56.653599+0000","last_undegraded":"2023-11-28T13:02:56.653599+0000","last_fullsized":"2023-11-28T13:02:56.653599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:28.472469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.551962+0000","last_change":"2023-11-28T13:02:53.551962+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.551962+0000","last_undegraded":"2023-11-28T13:02:53.551962+0000","last_fullsized":"2023-11-28T13:02:53.551962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654252+0000","last_change":"2023-11-28T13:02:54.576056+0000","last_active":"2023-11-28T13:02:56.654252+0000","last_peered":"2023-11-28T13:02:56.654252+0000","last_clean":"2023-11-28T13:02:56.654252+0000","last_became_active":"2023-11-28T13:02:54.575718+0000","last_became_peered":"2023-11-28T13:02:54.575718+0000","last_unstale":"2023-11-28T13:02:56.654252+0000","last_undegraded":"2023-11-28T13:02:56.654252+0000","last_fullsized":"2023-11-28T13:02:56.654252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:27.738591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654159+0000","last_change":"2023-11-28T13:02:54.969947+0000","last_active":"2023-11-28T13:02:56.654159+0000","last_peered":"2023-11-28T13:02:56.654159+0000","last_clean":"2023-11-28T13:02:56.654159+0000","last_became_active":"2023-11-28T13:02:54.969698+0000","last_became_peered":"2023-11-28T13:02:54.969698+0000","last_unstale":"2023-11-28T13:02:56.654159+0000","last_undegraded":"2023-11-28T13:02:56.654159+0000","last_fullsized":"2023-11-28T13:02:56.654159+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:05.666567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654306+0000","last_change":"2023-11-28T13:02:54.572156+0000","last_active":"2023-11-28T13:02:56.654306+0000","last_peered":"2023-11-28T13:02:56.654306+0000","last_clean":"2023-11-28T13:02:56.654306+0000","last_became_active":"2023-11-28T13:02:54.571876+0000","last_became_peered":"2023-11-28T13:02:54.571876+0000","last_unstale":"2023-11-28T13:02:56.654306+0000","last_undegraded":"2023-11-28T13:02:56.654306+0000","last_fullsized":"2023-11-28T13:02:56.654306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:50.653241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653781+0000","last_change":"2023-11-28T13:02:54.564777+0000","last_active":"2023-11-28T13:02:56.653781+0000","last_peered":"2023-11-28T13:02:56.653781+0000","last_clean":"2023-11-28T13:02:56.653781+0000","last_became_active":"2023-11-28T13:02:54.564503+0000","last_became_peered":"2023-11-28T13:02:54.564503+0000","last_unstale":"2023-11-28T13:02:56.653781+0000","last_undegraded":"2023-11-28T13:02:56.653781+0000","last_fullsized":"2023-11-28T13:02:56.653781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:39:38.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570949+0000","last_change":"2023-11-28T13:02:54.565557+0000","last_active":"2023-11-28T13:02:54.570949+0000","last_peered":"2023-11-28T13:02:54.570949+0000","last_clean":"2023-11-28T13:02:54.570949+0000","last_became_active":"2023-11-28T13:02:54.564651+0000","last_became_peered":"2023-11-28T13:02:54.564651+0000","last_unstale":"2023-11-28T13:02:54.570949+0000","last_undegraded":"2023-11-28T13:02:54.570949+0000","last_fullsized":"2023-11-28T13:02:54.570949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:36.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570862+0000","last_change":"2023-11-28T13:02:54.565910+0000","last_active":"2023-11-28T13:02:54.570862+0000","last_peered":"2023-11-28T13:02:54.570862+0000","last_clean":"2023-11-28T13:02:54.570862+0000","last_became_active":"2023-11-28T13:02:54.565445+0000","last_became_peered":"2023-11-28T13:02:54.565445+0000","last_unstale":"2023-11-28T13:02:54.570862+0000","last_undegraded":"2023-11-28T13:02:54.570862+0000","last_fullsized":"2023-11-28T13:02:54.570862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.321903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570596+0000","last_change":"2023-11-28T13:02:54.563263+0000","last_active":"2023-11-28T13:02:54.570596+0000","last_peered":"2023-11-28T13:02:54.570596+0000","last_clean":"2023-11-28T13:02:54.570596+0000","last_became_active":"2023-11-28T13:02:54.562391+0000","last_became_peered":"2023-11-28T13:02:54.562391+0000","last_unstale":"2023-11-28T13:02:54.570596+0000","last_undegraded":"2023-11-28T13:02:54.570596+0000","last_fullsized":"2023-11-28T13:02:54.570596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:55.820672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.558233+0000","last_change":"2023-11-28T13:02:54.558233+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.558233+0000","last_undegraded":"2023-11-28T13:02:54.558233+0000","last_fullsized":"2023-11-28T13:02:54.558233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:11.238594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-11-28T13:02:54.557956+0000","last_change":"2023-11-28T13:02:54.557956+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.557956+0000","last_undegraded":"2023-11-28T13:02:54.557956+0000","last_fullsized":"2023-11-28T13:02:54.557956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:02.456068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.552504+0000","last_change":"2023-11-28T13:02:53.552504+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.552504+0000","last_undegraded":"2023-11-28T13:02:53.552504+0000","last_fullsized":"2023-11-28T13:02:53.552504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:54.448798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653728+0000","last_change":"2023-11-28T13:02:54.979265+0000","last_active":"2023-11-28T13:02:56.653728+0000","last_peered":"2023-11-28T13:02:56.653728+0000","last_clean":"2023-11-28T13:02:56.653728+0000","last_became_active":"2023-11-28T13:02:54.978935+0000","last_became_peered":"2023-11-28T13:02:54.978935+0000","last_unstale":"2023-11-28T13:02:56.653728+0000","last_undegraded":"2023-11-28T13:02:56.653728+0000","last_fullsized":"2023-11-28T13:02:56.653728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:18.297439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654399+0000","last_change":"2023-11-28T13:02:54.571685+0000","last_active":"2023-11-28T13:02:56.654399+0000","last_peered":"2023-11-28T13:02:56.654399+0000","last_clean":"2023-11-28T13:02:56.654399+0000","last_became_active":"2023-11-28T13:02:54.571375+0000","last_became_peered":"2023-11-28T13:02:54.571375+0000","last_unstale":"2023-11-28T13:02:56.654399+0000","last_undegraded":"2023-11-28T13:02:56.654399+0000","last_fullsized":"2023-11-28T13:02:56.654399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:45.383406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.552099+0000","last_change":"2023-11-28T13:02:53.552099+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.552099+0000","last_undegraded":"2023-11-28T13:02:53.552099+0000","last_fullsized":"2023-11-28T13:02:53.552099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:24.750211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570836+0000","last_change":"2023-11-28T13:02:54.565731+0000","last_active":"2023-11-28T13:02:54.570836+0000","last_peered":"2023-11-28T13:02:54.570836+0000","last_clean":"2023-11-28T13:02:54.570836+0000","last_became_active":"2023-11-28T13:02:54.565059+0000","last_became_peered":"2023-11-28T13:02:54.565059+0000","last_unstale":"2023-11-28T13:02:54.570836+0000","last_undegraded":"2023-11-28T13:02:54.570836+0000","last_fullsized":"2023-11-28T13:02:54.570836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:36.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570628+0000","last_change":"2023-11-28T13:02:54.565314+0000","last_active":"2023-11-28T13:02:54.570628+0000","last_peered":"2023-11-28T13:02:54.570628+0000","last_clean":"2023-11-28T13:02:54.570628+0000","last_became_active":"2023-11-28T13:02:54.563242+0000","last_became_peered":"2023-11-28T13:02:54.563242+0000","last_unstale":"2023-11-28T13:02:54.570628+0000","last_undegraded":"2023-11-28T13:02:54.570628+0000","last_fullsized":"2023-11-28T13:02:54.570628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:36.187142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-11-28T13:02:53.551604+0000","last_change":"2023-11-28T13:02:53.551604+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.551604+0000","last_undegraded":"2023-11-28T13:02:53.551604+0000","last_fullsized":"2023-11-28T13:02:53.551604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:34:44.950034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:51.618966+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:02:51.618966+0000","last_peered":"2023-11-28T13:02:51.618966+0000","last_clean":"2023-11-28T13:02:51.618966+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:02:51.618966+0000","last_undegraded":"2023-11-28T13:02:51.618966+0000","last_fullsized":"2023-11-28T13:02:51.618966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653875+0000","last_change":"2023-11-28T13:02:54.572150+0000","last_active":"2023-11-28T13:02:56.653875+0000","last_peered":"2023-11-28T13:02:56.653875+0000","last_clean":"2023-11-28T13:02:56.653875+0000","last_became_active":"2023-11-28T13:02:54.571846+0000","last_became_peered":"2023-11-28T13:02:54.571846+0000","last_unstale":"2023-11-28T13:02:56.653875+0000","last_undegraded":"2023-11-28T13:02:56.653875+0000","last_fullsized":"2023-11-28T13:02:56.653875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.733701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436262+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:02:50.436262+0000","last_peered":"2023-11-28T13:02:50.436262+0000","last_clean":"2023-11-28T13:02:50.436262+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:02:50.436262+0000","last_undegraded":"2023-11-28T13:02:50.436262+0000","last_fullsized":"2023-11-28T13:02:50.436262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.654923+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:02:50.654923+0000","last_peered":"2023-11-28T13:02:50.654923+0000","last_clean":"2023-11-28T13:02:50.654923+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:02:50.654923+0000","last_undegraded":"2023-11-28T13:02:50.654923+0000","last_fullsized":"2023-11-28T13:02:50.654923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570580+0000","last_change":"2023-11-28T13:02:54.564196+0000","last_active":"2023-11-28T13:02:54.570580+0000","last_peered":"2023-11-28T13:02:54.570580+0000","last_clean":"2023-11-28T13:02:54.570580+0000","last_became_active":"2023-11-28T13:02:54.563920+0000","last_became_peered":"2023-11-28T13:02:54.563920+0000","last_unstale":"2023-11-28T13:02:54.570580+0000","last_undegraded":"2023-11-28T13:02:54.570580+0000","last_fullsized":"2023-11-28T13:02:54.570580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:57.576385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436267+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:02:50.436267+0000","last_peered":"2023-11-28T13:02:50.436267+0000","last_clean":"2023-11-28T13:02:50.436267+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:02:50.436267+0000","last_undegraded":"2023-11-28T13:02:50.436267+0000","last_fullsized":"2023-11-28T13:02:50.436267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436375+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:02:50.436375+0000","last_peered":"2023-11-28T13:02:50.436375+0000","last_clean":"2023-11-28T13:02:50.436375+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:02:50.436375+0000","last_undegraded":"2023-11-28T13:02:50.436375+0000","last_fullsized":"2023-11-28T13:02:50.436375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653826+0000","last_change":"2023-11-28T13:02:54.974128+0000","last_active":"2023-11-28T13:02:56.653826+0000","last_peered":"2023-11-28T13:02:56.653826+0000","last_clean":"2023-11-28T13:02:56.653826+0000","last_became_active":"2023-11-28T13:02:54.973776+0000","last_became_peered":"2023-11-28T13:02:54.973776+0000","last_unstale":"2023-11-28T13:02:56.653826+0000","last_undegraded":"2023-11-28T13:02:56.653826+0000","last_fullsized":"2023-11-28T13:02:56.653826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:49.099601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653780+0000","last_change":"2023-11-28T13:02:54.564097+0000","last_active":"2023-11-28T13:02:56.653780+0000","last_peered":"2023-11-28T13:02:56.653780+0000","last_clean":"2023-11-28T13:02:56.653780+0000","last_became_active":"2023-11-28T13:02:54.563816+0000","last_became_peered":"2023-11-28T13:02:54.563816+0000","last_unstale":"2023-11-28T13:02:56.653780+0000","last_undegraded":"2023-11-28T13:02:56.653780+0000","last_fullsized":"2023-11-28T13:02:56.653780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:52.277147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436342+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:02:50.436342+0000","last_peered":"2023-11-28T13:02:50.436342+0000","last_clean":"2023-11-28T13:02:50.436342+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:02:50.436342+0000","last_undegraded":"2023-11-28T13:02:50.436342+0000","last_fullsized":"2023-11-28T13:02:50.436342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:50.436334+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:02:50.436334+0000","last_peered":"2023-11-28T13:02:50.436334+0000","last_clean":"2023-11-28T13:02:50.436334+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:02:50.436334+0000","last_undegraded":"2023-11-28T13:02:50.436334+0000","last_fullsized":"2023-11-28T13:02:50.436334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:53.539146+0000","last_change":"2023-11-28T13:02:53.539146+0000","last_active":"2023-11-28T13:02:53.539146+0000","last_peered":"2023-11-28T13:02:53.539146+0000","last_clean":"2023-11-28T13:02:53.539146+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:53.539146+0000","last_undegraded":"2023-11-28T13:02:53.539146+0000","last_fullsized":"2023-11-28T13:02:53.539146+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T13:02:51.618962+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:02:51.618962+0000","last_peered":"2023-11-28T13:02:51.618962+0000","last_clean":"2023-11-28T13:02:51.618962+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:02:51.618962+0000","last_undegraded":"2023-11-28T13:02:51.618962+0000","last_fullsized":"2023-11-28T13:02:51.618962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:02:54.546564+0000","last_change":"2023-11-28T13:02:54.546564+0000","last_active":"2023-11-28T13:02:54.546564+0000","last_peered":"2023-11-28T13:02:54.546564+0000","last_clean":"2023-11-28T13:02:54.546564+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:02:54.546564+0000","last_undegraded":"2023-11-28T13:02:54.546564+0000","last_fullsized":"2023-11-28T13:02:54.546564+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.570476+0000","last_change":"2023-11-28T13:02:54.565033+0000","last_active":"2023-11-28T13:02:54.570476+0000","last_peered":"2023-11-28T13:02:54.570476+0000","last_clean":"2023-11-28T13:02:54.570476+0000","last_became_active":"2023-11-28T13:02:54.564748+0000","last_became_peered":"2023-11-28T13:02:54.564748+0000","last_unstale":"2023-11-28T13:02:54.570476+0000","last_undegraded":"2023-11-28T13:02:54.570476+0000","last_fullsized":"2023-11-28T13:02:54.570476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:58.581060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T13:02:49.647344+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:02:49.647344+0000","last_peered":"2023-11-28T13:02:49.647344+0000","last_clean":"2023-11-28T13:02:49.647344+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:02:49.647344+0000","last_undegraded":"2023-11-28T13:02:49.647344+0000","last_fullsized":"2023-11-28T13:02:49.647344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":34,"acting":34,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9600,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362240,"statfs":{"total":96636764160,"available":96626933760,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.497,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.63,"15min":0.67800000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.498,"5min":8.218,"15min":8.218},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39700000000000002,"15min":0.40400000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.499,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.40400000000000003,"15min":0.41299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.47599999999999998,"5min":1.355,"15min":1.355},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.871,"15min":0.95199999999999996},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55400000000000005,"5min":13.384,"15min":13.384},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.58999999999999997,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52300000000000002,"5min":5.226,"15min":5.226},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.75900000000000001,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79400000000000004,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.80100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.85399999999999998,"15min":0.89700000000000002},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81899999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.79100000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.51300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93799999999999994,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.88300000000000001,"15min":0.92900000000000005},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.78900000000000003,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.85999999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.34100000000000003,"15min":0.33500000000000002},"min":{"1min":0.216,"5min":0.187,"15min":0.187},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.28599999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.436},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53700000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":9332,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362508,"statfs":{"total":96636764160,"available":96627208192,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.496}]},{"osd":1,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.502,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.753,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59799999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14756,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357084,"statfs":{"total":96636764160,"available":96621654016,"internally_reserved":0,"allocated":1339392,"data_stored":917739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.85299999999999998,"15min":0.93999999999999995},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.85499999999999998,"15min":0.93799999999999994},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61399999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.86199999999999999,"15min":0.93899999999999995},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.98799999999999999,"15min":1.091},"min":{"1min":0.247,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58399999999999996,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.69099999999999995,"15min":0.70899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94299999999999995,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.218,"15min":1.3280000000000001},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68999999999999995,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.204,"15min":1.3109999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76100000000000001,"5min":19.02,"15min":19.02},"last":0.443},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.503}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.748,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.83099999999999996,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.2250000000000001,"15min":1.3320000000000001},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.40999999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10488,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361352,"statfs":{"total":96636764160,"available":96626024448,"internally_reserved":0,"allocated":1294336,"data_stored":891937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":1.415,"15min":1.415},"last":0.59099999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":1.867,"15min":1.867},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.54900000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71399999999999997,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2749999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.8440000000000003,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":0.623}]},{"osd":4,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.51200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.528,"15min":1.528},"last":0.505}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.45000000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9332,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362508,"statfs":{"total":96636764160,"available":96627208192,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":1.419,"15min":1.419},"last":0.505},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.41699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.442},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.626}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.40300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79900000000000004,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.68899999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52900000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47899999999999998},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67100000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60599999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73399999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.748,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.70599999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9140,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362700,"statfs":{"total":96636764160,"available":96627404800,"internally_reserved":0,"allocated":700416,"data_stored":301569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.44,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.66200000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.623},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.497}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.441,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55600000000000005}]},{"osd":4,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.5,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58499999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":712704,"data_stored":307739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.20999999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.45900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.33600000000000002,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.441,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52600000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.307},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.373}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54900000000000004,"5min":0.752,"15min":0.752},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.51600000000000001,"5min":0.875,"15min":0.875},"last":0.39600000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":1.044,"15min":1.044},"last":0.44800000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75900000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71299999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68400000000000005,"5min":1.089,"15min":1.089},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72499999999999998,"5min":1.202,"15min":1.202},"last":0.496},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74399999999999999,"5min":1.02,"15min":1.02},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2023-11-28T13:02:59.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2023-11-28T13:02:59.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2023-11-28T13:02:59.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:54.557187+0000', 'last_change': '2023-11-28T13:02:54.557187+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.557187+0000', 'last_undegraded': '2023-11-28T13:02:54.557187+0000', 'last_fullsized': '2023-11-28T13:02:54.557187+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:10:21.818517+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:54.557410+0000', 'last_change': '2023-11-28T13:02:54.557410+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.557410+0000', 'last_undegraded': '2023-11-28T13:02:54.557410+0000', 'last_fullsized': '2023-11-28T13:02:54.557410+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:10:09.427676+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:54.557425+0000', 'last_change': '2023-11-28T13:02:54.557425+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.557425+0000', 'last_undegraded': '2023-11-28T13:02:54.557425+0000', 'last_fullsized': '2023-11-28T13:02:54.557425+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-30T00:11:28.957204+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2023-11-28T13:02:59.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2023-11-28T13:02:59.797 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:53.551962+0000', 'last_change': '2023-11-28T13:02:53.551962+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.551962+0000', 'last_undegraded': '2023-11-28T13:02:53.551962+0000', 'last_fullsized': '2023-11-28T13:02:53.551962+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:45:19.361169+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-11-28T13:02:59.798 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-11-28T13:02:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:54.558233+0000', 'last_change': '2023-11-28T13:02:54.558233+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.558233+0000', 'last_undegraded': '2023-11-28T13:02:54.558233+0000', 'last_fullsized': '2023-11-28T13:02:54.558233+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:23:11.238594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:54.557956+0000', 'last_change': '2023-11-28T13:02:54.557956+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.557956+0000', 'last_undegraded': '2023-11-28T13:02:54.557956+0000', 'last_fullsized': '2023-11-28T13:02:54.557956+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:57:02.456068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T13:02:59.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:53.552504+0000', 'last_change': '2023-11-28T13:02:53.552504+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.552504+0000', 'last_undegraded': '2023-11-28T13:02:53.552504+0000', 'last_fullsized': '2023-11-28T13:02:53.552504+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-30T00:05:54.448798+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T13:02:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-11-28T13:02:59.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:53.552099+0000', 'last_change': '2023-11-28T13:02:53.552099+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.552099+0000', 'last_undegraded': '2023-11-28T13:02:53.552099+0000', 'last_fullsized': '2023-11-28T13:02:53.552099+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:16:24.750211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-11-28T13:02:53.551604+0000', 'last_change': '2023-11-28T13:02:53.551604+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.551604+0000', 'last_undegraded': '2023-11-28T13:02:53.551604+0000', 'last_fullsized': '2023-11-28T13:02:53.551604+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-30T00:34:44.950034+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T13:02:59.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-28T13:02:59.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-11-28T13:02:59.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:53.539146+0000', 'last_change': '2023-11-28T13:02:53.539146+0000', 'last_active': '2023-11-28T13:02:53.539146+0000', 'last_peered': '2023-11-28T13:02:53.539146+0000', 'last_clean': '2023-11-28T13:02:53.539146+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:53.539146+0000', 'last_undegraded': '2023-11-28T13:02:53.539146+0000', 'last_fullsized': '2023-11-28T13:02:53.539146+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:53.539146+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-11-28T13:02:59.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T13:02:54.546564+0000', 'last_change': '2023-11-28T13:02:54.546564+0000', 'last_active': '2023-11-28T13:02:54.546564+0000', 'last_peered': '2023-11-28T13:02:54.546564+0000', 'last_clean': '2023-11-28T13:02:54.546564+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T13:02:54.546564+0000', 'last_undegraded': '2023-11-28T13:02:54.546564+0000', 'last_fullsized': '2023-11-28T13:02:54.546564+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'last_clean_scrub_stamp': '2023-11-28T13:02:54.546564+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T13:02:59.808 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T13:02:59.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.945+0000 7f73b2c53700 1 Processor -- start 2023-11-28T13:02:59.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.945+0000 7f73b2c53700 1 -- start start 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b2c53700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f73ac06dee0 0x7f73ac06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b2c53700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac160540 0x7f73ac1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b2c53700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac15d060 con 0x7f73ac071040 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b2c53700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac15d340 con 0x7f73ac06dee0 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b2c53700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15d620 con 0x7f73ac160540 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73abfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f73ac06dee0 0x7f73ac06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b11f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac160540 0x7f73ac1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b11f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac160540 0x7f73ac1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59868/0 (socket says 172.21.15.153:59868) 2023-11-28T13:02:59.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73abfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f73ac06dee0 0x7f73ac06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:51722/0 (socket says 172.21.15.153:51722) 2023-11-28T13:02:59.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b11f0700 1 -- 172.21.15.153:0/1871533082 learned_addr learned my addr 172.21.15.153:0/1871533082 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:59.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.946+0000 7f73b09ef700 1 --1- 172.21.15.153:0/1871533082 >> v1:172.21.15.177:6789/0 conn(0x7f73ac071040 0x7f73ac071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:37032/0 (socket says 172.21.15.153:37032) 2023-11-28T13:02:59.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73b11f0700 1 -- 172.21.15.153:0/1871533082 >> v2:172.21.15.177:3300/0 conn(0x7f73ac06dee0 msgr2=0x7f73ac06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:59.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73b11f0700 1 --2- 172.21.15.153:0/1871533082 >> v2:172.21.15.177:3300/0 conn(0x7f73ac06dee0 0x7f73ac06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73b11f0700 1 -- 172.21.15.153:0/1871533082 >> v1:172.21.15.177:6789/0 conn(0x7f73ac071040 legacy=0x7f73ac071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73b11f0700 1 -- 172.21.15.153:0/1871533082 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac15d900 con 0x7f73ac160540 2023-11-28T13:02:59.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73abfff700 1 --2- 172.21.15.153:0/1871533082 >> v2:172.21.15.177:3300/0 conn(0x7f73ac06dee0 0x7f73ac06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:59.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73b11f0700 1 --2- 172.21.15.153:0/1871533082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac160540 0x7f73ac1609c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009e50 tx=0x7f73a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6ec648dd7cd8a77 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73ab7fe700 1 -- 172.21.15.153:0/1871533082 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400e070 con 0x7f73ac160540 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73ab7fe700 1 -- 172.21.15.153:0/1871533082 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a400b910 con 0x7f73ac160540 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.947+0000 7f73ab7fe700 1 -- 172.21.15.153:0/1871533082 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400bcc0 con 0x7f73ac160540 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 -- 172.21.15.153:0/1871533082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac160540 msgr2=0x7f73ac1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 --2- 172.21.15.153:0/1871533082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac160540 0x7f73ac1609c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009e50 tx=0x7f73a400b330 comp rx=0 tx=0).stop 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 -- 172.21.15.153:0/1871533082 shutdown_connections 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 --2- 172.21.15.153:0/1871533082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac160540 0x7f73ac1609c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 --2- 172.21.15.153:0/1871533082 >> v2:172.21.15.177:3300/0 conn(0x7f73ac06dee0 0x7f73ac06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 -- 172.21.15.153:0/1871533082 >> 172.21.15.153:0/1871533082 conn(0x7f73ac0d0000 msgr2=0x7f73ac0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 -- 172.21.15.153:0/1871533082 shutdown_connections 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.948+0000 7f73b2c53700 1 -- 172.21.15.153:0/1871533082 wait complete. 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.949+0000 7f73b2c53700 1 Processor -- start 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.949+0000 7f73b2c53700 1 -- start start 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b2c53700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac06dee0 0x7f73ac0c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b2c53700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73ac071040 0x7f73ac0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b2c53700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ac160540 0x7f73ac1f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b2c53700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1f2b60 con 0x7f73ac071040 2023-11-28T13:02:59.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b2c53700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1f64b0 con 0x7f73ac160540 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b2c53700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1f6790 con 0x7f73ac06dee0 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73ac071040 0x7f73ac0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73ac071040 0x7f73ac0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42262/0 (socket says 172.21.15.153:42262) 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 -- 172.21.15.153:0/335990097 learned_addr learned my addr 172.21.15.153:0/335990097 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b09ef700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac06dee0 0x7f73ac0c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b11f0700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ac160540 0x7f73ac1f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 -- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac06dee0 msgr2=0x7f73ac0c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac06dee0 0x7f73ac0c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 -- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ac160540 msgr2=0x7f73ac1f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ac160540 0x7f73ac1f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 -- 172.21.15.153:0/335990097 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980094a0 con 0x7f73ac071040 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b09ef700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac06dee0 0x7f73ac0c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73abfff700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73ac071040 0x7f73ac0b4e90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f739800ebe0 tx=0x7f7398009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73a97fa700 1 -- 172.21.15.153:0/335990097 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800c8b0 con 0x7f73ac071040 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73a97fa700 1 -- 172.21.15.153:0/335990097 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739800ca10 con 0x7f73ac071040 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.950+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a4009b30 con 0x7f73ac071040 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.951+0000 7f73a97fa700 1 -- 172.21.15.153:0/335990097 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800cda0 con 0x7f73ac071040 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.951+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac15ccd0 con 0x7f73ac071040 2023-11-28T13:02:59.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.951+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac06e870 con 0x7f73ac071040 2023-11-28T13:02:59.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.952+0000 7f73a97fa700 1 -- 172.21.15.153:0/335990097 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7398019030 con 0x7f73ac071040 2023-11-28T13:02:59.957 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.952+0000 7f73a97fa700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73900682f0 0x7f739006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.952+0000 7f73a97fa700 1 -- 172.21.15.153:0/335990097 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7398085cd0 con 0x7f73ac071040 2023-11-28T13:02:59.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.952+0000 7f73b09ef700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73900682f0 0x7f739006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.953+0000 7f73b09ef700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73900682f0 0x7f739006a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000f60 tx=0x7f73a0007dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:02:59.956+0000 7f73a97fa700 1 -- 172.21.15.153:0/335990097 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73980524b0 con 0x7f73ac071040 2023-11-28T13:03:00.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.216+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f73ac06a270 con 0x7f73ac071040 2023-11-28T13:03:00.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.216+0000 7f73a97fa700 1 -- 172.21.15.153:0/335990097 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1589 (secure 0 0 0) 0x7f7398055ad0 con 0x7f73ac071040 2023-11-28T13:03:00.217 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:00.218 INFO:teuthology.orchestra.run.smithi153.stdout:{"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":187,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":63,"num_osds":8,"num_up_osds":8,"osd_up_since":1701176403,"num_in_osds":8,"osd_in_since":1701176395,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":69},{"state_name":"active+clean","count":27},{"state_name":"creating+peering","count":9}],"num_pgs":105,"num_pools":4,"num_objects":5,"data_bytes":590368,"bytes_used":83156992,"bytes_avail":773010956288,"bytes_total":773094113280,"unknown_pgs_ratio":0.65714287757873535,"inactive_pgs_ratio":0.085714288055896759,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":31,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":5991}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-28T13:02:44.179451+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T13:03:00.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73900682f0 msgr2=0x7f739006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73900682f0 0x7f739006a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000f60 tx=0x7f73a0007dd0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73ac071040 msgr2=0x7f73ac0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73ac071040 0x7f73ac0b4e90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f739800ebe0 tx=0x7f7398009e20 comp rx=0 tx=0).stop 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 shutdown_connections 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ac160540 0x7f73ac1f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73ac06dee0 0x7f73ac0c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f73900682f0 0x7f739006a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 --2- 172.21.15.153:0/335990097 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73ac071040 0x7f73ac0b4e90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 >> 172.21.15.153:0/335990097 conn(0x7f73ac0d0000 msgr2=0x7f73ac0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 shutdown_connections 2023-11-28T13:03:00.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.218+0000 7f73b2c53700 1 -- 172.21.15.153:0/335990097 wait complete. 2023-11-28T13:03:00.231 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T13:03:00.231 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:03:00.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.352+0000 7f7768055700 1 Processor -- start 2023-11-28T13:03:00.352 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.352+0000 7f7768055700 1 -- start start 2023-11-28T13:03:00.354 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.352+0000 7f7768055700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f77600716d0 0x7f77600cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.354 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7768055700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776015c7d0 0x7f776015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.354 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7768055700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77600c9c60 con 0x7f77600c9730 2023-11-28T13:03:00.354 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7768055700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776015d1b0 con 0x7f77600716d0 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7768055700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77600cb350 con 0x7f776015c7d0 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7765df1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f77600716d0 0x7f77600cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f77665f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776015c7d0 0x7f776015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7765df1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f77600716d0 0x7f77600cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:51734/0 (socket says 172.21.15.153:51734) 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f77665f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776015c7d0 0x7f776015cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:59882/0 (socket says 172.21.15.153:59882) 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f77655f0700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f77600c9730 0x7f776006df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:37046/0 (socket says 172.21.15.153:37046) 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7765df1700 1 -- 172.21.15.153:0/2296918398 learned_addr learned my addr 172.21.15.153:0/2296918398 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f77665f2700 1 -- 172.21.15.153:0/2296918398 >> v2:172.21.15.177:3300/0 conn(0x7f77600716d0 msgr2=0x7f77600cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f77665f2700 1 --2- 172.21.15.153:0/2296918398 >> v2:172.21.15.177:3300/0 conn(0x7f77600716d0 0x7f77600cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f77665f2700 1 -- 172.21.15.153:0/2296918398 >> v1:172.21.15.177:6789/0 conn(0x7f77600c9730 legacy=0x7f776006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f77665f2700 1 -- 172.21.15.153:0/2296918398 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77600cb630 con 0x7f776015c7d0 2023-11-28T13:03:00.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.353+0000 7f7765df1700 1 --2- 172.21.15.153:0/2296918398 >> v2:172.21.15.177:3300/0 conn(0x7f77600716d0 0x7f77600cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:03:00.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f77665f2700 1 --2- 172.21.15.153:0/2296918398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776015c7d0 0x7f776015cc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7758009dc0 tx=0x7f775800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f13d1049fb05bfa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7764def700 1 -- 172.21.15.153:0/2296918398 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800e070 con 0x7f776015c7d0 2023-11-28T13:03:00.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7764def700 1 -- 172.21.15.153:0/2296918398 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775800bae0 con 0x7f776015c7d0 2023-11-28T13:03:00.356 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7764def700 1 -- 172.21.15.153:0/2296918398 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775800be90 con 0x7f776015c7d0 2023-11-28T13:03:00.361 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 -- 172.21.15.153:0/2296918398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776015c7d0 msgr2=0x7f776015cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 --2- 172.21.15.153:0/2296918398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776015c7d0 0x7f776015cc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7758009dc0 tx=0x7f775800b330 comp rx=0 tx=0).stop 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 -- 172.21.15.153:0/2296918398 shutdown_connections 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 --2- 172.21.15.153:0/2296918398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f776015c7d0 0x7f776015cc70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 --2- 172.21.15.153:0/2296918398 >> v2:172.21.15.177:3300/0 conn(0x7f77600716d0 0x7f77600cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 -- 172.21.15.153:0/2296918398 >> 172.21.15.153:0/2296918398 conn(0x7f77600b0c30 msgr2=0x7f77600b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 -- 172.21.15.153:0/2296918398 shutdown_connections 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.354+0000 7f7768055700 1 -- 172.21.15.153:0/2296918398 wait complete. 2023-11-28T13:03:00.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 Processor -- start 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 -- start start 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77600716d0 0x7f77601f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77600c9730 0x7f77600c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f776015c7d0 0x7f77600c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760145f50 con 0x7f776015c7d0 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760145c70 con 0x7f77600716d0 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f7768055700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77600c62e0 con 0x7f77600c9730 2023-11-28T13:03:00.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.356+0000 7f77665f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f776015c7d0 0x7f77600c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f776015c7d0 0x7f77600c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42278/0 (socket says 172.21.15.153:42278) 2023-11-28T13:03:00.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 -- 172.21.15.153:0/387814042 learned_addr learned my addr 172.21.15.153:0/387814042 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:00.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f7765df1700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77600716d0 0x7f77601f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77655f0700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77600c9730 0x7f77600c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 -- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77600c9730 msgr2=0x7f77600c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77600c9730 0x7f77600c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.364 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 -- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77600716d0 msgr2=0x7f77601f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77600716d0 0x7f77601f0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 -- 172.21.15.153:0/387814042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775000a6f0 con 0x7f776015c7d0 2023-11-28T13:03:00.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77665f2700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f776015c7d0 0x7f77600c7840 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7758014040 tx=0x7f7758004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77567fc700 1 -- 172.21.15.153:0/387814042 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758004550 con 0x7f776015c7d0 2023-11-28T13:03:00.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77567fc700 1 -- 172.21.15.153:0/387814042 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7758020070 con 0x7f776015c7d0 2023-11-28T13:03:00.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7758009a70 con 0x7f776015c7d0 2023-11-28T13:03:00.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f77567fc700 1 -- 172.21.15.153:0/387814042 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77580048f0 con 0x7f776015c7d0 2023-11-28T13:03:00.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.357+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77600cbbf0 con 0x7f776015c7d0 2023-11-28T13:03:00.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.358+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77600cb910 con 0x7f776015c7d0 2023-11-28T13:03:00.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.358+0000 7f77567fc700 1 -- 172.21.15.153:0/387814042 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f775802e070 con 0x7f776015c7d0 2023-11-28T13:03:00.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.359+0000 7f77567fc700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f77440682f0 0x7f774406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.359+0000 7f77567fc700 1 -- 172.21.15.153:0/387814042 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7758022070 con 0x7f776015c7d0 2023-11-28T13:03:00.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.359+0000 7f7765df1700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f77440682f0 0x7f774406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.366 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.359+0000 7f7765df1700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f77440682f0 0x7f774406a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7750006d30 tx=0x7f775000a3d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.363+0000 7f77567fc700 1 -- 172.21.15.153:0/387814042 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7758054140 con 0x7f776015c7d0 2023-11-28T13:03:00.537 INFO:tasks.ceph.mds.c.smithi153.stderr:2023-11-28T13:03:00.536+0000 7fe712487700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:03:00.540 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.540+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7760071aa0 con 0x7f77440682f0 2023-11-28T13:03:00.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.548+0000 7f77567fc700 1 -- 172.21.15.153:0/387814042 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311450 (secure 0 0 0) 0x7f7760071aa0 con 0x7f77440682f0 2023-11-28T13:03:00.553 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:00.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f77440682f0 msgr2=0x7f774406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f77440682f0 0x7f774406a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7750006d30 tx=0x7f775000a3d0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f776015c7d0 msgr2=0x7f77600c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f776015c7d0 0x7f77600c7840 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7758014040 tx=0x7f7758004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 shutdown_connections 2023-11-28T13:03:00.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f77440682f0 0x7f774406a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f776015c7d0 0x7f77600c7840 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77600716d0 0x7f77601f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 --2- 172.21.15.153:0/387814042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77600c9730 0x7f77600c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 >> 172.21.15.153:0/387814042 conn(0x7f77600b0c30 msgr2=0x7f7760156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 shutdown_connections 2023-11-28T13:03:00.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:00.552+0000 7f7768055700 1 -- 172.21.15.153:0/387814042 wait complete. 2023-11-28T13:03:00.556 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:03:00.565 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-11-28T13:03:00.182558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82660,"kb_used_data":7076,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":754892060,"statfs":{"total":773094113280,"available":773009469440,"internally_reserved":0,"allocated":7245824,"data_stored":3809747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001251"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653834+0000","last_change":"2023-11-28T13:02:54.967102+0000","last_active":"2023-11-28T13:02:56.653834+0000","last_peered":"2023-11-28T13:02:56.653834+0000","last_clean":"2023-11-28T13:02:56.653834+0000","last_became_active":"2023-11-28T13:02:54.966804+0000","last_became_peered":"2023-11-28T13:02:54.966804+0000","last_unstale":"2023-11-28T13:02:56.653834+0000","last_undegraded":"2023-11-28T13:02:56.653834+0000","last_fullsized":"2023-11-28T13:02:56.653834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:25.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654114+0000","last_change":"2023-11-28T13:02:54.971255+0000","last_active":"2023-11-28T13:02:56.654114+0000","last_peered":"2023-11-28T13:02:56.654114+0000","last_clean":"2023-11-28T13:02:56.654114+0000","last_became_active":"2023-11-28T13:02:54.970382+0000","last_became_peered":"2023-11-28T13:02:54.970382+0000","last_unstale":"2023-11-28T13:02:56.654114+0000","last_undegraded":"2023-11-28T13:02:56.654114+0000","last_fullsized":"2023-11-28T13:02:56.654114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:44.841594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654838+0000","last_change":"2023-11-28T13:02:54.986116+0000","last_active":"2023-11-28T13:02:56.654838+0000","last_peered":"2023-11-28T13:02:56.654838+0000","last_clean":"2023-11-28T13:02:56.654838+0000","last_became_active":"2023-11-28T13:02:54.985837+0000","last_became_peered":"2023-11-28T13:02:54.985837+0000","last_unstale":"2023-11-28T13:02:56.654838+0000","last_undegraded":"2023-11-28T13:02:56.654838+0000","last_fullsized":"2023-11-28T13:02:56.654838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:16.518326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654572+0000","last_change":"2023-11-28T13:02:54.982949+0000","last_active":"2023-11-28T13:02:56.654572+0000","last_peered":"2023-11-28T13:02:56.654572+0000","last_clean":"2023-11-28T13:02:56.654572+0000","last_became_active":"2023-11-28T13:02:54.982553+0000","last_became_peered":"2023-11-28T13:02:54.982553+0000","last_unstale":"2023-11-28T13:02:56.654572+0000","last_undegraded":"2023-11-28T13:02:56.654572+0000","last_fullsized":"2023-11-28T13:02:56.654572+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:39.068694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969244+0000","last_change":"2023-11-28T13:02:54.975631+0000","last_active":"2023-11-28T13:02:55.969244+0000","last_peered":"2023-11-28T13:02:55.969244+0000","last_clean":"2023-11-28T13:02:55.969244+0000","last_became_active":"2023-11-28T13:02:54.975355+0000","last_became_peered":"2023-11-28T13:02:54.975355+0000","last_unstale":"2023-11-28T13:02:55.969244+0000","last_undegraded":"2023-11-28T13:02:55.969244+0000","last_fullsized":"2023-11-28T13:02:55.969244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:51.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654061+0000","last_change":"2023-11-28T13:02:54.984256+0000","last_active":"2023-11-28T13:02:56.654061+0000","last_peered":"2023-11-28T13:02:56.654061+0000","last_clean":"2023-11-28T13:02:56.654061+0000","last_became_active":"2023-11-28T13:02:54.984012+0000","last_became_peered":"2023-11-28T13:02:54.984012+0000","last_unstale":"2023-11-28T13:02:56.654061+0000","last_undegraded":"2023-11-28T13:02:56.654061+0000","last_fullsized":"2023-11-28T13:02:56.654061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:58.068458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152743+0000","last_change":"2023-11-28T13:02:54.977048+0000","last_active":"2023-11-28T13:02:56.152743+0000","last_peered":"2023-11-28T13:02:56.152743+0000","last_clean":"2023-11-28T13:02:56.152743+0000","last_became_active":"2023-11-28T13:02:54.976778+0000","last_became_peered":"2023-11-28T13:02:54.976778+0000","last_unstale":"2023-11-28T13:02:56.152743+0000","last_undegraded":"2023-11-28T13:02:56.152743+0000","last_fullsized":"2023-11-28T13:02:56.152743+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:52.444793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968064+0000","last_change":"2023-11-28T13:02:54.980339+0000","last_active":"2023-11-28T13:02:55.968064+0000","last_peered":"2023-11-28T13:02:55.968064+0000","last_clean":"2023-11-28T13:02:55.968064+0000","last_became_active":"2023-11-28T13:02:54.980058+0000","last_became_peered":"2023-11-28T13:02:54.980058+0000","last_unstale":"2023-11-28T13:02:55.968064+0000","last_undegraded":"2023-11-28T13:02:55.968064+0000","last_fullsized":"2023-11-28T13:02:55.968064+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:36.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654535+0000","last_change":"2023-11-28T13:02:54.988713+0000","last_active":"2023-11-28T13:02:56.654535+0000","last_peered":"2023-11-28T13:02:56.654535+0000","last_clean":"2023-11-28T13:02:56.654535+0000","last_became_active":"2023-11-28T13:02:54.988216+0000","last_became_peered":"2023-11-28T13:02:54.988216+0000","last_unstale":"2023-11-28T13:02:56.654535+0000","last_undegraded":"2023-11-28T13:02:56.654535+0000","last_fullsized":"2023-11-28T13:02:56.654535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:46.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654142+0000","last_change":"2023-11-28T13:02:54.987462+0000","last_active":"2023-11-28T13:02:56.654142+0000","last_peered":"2023-11-28T13:02:56.654142+0000","last_clean":"2023-11-28T13:02:56.654142+0000","last_became_active":"2023-11-28T13:02:54.987168+0000","last_became_peered":"2023-11-28T13:02:54.987168+0000","last_unstale":"2023-11-28T13:02:56.654142+0000","last_undegraded":"2023-11-28T13:02:56.654142+0000","last_fullsized":"2023-11-28T13:02:56.654142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:36.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654594+0000","last_change":"2023-11-28T13:02:54.974431+0000","last_active":"2023-11-28T13:02:56.654594+0000","last_peered":"2023-11-28T13:02:56.654594+0000","last_clean":"2023-11-28T13:02:56.654594+0000","last_became_active":"2023-11-28T13:02:54.974076+0000","last_became_peered":"2023-11-28T13:02:54.974076+0000","last_unstale":"2023-11-28T13:02:56.654594+0000","last_undegraded":"2023-11-28T13:02:56.654594+0000","last_fullsized":"2023-11-28T13:02:56.654594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:23.744383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968373+0000","last_change":"2023-11-28T13:02:54.977275+0000","last_active":"2023-11-28T13:02:55.968373+0000","last_peered":"2023-11-28T13:02:55.968373+0000","last_clean":"2023-11-28T13:02:55.968373+0000","last_became_active":"2023-11-28T13:02:54.976935+0000","last_became_peered":"2023-11-28T13:02:54.976935+0000","last_unstale":"2023-11-28T13:02:55.968373+0000","last_undegraded":"2023-11-28T13:02:55.968373+0000","last_fullsized":"2023-11-28T13:02:55.968373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:51.741120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968480+0000","last_change":"2023-11-28T13:02:54.974143+0000","last_active":"2023-11-28T13:02:55.968480+0000","last_peered":"2023-11-28T13:02:55.968480+0000","last_clean":"2023-11-28T13:02:55.968480+0000","last_became_active":"2023-11-28T13:02:54.973897+0000","last_became_peered":"2023-11-28T13:02:54.973897+0000","last_unstale":"2023-11-28T13:02:55.968480+0000","last_undegraded":"2023-11-28T13:02:55.968480+0000","last_fullsized":"2023-11-28T13:02:55.968480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:11.038463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654865+0000","last_change":"2023-11-28T13:02:54.986158+0000","last_active":"2023-11-28T13:02:56.654865+0000","last_peered":"2023-11-28T13:02:56.654865+0000","last_clean":"2023-11-28T13:02:56.654865+0000","last_became_active":"2023-11-28T13:02:54.985916+0000","last_became_peered":"2023-11-28T13:02:54.985916+0000","last_unstale":"2023-11-28T13:02:56.654865+0000","last_undegraded":"2023-11-28T13:02:56.654865+0000","last_fullsized":"2023-11-28T13:02:56.654865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:27:44.453808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654732+0000","last_change":"2023-11-28T13:02:54.984129+0000","last_active":"2023-11-28T13:02:56.654732+0000","last_peered":"2023-11-28T13:02:56.654732+0000","last_clean":"2023-11-28T13:02:56.654732+0000","last_became_active":"2023-11-28T13:02:54.983853+0000","last_became_peered":"2023-11-28T13:02:54.983853+0000","last_unstale":"2023-11-28T13:02:56.654732+0000","last_undegraded":"2023-11-28T13:02:56.654732+0000","last_fullsized":"2023-11-28T13:02:56.654732+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.723540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152703+0000","last_change":"2023-11-28T13:02:54.970706+0000","last_active":"2023-11-28T13:02:56.152703+0000","last_peered":"2023-11-28T13:02:56.152703+0000","last_clean":"2023-11-28T13:02:56.152703+0000","last_became_active":"2023-11-28T13:02:54.970422+0000","last_became_peered":"2023-11-28T13:02:54.970422+0000","last_unstale":"2023-11-28T13:02:56.152703+0000","last_undegraded":"2023-11-28T13:02:56.152703+0000","last_fullsized":"2023-11-28T13:02:56.152703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:37.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653973+0000","last_change":"2023-11-28T13:02:54.977040+0000","last_active":"2023-11-28T13:02:56.653973+0000","last_peered":"2023-11-28T13:02:56.653973+0000","last_clean":"2023-11-28T13:02:56.653973+0000","last_became_active":"2023-11-28T13:02:54.976803+0000","last_became_peered":"2023-11-28T13:02:54.976803+0000","last_unstale":"2023-11-28T13:02:56.653973+0000","last_undegraded":"2023-11-28T13:02:56.653973+0000","last_fullsized":"2023-11-28T13:02:56.653973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:43:34.185333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654057+0000","last_change":"2023-11-28T13:02:54.979526+0000","last_active":"2023-11-28T13:02:56.654057+0000","last_peered":"2023-11-28T13:02:56.654057+0000","last_clean":"2023-11-28T13:02:56.654057+0000","last_became_active":"2023-11-28T13:02:54.979273+0000","last_became_peered":"2023-11-28T13:02:54.979273+0000","last_unstale":"2023-11-28T13:02:56.654057+0000","last_undegraded":"2023-11-28T13:02:56.654057+0000","last_fullsized":"2023-11-28T13:02:56.654057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:51.025655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969364+0000","last_change":"2023-11-28T13:02:54.974515+0000","last_active":"2023-11-28T13:02:55.969364+0000","last_peered":"2023-11-28T13:02:55.969364+0000","last_clean":"2023-11-28T13:02:55.969364+0000","last_became_active":"2023-11-28T13:02:54.974230+0000","last_became_peered":"2023-11-28T13:02:54.974230+0000","last_unstale":"2023-11-28T13:02:55.969364+0000","last_undegraded":"2023-11-28T13:02:55.969364+0000","last_fullsized":"2023-11-28T13:02:55.969364+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:01.077047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.976273+0000","last_change":"2023-11-28T13:02:54.970059+0000","last_active":"2023-11-28T13:02:54.976273+0000","last_peered":"2023-11-28T13:02:54.976273+0000","last_clean":"2023-11-28T13:02:54.976273+0000","last_became_active":"2023-11-28T13:02:54.969802+0000","last_became_peered":"2023-11-28T13:02:54.969802+0000","last_unstale":"2023-11-28T13:02:54.976273+0000","last_undegraded":"2023-11-28T13:02:54.976273+0000","last_fullsized":"2023-11-28T13:02:54.976273+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:21.818517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968668+0000","last_change":"2023-11-28T13:02:54.964650+0000","last_active":"2023-11-28T13:02:55.968668+0000","last_peered":"2023-11-28T13:02:55.968668+0000","last_clean":"2023-11-28T13:02:55.968668+0000","last_became_active":"2023-11-28T13:02:54.964340+0000","last_became_peered":"2023-11-28T13:02:54.964340+0000","last_unstale":"2023-11-28T13:02:55.968668+0000","last_undegraded":"2023-11-28T13:02:55.968668+0000","last_fullsized":"2023-11-28T13:02:55.968668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:16.518326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.984525+0000","last_change":"2023-11-28T13:02:54.983831+0000","last_active":"2023-11-28T13:02:54.984525+0000","last_peered":"2023-11-28T13:02:54.984525+0000","last_clean":"2023-11-28T13:02:54.984525+0000","last_became_active":"2023-11-28T13:02:54.983570+0000","last_became_peered":"2023-11-28T13:02:54.983570+0000","last_unstale":"2023-11-28T13:02:54.984525+0000","last_undegraded":"2023-11-28T13:02:54.984525+0000","last_fullsized":"2023-11-28T13:02:54.984525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:09.427676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.978254+0000","last_change":"2023-11-28T13:02:54.977435+0000","last_active":"2023-11-28T13:02:54.978254+0000","last_peered":"2023-11-28T13:02:54.978254+0000","last_clean":"2023-11-28T13:02:54.978254+0000","last_became_active":"2023-11-28T13:02:54.977195+0000","last_became_peered":"2023-11-28T13:02:54.977195+0000","last_unstale":"2023-11-28T13:02:54.978254+0000","last_undegraded":"2023-11-28T13:02:54.978254+0000","last_fullsized":"2023-11-28T13:02:54.978254+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:28.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654013+0000","last_change":"2023-11-28T13:02:54.984296+0000","last_active":"2023-11-28T13:02:56.654013+0000","last_peered":"2023-11-28T13:02:56.654013+0000","last_clean":"2023-11-28T13:02:56.654013+0000","last_became_active":"2023-11-28T13:02:54.984032+0000","last_became_peered":"2023-11-28T13:02:54.984032+0000","last_unstale":"2023-11-28T13:02:56.654013+0000","last_undegraded":"2023-11-28T13:02:56.654013+0000","last_fullsized":"2023-11-28T13:02:56.654013+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:37.697972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968528+0000","last_change":"2023-11-28T13:02:54.969618+0000","last_active":"2023-11-28T13:02:55.968528+0000","last_peered":"2023-11-28T13:02:55.968528+0000","last_clean":"2023-11-28T13:02:55.968528+0000","last_became_active":"2023-11-28T13:02:54.969252+0000","last_became_peered":"2023-11-28T13:02:54.969252+0000","last_unstale":"2023-11-28T13:02:55.968528+0000","last_undegraded":"2023-11-28T13:02:55.968528+0000","last_fullsized":"2023-11-28T13:02:55.968528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:37.697972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968474+0000","last_change":"2023-11-28T13:02:54.969033+0000","last_active":"2023-11-28T13:02:55.968474+0000","last_peered":"2023-11-28T13:02:55.968474+0000","last_clean":"2023-11-28T13:02:55.968474+0000","last_became_active":"2023-11-28T13:02:54.967766+0000","last_became_peered":"2023-11-28T13:02:54.967766+0000","last_unstale":"2023-11-28T13:02:55.968474+0000","last_undegraded":"2023-11-28T13:02:55.968474+0000","last_fullsized":"2023-11-28T13:02:55.968474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:46.754083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654277+0000","last_change":"2023-11-28T13:02:54.976309+0000","last_active":"2023-11-28T13:02:56.654277+0000","last_peered":"2023-11-28T13:02:56.654277+0000","last_clean":"2023-11-28T13:02:56.654277+0000","last_became_active":"2023-11-28T13:02:54.975431+0000","last_became_peered":"2023-11-28T13:02:54.975431+0000","last_unstale":"2023-11-28T13:02:56.654277+0000","last_undegraded":"2023-11-28T13:02:56.654277+0000","last_fullsized":"2023-11-28T13:02:56.654277+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:38.197496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654243+0000","last_change":"2023-11-28T13:02:54.972357+0000","last_active":"2023-11-28T13:02:56.654243+0000","last_peered":"2023-11-28T13:02:56.654243+0000","last_clean":"2023-11-28T13:02:56.654243+0000","last_became_active":"2023-11-28T13:02:54.972103+0000","last_became_peered":"2023-11-28T13:02:54.972103+0000","last_unstale":"2023-11-28T13:02:56.654243+0000","last_undegraded":"2023-11-28T13:02:56.654243+0000","last_fullsized":"2023-11-28T13:02:56.654243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:52:44.285754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654212+0000","last_change":"2023-11-28T13:02:54.988699+0000","last_active":"2023-11-28T13:02:56.654212+0000","last_peered":"2023-11-28T13:02:56.654212+0000","last_clean":"2023-11-28T13:02:56.654212+0000","last_became_active":"2023-11-28T13:02:54.988399+0000","last_became_peered":"2023-11-28T13:02:54.988399+0000","last_unstale":"2023-11-28T13:02:56.654212+0000","last_undegraded":"2023-11-28T13:02:56.654212+0000","last_fullsized":"2023-11-28T13:02:56.654212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:46.754083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968182+0000","last_change":"2023-11-28T13:02:54.984161+0000","last_active":"2023-11-28T13:02:55.968182+0000","last_peered":"2023-11-28T13:02:55.968182+0000","last_clean":"2023-11-28T13:02:55.968182+0000","last_became_active":"2023-11-28T13:02:54.983896+0000","last_became_peered":"2023-11-28T13:02:54.983896+0000","last_unstale":"2023-11-28T13:02:55.968182+0000","last_undegraded":"2023-11-28T13:02:55.968182+0000","last_fullsized":"2023-11-28T13:02:55.968182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:53.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654016+0000","last_change":"2023-11-28T13:02:54.975745+0000","last_active":"2023-11-28T13:02:56.654016+0000","last_peered":"2023-11-28T13:02:56.654016+0000","last_clean":"2023-11-28T13:02:56.654016+0000","last_became_active":"2023-11-28T13:02:54.975472+0000","last_became_peered":"2023-11-28T13:02:54.975472+0000","last_unstale":"2023-11-28T13:02:56.654016+0000","last_undegraded":"2023-11-28T13:02:56.654016+0000","last_fullsized":"2023-11-28T13:02:56.654016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:05.196494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653599+0000","last_change":"2023-11-28T13:02:54.973929+0000","last_active":"2023-11-28T13:02:56.653599+0000","last_peered":"2023-11-28T13:02:56.653599+0000","last_clean":"2023-11-28T13:02:56.653599+0000","last_became_active":"2023-11-28T13:02:54.973639+0000","last_became_peered":"2023-11-28T13:02:54.973639+0000","last_unstale":"2023-11-28T13:02:56.653599+0000","last_undegraded":"2023-11-28T13:02:56.653599+0000","last_fullsized":"2023-11-28T13:02:56.653599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:28.472469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969354+0000","last_change":"2023-11-28T13:02:54.979291+0000","last_active":"2023-11-28T13:02:55.969354+0000","last_peered":"2023-11-28T13:02:55.969354+0000","last_clean":"2023-11-28T13:02:55.969354+0000","last_became_active":"2023-11-28T13:02:54.977003+0000","last_became_peered":"2023-11-28T13:02:54.977003+0000","last_unstale":"2023-11-28T13:02:55.969354+0000","last_undegraded":"2023-11-28T13:02:55.969354+0000","last_fullsized":"2023-11-28T13:02:55.969354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:40.499752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.974611+0000","last_change":"2023-11-28T13:02:54.574619+0000","last_active":"2023-11-28T13:02:54.974611+0000","last_peered":"2023-11-28T13:02:54.974611+0000","last_clean":"2023-11-28T13:02:54.974611+0000","last_became_active":"2023-11-28T13:02:54.574236+0000","last_became_peered":"2023-11-28T13:02:54.574236+0000","last_unstale":"2023-11-28T13:02:54.974611+0000","last_undegraded":"2023-11-28T13:02:54.974611+0000","last_fullsized":"2023-11-28T13:02:54.974611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654394+0000","last_change":"2023-11-28T13:02:54.979052+0000","last_active":"2023-11-28T13:02:56.654394+0000","last_peered":"2023-11-28T13:02:56.654394+0000","last_clean":"2023-11-28T13:02:56.654394+0000","last_became_active":"2023-11-28T13:02:54.978662+0000","last_became_peered":"2023-11-28T13:02:54.978662+0000","last_unstale":"2023-11-28T13:02:56.654394+0000","last_undegraded":"2023-11-28T13:02:56.654394+0000","last_fullsized":"2023-11-28T13:02:56.654394+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:03.502160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654252+0000","last_change":"2023-11-28T13:02:54.576056+0000","last_active":"2023-11-28T13:02:56.654252+0000","last_peered":"2023-11-28T13:02:56.654252+0000","last_clean":"2023-11-28T13:02:56.654252+0000","last_became_active":"2023-11-28T13:02:54.575718+0000","last_became_peered":"2023-11-28T13:02:54.575718+0000","last_unstale":"2023-11-28T13:02:56.654252+0000","last_undegraded":"2023-11-28T13:02:56.654252+0000","last_fullsized":"2023-11-28T13:02:56.654252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:27.738591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654159+0000","last_change":"2023-11-28T13:02:54.969947+0000","last_active":"2023-11-28T13:02:56.654159+0000","last_peered":"2023-11-28T13:02:56.654159+0000","last_clean":"2023-11-28T13:02:56.654159+0000","last_became_active":"2023-11-28T13:02:54.969698+0000","last_became_peered":"2023-11-28T13:02:54.969698+0000","last_unstale":"2023-11-28T13:02:56.654159+0000","last_undegraded":"2023-11-28T13:02:56.654159+0000","last_fullsized":"2023-11-28T13:02:56.654159+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:05.666567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654306+0000","last_change":"2023-11-28T13:02:54.572156+0000","last_active":"2023-11-28T13:02:56.654306+0000","last_peered":"2023-11-28T13:02:56.654306+0000","last_clean":"2023-11-28T13:02:56.654306+0000","last_became_active":"2023-11-28T13:02:54.571876+0000","last_became_peered":"2023-11-28T13:02:54.571876+0000","last_unstale":"2023-11-28T13:02:56.654306+0000","last_undegraded":"2023-11-28T13:02:56.654306+0000","last_fullsized":"2023-11-28T13:02:56.654306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:50.653241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654338+0000","last_change":"2023-11-28T13:02:54.987454+0000","last_active":"2023-11-28T13:02:56.654338+0000","last_peered":"2023-11-28T13:02:56.654338+0000","last_clean":"2023-11-28T13:02:56.654338+0000","last_became_active":"2023-11-28T13:02:54.987193+0000","last_became_peered":"2023-11-28T13:02:54.987193+0000","last_unstale":"2023-11-28T13:02:56.654338+0000","last_undegraded":"2023-11-28T13:02:56.654338+0000","last_fullsized":"2023-11-28T13:02:56.654338+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:51.741120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653781+0000","last_change":"2023-11-28T13:02:54.564777+0000","last_active":"2023-11-28T13:02:56.653781+0000","last_peered":"2023-11-28T13:02:56.653781+0000","last_clean":"2023-11-28T13:02:56.653781+0000","last_became_active":"2023-11-28T13:02:54.564503+0000","last_became_peered":"2023-11-28T13:02:54.564503+0000","last_unstale":"2023-11-28T13:02:56.653781+0000","last_undegraded":"2023-11-28T13:02:56.653781+0000","last_fullsized":"2023-11-28T13:02:56.653781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:39:38.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152777+0000","last_change":"2023-11-28T13:02:54.967759+0000","last_active":"2023-11-28T13:02:56.152777+0000","last_peered":"2023-11-28T13:02:56.152777+0000","last_clean":"2023-11-28T13:02:56.152777+0000","last_became_active":"2023-11-28T13:02:54.967420+0000","last_became_peered":"2023-11-28T13:02:54.967420+0000","last_unstale":"2023-11-28T13:02:56.152777+0000","last_undegraded":"2023-11-28T13:02:56.152777+0000","last_fullsized":"2023-11-28T13:02:56.152777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:19.756048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.968840+0000","last_change":"2023-11-28T13:02:54.565557+0000","last_active":"2023-11-28T13:02:54.968840+0000","last_peered":"2023-11-28T13:02:54.968840+0000","last_clean":"2023-11-28T13:02:54.968840+0000","last_became_active":"2023-11-28T13:02:54.564651+0000","last_became_peered":"2023-11-28T13:02:54.564651+0000","last_unstale":"2023-11-28T13:02:54.968840+0000","last_undegraded":"2023-11-28T13:02:54.968840+0000","last_fullsized":"2023-11-28T13:02:54.968840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:36.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654121+0000","last_change":"2023-11-28T13:02:54.975582+0000","last_active":"2023-11-28T13:02:56.654121+0000","last_peered":"2023-11-28T13:02:56.654121+0000","last_clean":"2023-11-28T13:02:56.654121+0000","last_became_active":"2023-11-28T13:02:54.975328+0000","last_became_peered":"2023-11-28T13:02:54.975328+0000","last_unstale":"2023-11-28T13:02:56.654121+0000","last_undegraded":"2023-11-28T13:02:56.654121+0000","last_fullsized":"2023-11-28T13:02:56.654121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:36.435453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.968695+0000","last_change":"2023-11-28T13:02:54.565910+0000","last_active":"2023-11-28T13:02:54.968695+0000","last_peered":"2023-11-28T13:02:54.968695+0000","last_clean":"2023-11-28T13:02:54.968695+0000","last_became_active":"2023-11-28T13:02:54.565445+0000","last_became_peered":"2023-11-28T13:02:54.565445+0000","last_unstale":"2023-11-28T13:02:54.968695+0000","last_undegraded":"2023-11-28T13:02:54.968695+0000","last_fullsized":"2023-11-28T13:02:54.968695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.321903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968974+0000","last_change":"2023-11-28T13:02:54.979001+0000","last_active":"2023-11-28T13:02:55.968974+0000","last_peered":"2023-11-28T13:02:55.968974+0000","last_clean":"2023-11-28T13:02:55.968974+0000","last_became_active":"2023-11-28T13:02:54.978737+0000","last_became_peered":"2023-11-28T13:02:54.978737+0000","last_unstale":"2023-11-28T13:02:55.968974+0000","last_undegraded":"2023-11-28T13:02:55.968974+0000","last_fullsized":"2023-11-28T13:02:55.968974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.589733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965122+0000","last_change":"2023-11-28T13:02:54.563263+0000","last_active":"2023-11-28T13:02:54.965122+0000","last_peered":"2023-11-28T13:02:54.965122+0000","last_clean":"2023-11-28T13:02:54.965122+0000","last_became_active":"2023-11-28T13:02:54.562391+0000","last_became_peered":"2023-11-28T13:02:54.562391+0000","last_unstale":"2023-11-28T13:02:54.965122+0000","last_undegraded":"2023-11-28T13:02:54.965122+0000","last_fullsized":"2023-11-28T13:02:54.965122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:55.820672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.976657+0000","last_change":"2023-11-28T13:02:54.975878+0000","last_active":"2023-11-28T13:02:54.976657+0000","last_peered":"2023-11-28T13:02:54.976657+0000","last_clean":"2023-11-28T13:02:54.976657+0000","last_became_active":"2023-11-28T13:02:54.975567+0000","last_became_peered":"2023-11-28T13:02:54.975567+0000","last_unstale":"2023-11-28T13:02:54.976657+0000","last_undegraded":"2023-11-28T13:02:54.976657+0000","last_fullsized":"2023-11-28T13:02:54.976657+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:11.238594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968897+0000","last_change":"2023-11-28T13:02:54.574400+0000","last_active":"2023-11-28T13:02:55.968897+0000","last_peered":"2023-11-28T13:02:55.968897+0000","last_clean":"2023-11-28T13:02:55.968897+0000","last_became_active":"2023-11-28T13:02:54.574064+0000","last_became_peered":"2023-11-28T13:02:54.574064+0000","last_unstale":"2023-11-28T13:02:55.968897+0000","last_undegraded":"2023-11-28T13:02:55.968897+0000","last_fullsized":"2023-11-28T13:02:55.968897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:33.186957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654415+0000","last_change":"2023-11-28T13:02:54.972511+0000","last_active":"2023-11-28T13:02:56.654415+0000","last_peered":"2023-11-28T13:02:56.654415+0000","last_clean":"2023-11-28T13:02:56.654415+0000","last_became_active":"2023-11-28T13:02:54.972263+0000","last_became_peered":"2023-11-28T13:02:54.972263+0000","last_unstale":"2023-11-28T13:02:56.654415+0000","last_undegraded":"2023-11-28T13:02:56.654415+0000","last_fullsized":"2023-11-28T13:02:56.654415+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:36.364613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654429+0000","last_change":"2023-11-28T13:02:54.574976+0000","last_active":"2023-11-28T13:02:56.654429+0000","last_peered":"2023-11-28T13:02:56.654429+0000","last_clean":"2023-11-28T13:02:56.654429+0000","last_became_active":"2023-11-28T13:02:54.574585+0000","last_became_peered":"2023-11-28T13:02:54.574585+0000","last_unstale":"2023-11-28T13:02:56.654429+0000","last_undegraded":"2023-11-28T13:02:56.654429+0000","last_fullsized":"2023-11-28T13:02:56.654429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:14.566824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.976647+0000","last_change":"2023-11-28T13:02:54.973231+0000","last_active":"2023-11-28T13:02:54.976647+0000","last_peered":"2023-11-28T13:02:54.976647+0000","last_clean":"2023-11-28T13:02:54.976647+0000","last_became_active":"2023-11-28T13:02:54.972974+0000","last_became_peered":"2023-11-28T13:02:54.972974+0000","last_unstale":"2023-11-28T13:02:54.976647+0000","last_undegraded":"2023-11-28T13:02:54.976647+0000","last_fullsized":"2023-11-28T13:02:54.976647+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:02.456068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654772+0000","last_change":"2023-11-28T13:02:54.569392+0000","last_active":"2023-11-28T13:02:56.654772+0000","last_peered":"2023-11-28T13:02:56.654772+0000","last_clean":"2023-11-28T13:02:56.654772+0000","last_became_active":"2023-11-28T13:02:54.569061+0000","last_became_peered":"2023-11-28T13:02:54.569061+0000","last_unstale":"2023-11-28T13:02:56.654772+0000","last_undegraded":"2023-11-28T13:02:56.654772+0000","last_fullsized":"2023-11-28T13:02:56.654772+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:07.582314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969231+0000","last_change":"2023-11-28T13:02:54.973297+0000","last_active":"2023-11-28T13:02:55.969231+0000","last_peered":"2023-11-28T13:02:55.969231+0000","last_clean":"2023-11-28T13:02:55.969231+0000","last_became_active":"2023-11-28T13:02:54.973015+0000","last_became_peered":"2023-11-28T13:02:54.973015+0000","last_unstale":"2023-11-28T13:02:55.969231+0000","last_undegraded":"2023-11-28T13:02:55.969231+0000","last_fullsized":"2023-11-28T13:02:55.969231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:15.574242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.655005+0000","last_change":"2023-11-28T13:02:54.575482+0000","last_active":"2023-11-28T13:02:56.655005+0000","last_peered":"2023-11-28T13:02:56.655005+0000","last_clean":"2023-11-28T13:02:56.655005+0000","last_became_active":"2023-11-28T13:02:54.575196+0000","last_became_peered":"2023-11-28T13:02:54.575196+0000","last_unstale":"2023-11-28T13:02:56.655005+0000","last_undegraded":"2023-11-28T13:02:56.655005+0000","last_fullsized":"2023-11-28T13:02:56.655005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:13.427513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.655078+0000","last_change":"2023-11-28T13:02:54.976848+0000","last_active":"2023-11-28T13:02:56.655078+0000","last_peered":"2023-11-28T13:02:56.655078+0000","last_clean":"2023-11-28T13:02:56.655078+0000","last_became_active":"2023-11-28T13:02:54.976192+0000","last_became_peered":"2023-11-28T13:02:54.976192+0000","last_unstale":"2023-11-28T13:02:56.655078+0000","last_undegraded":"2023-11-28T13:02:56.655078+0000","last_fullsized":"2023-11-28T13:02:56.655078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:05.211009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.983194+0000","last_change":"2023-11-28T13:02:54.571891+0000","last_active":"2023-11-28T13:02:54.983194+0000","last_peered":"2023-11-28T13:02:54.983194+0000","last_clean":"2023-11-28T13:02:54.983194+0000","last_became_active":"2023-11-28T13:02:54.571427+0000","last_became_peered":"2023-11-28T13:02:54.571427+0000","last_unstale":"2023-11-28T13:02:54.983194+0000","last_undegraded":"2023-11-28T13:02:54.983194+0000","last_fullsized":"2023-11-28T13:02:54.983194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:54.448798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653846+0000","last_change":"2023-11-28T13:02:54.980145+0000","last_active":"2023-11-28T13:02:56.653846+0000","last_peered":"2023-11-28T13:02:56.653846+0000","last_clean":"2023-11-28T13:02:56.653846+0000","last_became_active":"2023-11-28T13:02:54.979795+0000","last_became_peered":"2023-11-28T13:02:54.979795+0000","last_unstale":"2023-11-28T13:02:56.653846+0000","last_undegraded":"2023-11-28T13:02:56.653846+0000","last_fullsized":"2023-11-28T13:02:56.653846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:46.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654319+0000","last_change":"2023-11-28T13:02:54.573939+0000","last_active":"2023-11-28T13:02:56.654319+0000","last_peered":"2023-11-28T13:02:56.654319+0000","last_clean":"2023-11-28T13:02:56.654319+0000","last_became_active":"2023-11-28T13:02:54.573656+0000","last_became_peered":"2023-11-28T13:02:54.573656+0000","last_unstale":"2023-11-28T13:02:56.654319+0000","last_undegraded":"2023-11-28T13:02:56.654319+0000","last_fullsized":"2023-11-28T13:02:56.654319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:34:26.543872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968064+0000","last_change":"2023-11-28T13:02:54.966758+0000","last_active":"2023-11-28T13:02:55.968064+0000","last_peered":"2023-11-28T13:02:55.968064+0000","last_clean":"2023-11-28T13:02:55.968064+0000","last_became_active":"2023-11-28T13:02:54.965319+0000","last_became_peered":"2023-11-28T13:02:54.965319+0000","last_unstale":"2023-11-28T13:02:55.968064+0000","last_undegraded":"2023-11-28T13:02:55.968064+0000","last_fullsized":"2023-11-28T13:02:55.968064+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:48.223379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968120+0000","last_change":"2023-11-28T13:02:54.575183+0000","last_active":"2023-11-28T13:02:55.968120+0000","last_peered":"2023-11-28T13:02:55.968120+0000","last_clean":"2023-11-28T13:02:55.968120+0000","last_became_active":"2023-11-28T13:02:54.574741+0000","last_became_peered":"2023-11-28T13:02:54.574741+0000","last_unstale":"2023-11-28T13:02:55.968120+0000","last_undegraded":"2023-11-28T13:02:55.968120+0000","last_fullsized":"2023-11-28T13:02:55.968120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:13.427513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969101+0000","last_change":"2023-11-28T13:02:54.974494+0000","last_active":"2023-11-28T13:02:55.969101+0000","last_peered":"2023-11-28T13:02:55.969101+0000","last_clean":"2023-11-28T13:02:55.969101+0000","last_became_active":"2023-11-28T13:02:54.974184+0000","last_became_peered":"2023-11-28T13:02:54.974184+0000","last_unstale":"2023-11-28T13:02:55.969101+0000","last_undegraded":"2023-11-28T13:02:55.969101+0000","last_fullsized":"2023-11-28T13:02:55.969101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:49:28.612886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654901+0000","last_change":"2023-11-28T13:02:54.575271+0000","last_active":"2023-11-28T13:02:56.654901+0000","last_peered":"2023-11-28T13:02:56.654901+0000","last_clean":"2023-11-28T13:02:56.654901+0000","last_became_active":"2023-11-28T13:02:54.575026+0000","last_became_peered":"2023-11-28T13:02:54.575026+0000","last_unstale":"2023-11-28T13:02:56.654901+0000","last_undegraded":"2023-11-28T13:02:56.654901+0000","last_fullsized":"2023-11-28T13:02:56.654901+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:50.857354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654027+0000","last_change":"2023-11-28T13:02:54.977663+0000","last_active":"2023-11-28T13:02:56.654027+0000","last_peered":"2023-11-28T13:02:56.654027+0000","last_clean":"2023-11-28T13:02:56.654027+0000","last_became_active":"2023-11-28T13:02:54.976735+0000","last_became_peered":"2023-11-28T13:02:54.976735+0000","last_unstale":"2023-11-28T13:02:56.654027+0000","last_undegraded":"2023-11-28T13:02:56.654027+0000","last_fullsized":"2023-11-28T13:02:56.654027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:35.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653846+0000","last_change":"2023-11-28T13:02:54.576598+0000","last_active":"2023-11-28T13:02:56.653846+0000","last_peered":"2023-11-28T13:02:56.653846+0000","last_clean":"2023-11-28T13:02:56.653846+0000","last_became_active":"2023-11-28T13:02:54.576292+0000","last_became_peered":"2023-11-28T13:02:54.576292+0000","last_unstale":"2023-11-28T13:02:56.653846+0000","last_undegraded":"2023-11-28T13:02:56.653846+0000","last_fullsized":"2023-11-28T13:02:56.653846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:13.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653728+0000","last_change":"2023-11-28T13:02:54.979265+0000","last_active":"2023-11-28T13:02:56.653728+0000","last_peered":"2023-11-28T13:02:56.653728+0000","last_clean":"2023-11-28T13:02:56.653728+0000","last_became_active":"2023-11-28T13:02:54.978935+0000","last_became_peered":"2023-11-28T13:02:54.978935+0000","last_unstale":"2023-11-28T13:02:56.653728+0000","last_undegraded":"2023-11-28T13:02:56.653728+0000","last_fullsized":"2023-11-28T13:02:56.653728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:18.297439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968732+0000","last_change":"2023-11-28T13:02:54.574817+0000","last_active":"2023-11-28T13:02:55.968732+0000","last_peered":"2023-11-28T13:02:55.968732+0000","last_clean":"2023-11-28T13:02:55.968732+0000","last_became_active":"2023-11-28T13:02:54.574382+0000","last_became_peered":"2023-11-28T13:02:54.574382+0000","last_unstale":"2023-11-28T13:02:55.968732+0000","last_undegraded":"2023-11-28T13:02:55.968732+0000","last_fullsized":"2023-11-28T13:02:55.968732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:34:26.543872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152942+0000","last_change":"2023-11-28T13:02:54.970164+0000","last_active":"2023-11-28T13:02:56.152942+0000","last_peered":"2023-11-28T13:02:56.152942+0000","last_clean":"2023-11-28T13:02:56.152942+0000","last_became_active":"2023-11-28T13:02:54.969358+0000","last_became_peered":"2023-11-28T13:02:54.969358+0000","last_unstale":"2023-11-28T13:02:56.152942+0000","last_undegraded":"2023-11-28T13:02:56.152942+0000","last_fullsized":"2023-11-28T13:02:56.152942+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:23:29.634193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968648+0000","last_change":"2023-11-28T13:02:54.573909+0000","last_active":"2023-11-28T13:02:55.968648+0000","last_peered":"2023-11-28T13:02:55.968648+0000","last_clean":"2023-11-28T13:02:55.968648+0000","last_became_active":"2023-11-28T13:02:54.573635+0000","last_became_peered":"2023-11-28T13:02:54.573635+0000","last_unstale":"2023-11-28T13:02:55.968648+0000","last_undegraded":"2023-11-28T13:02:55.968648+0000","last_fullsized":"2023-11-28T13:02:55.968648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:14.566824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968511+0000","last_change":"2023-11-28T13:02:54.964265+0000","last_active":"2023-11-28T13:02:55.968511+0000","last_peered":"2023-11-28T13:02:55.968511+0000","last_clean":"2023-11-28T13:02:55.968511+0000","last_became_active":"2023-11-28T13:02:54.963983+0000","last_became_peered":"2023-11-28T13:02:54.963983+0000","last_unstale":"2023-11-28T13:02:55.968511+0000","last_undegraded":"2023-11-28T13:02:55.968511+0000","last_fullsized":"2023-11-28T13:02:55.968511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.723540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654399+0000","last_change":"2023-11-28T13:02:54.571685+0000","last_active":"2023-11-28T13:02:56.654399+0000","last_peered":"2023-11-28T13:02:56.654399+0000","last_clean":"2023-11-28T13:02:56.654399+0000","last_became_active":"2023-11-28T13:02:54.571375+0000","last_became_peered":"2023-11-28T13:02:54.571375+0000","last_unstale":"2023-11-28T13:02:56.654399+0000","last_undegraded":"2023-11-28T13:02:56.654399+0000","last_fullsized":"2023-11-28T13:02:56.654399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:45.383406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969336+0000","last_change":"2023-11-28T13:02:54.978652+0000","last_active":"2023-11-28T13:02:55.969336+0000","last_peered":"2023-11-28T13:02:55.969336+0000","last_clean":"2023-11-28T13:02:55.969336+0000","last_became_active":"2023-11-28T13:02:54.978214+0000","last_became_peered":"2023-11-28T13:02:54.978214+0000","last_unstale":"2023-11-28T13:02:55.969336+0000","last_undegraded":"2023-11-28T13:02:55.969336+0000","last_fullsized":"2023-11-28T13:02:55.969336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:15.000419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.960507+0000","last_change":"2023-11-28T13:02:54.572070+0000","last_active":"2023-11-28T13:02:54.960507+0000","last_peered":"2023-11-28T13:02:54.960507+0000","last_clean":"2023-11-28T13:02:54.960507+0000","last_became_active":"2023-11-28T13:02:54.571624+0000","last_became_peered":"2023-11-28T13:02:54.571624+0000","last_unstale":"2023-11-28T13:02:54.960507+0000","last_undegraded":"2023-11-28T13:02:54.960507+0000","last_fullsized":"2023-11-28T13:02:54.960507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:24.750211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968891+0000","last_change":"2023-11-28T13:02:54.978080+0000","last_active":"2023-11-28T13:02:55.968891+0000","last_peered":"2023-11-28T13:02:55.968891+0000","last_clean":"2023-11-28T13:02:55.968891+0000","last_became_active":"2023-11-28T13:02:54.977364+0000","last_became_peered":"2023-11-28T13:02:54.977364+0000","last_unstale":"2023-11-28T13:02:55.968891+0000","last_undegraded":"2023-11-28T13:02:55.968891+0000","last_fullsized":"2023-11-28T13:02:55.968891+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:31.551434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654411+0000","last_change":"2023-11-28T13:02:54.574565+0000","last_active":"2023-11-28T13:02:56.654411+0000","last_peered":"2023-11-28T13:02:56.654411+0000","last_clean":"2023-11-28T13:02:56.654411+0000","last_became_active":"2023-11-28T13:02:54.574145+0000","last_became_peered":"2023-11-28T13:02:54.574145+0000","last_unstale":"2023-11-28T13:02:56.654411+0000","last_undegraded":"2023-11-28T13:02:56.654411+0000","last_fullsized":"2023-11-28T13:02:56.654411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:16.453796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654003+0000","last_change":"2023-11-28T13:02:54.978426+0000","last_active":"2023-11-28T13:02:56.654003+0000","last_peered":"2023-11-28T13:02:56.654003+0000","last_clean":"2023-11-28T13:02:56.654003+0000","last_became_active":"2023-11-28T13:02:54.978156+0000","last_became_peered":"2023-11-28T13:02:54.978156+0000","last_unstale":"2023-11-28T13:02:56.654003+0000","last_undegraded":"2023-11-28T13:02:56.654003+0000","last_fullsized":"2023-11-28T13:02:56.654003+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:45:41.711431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965515+0000","last_change":"2023-11-28T13:02:54.565731+0000","last_active":"2023-11-28T13:02:54.965515+0000","last_peered":"2023-11-28T13:02:54.965515+0000","last_clean":"2023-11-28T13:02:54.965515+0000","last_became_active":"2023-11-28T13:02:54.565059+0000","last_became_peered":"2023-11-28T13:02:54.565059+0000","last_unstale":"2023-11-28T13:02:54.965515+0000","last_undegraded":"2023-11-28T13:02:54.965515+0000","last_fullsized":"2023-11-28T13:02:54.965515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:36.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653664+0000","last_change":"2023-11-28T13:02:54.975476+0000","last_active":"2023-11-28T13:02:56.653664+0000","last_peered":"2023-11-28T13:02:56.653664+0000","last_clean":"2023-11-28T13:02:56.653664+0000","last_became_active":"2023-11-28T13:02:54.975273+0000","last_became_peered":"2023-11-28T13:02:54.975273+0000","last_unstale":"2023-11-28T13:02:56.653664+0000","last_undegraded":"2023-11-28T13:02:56.653664+0000","last_fullsized":"2023-11-28T13:02:56.653664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:41.576824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965223+0000","last_change":"2023-11-28T13:02:54.565314+0000","last_active":"2023-11-28T13:02:54.965223+0000","last_peered":"2023-11-28T13:02:54.965223+0000","last_clean":"2023-11-28T13:02:54.965223+0000","last_became_active":"2023-11-28T13:02:54.563242+0000","last_became_peered":"2023-11-28T13:02:54.563242+0000","last_unstale":"2023-11-28T13:02:54.965223+0000","last_undegraded":"2023-11-28T13:02:54.965223+0000","last_fullsized":"2023-11-28T13:02:54.965223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:36.187142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152895+0000","last_change":"2023-11-28T13:02:54.970716+0000","last_active":"2023-11-28T13:02:56.152895+0000","last_peered":"2023-11-28T13:02:56.152895+0000","last_clean":"2023-11-28T13:02:56.152895+0000","last_became_active":"2023-11-28T13:02:54.970465+0000","last_became_peered":"2023-11-28T13:02:54.970465+0000","last_unstale":"2023-11-28T13:02:56.152895+0000","last_undegraded":"2023-11-28T13:02:56.152895+0000","last_fullsized":"2023-11-28T13:02:56.152895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:12.376175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.959325+0000","last_change":"2023-11-28T13:02:54.568037+0000","last_active":"2023-11-28T13:02:54.959325+0000","last_peered":"2023-11-28T13:02:54.959325+0000","last_clean":"2023-11-28T13:02:54.959325+0000","last_became_active":"2023-11-28T13:02:54.566895+0000","last_became_peered":"2023-11-28T13:02:54.566895+0000","last_unstale":"2023-11-28T13:02:54.959325+0000","last_undegraded":"2023-11-28T13:02:54.959325+0000","last_fullsized":"2023-11-28T13:02:54.959325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:34:44.950034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654632+0000","last_change":"2023-11-28T13:02:54.990785+0000","last_active":"2023-11-28T13:02:56.654632+0000","last_peered":"2023-11-28T13:02:56.654632+0000","last_clean":"2023-11-28T13:02:56.654632+0000","last_became_active":"2023-11-28T13:02:54.990473+0000","last_became_peered":"2023-11-28T13:02:54.990473+0000","last_unstale":"2023-11-28T13:02:56.654632+0000","last_undegraded":"2023-11-28T13:02:56.654632+0000","last_fullsized":"2023-11-28T13:02:56.654632+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:48.223379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968647+0000","last_change":"2023-11-28T13:02:54.575460+0000","last_active":"2023-11-28T13:02:55.968647+0000","last_peered":"2023-11-28T13:02:55.968647+0000","last_clean":"2023-11-28T13:02:55.968647+0000","last_became_active":"2023-11-28T13:02:54.575157+0000","last_became_peered":"2023-11-28T13:02:54.575157+0000","last_unstale":"2023-11-28T13:02:55.968647+0000","last_undegraded":"2023-11-28T13:02:55.968647+0000","last_fullsized":"2023-11-28T13:02:55.968647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:24.555799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.960024+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:02:54.960024+0000","last_peered":"2023-11-28T13:02:54.960024+0000","last_clean":"2023-11-28T13:02:54.960024+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:02:54.960024+0000","last_undegraded":"2023-11-28T13:02:54.960024+0000","last_fullsized":"2023-11-28T13:02:54.960024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969101+0000","last_change":"2023-11-28T13:02:54.978861+0000","last_active":"2023-11-28T13:02:55.969101+0000","last_peered":"2023-11-28T13:02:55.969101+0000","last_clean":"2023-11-28T13:02:55.969101+0000","last_became_active":"2023-11-28T13:02:54.978374+0000","last_became_peered":"2023-11-28T13:02:54.978374+0000","last_unstale":"2023-11-28T13:02:55.969101+0000","last_undegraded":"2023-11-28T13:02:55.969101+0000","last_fullsized":"2023-11-28T13:02:55.969101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:10.520996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653875+0000","last_change":"2023-11-28T13:02:54.572150+0000","last_active":"2023-11-28T13:02:56.653875+0000","last_peered":"2023-11-28T13:02:56.653875+0000","last_clean":"2023-11-28T13:02:56.653875+0000","last_became_active":"2023-11-28T13:02:54.571846+0000","last_became_peered":"2023-11-28T13:02:54.571846+0000","last_unstale":"2023-11-28T13:02:56.653875+0000","last_undegraded":"2023-11-28T13:02:56.653875+0000","last_fullsized":"2023-11-28T13:02:56.653875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.733701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.959518+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:02:54.959518+0000","last_peered":"2023-11-28T13:02:54.959518+0000","last_clean":"2023-11-28T13:02:54.959518+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:02:54.959518+0000","last_undegraded":"2023-11-28T13:02:54.959518+0000","last_fullsized":"2023-11-28T13:02:54.959518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968178+0000","last_change":"2023-11-28T13:02:54.969468+0000","last_active":"2023-11-28T13:02:55.968178+0000","last_peered":"2023-11-28T13:02:55.968178+0000","last_clean":"2023-11-28T13:02:55.968178+0000","last_became_active":"2023-11-28T13:02:54.969194+0000","last_became_peered":"2023-11-28T13:02:54.969194+0000","last_unstale":"2023-11-28T13:02:55.968178+0000","last_undegraded":"2023-11-28T13:02:55.968178+0000","last_fullsized":"2023-11-28T13:02:55.968178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:33.775943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968213+0000","last_change":"2023-11-28T13:02:54.575006+0000","last_active":"2023-11-28T13:02:55.968213+0000","last_peered":"2023-11-28T13:02:55.968213+0000","last_clean":"2023-11-28T13:02:55.968213+0000","last_became_active":"2023-11-28T13:02:54.574370+0000","last_became_peered":"2023-11-28T13:02:54.574370+0000","last_unstale":"2023-11-28T13:02:55.968213+0000","last_undegraded":"2023-11-28T13:02:55.968213+0000","last_fullsized":"2023-11-28T13:02:55.968213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:50.857354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.966385+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:02:54.966385+0000","last_peered":"2023-11-28T13:02:54.966385+0000","last_clean":"2023-11-28T13:02:54.966385+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:02:54.966385+0000","last_undegraded":"2023-11-28T13:02:54.966385+0000","last_fullsized":"2023-11-28T13:02:54.966385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969566+0000","last_change":"2023-11-28T13:02:54.977949+0000","last_active":"2023-11-28T13:02:55.969566+0000","last_peered":"2023-11-28T13:02:55.969566+0000","last_clean":"2023-11-28T13:02:55.969566+0000","last_became_active":"2023-11-28T13:02:54.977680+0000","last_became_peered":"2023-11-28T13:02:54.977680+0000","last_unstale":"2023-11-28T13:02:55.969566+0000","last_undegraded":"2023-11-28T13:02:55.969566+0000","last_fullsized":"2023-11-28T13:02:55.969566+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:29.898926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.964771+0000","last_change":"2023-11-28T13:02:54.564196+0000","last_active":"2023-11-28T13:02:54.964771+0000","last_peered":"2023-11-28T13:02:54.964771+0000","last_clean":"2023-11-28T13:02:54.964771+0000","last_became_active":"2023-11-28T13:02:54.563920+0000","last_became_peered":"2023-11-28T13:02:54.563920+0000","last_unstale":"2023-11-28T13:02:54.964771+0000","last_undegraded":"2023-11-28T13:02:54.964771+0000","last_fullsized":"2023-11-28T13:02:54.964771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:57.576385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.959297+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:02:54.959297+0000","last_peered":"2023-11-28T13:02:54.959297+0000","last_clean":"2023-11-28T13:02:54.959297+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:02:54.959297+0000","last_undegraded":"2023-11-28T13:02:54.959297+0000","last_fullsized":"2023-11-28T13:02:54.959297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.979828+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:02:54.979828+0000","last_peered":"2023-11-28T13:02:54.979828+0000","last_clean":"2023-11-28T13:02:54.979828+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:02:54.979828+0000","last_undegraded":"2023-11-28T13:02:54.979828+0000","last_fullsized":"2023-11-28T13:02:54.979828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653826+0000","last_change":"2023-11-28T13:02:54.974128+0000","last_active":"2023-11-28T13:02:56.653826+0000","last_peered":"2023-11-28T13:02:56.653826+0000","last_clean":"2023-11-28T13:02:56.653826+0000","last_became_active":"2023-11-28T13:02:54.973776+0000","last_became_peered":"2023-11-28T13:02:54.973776+0000","last_unstale":"2023-11-28T13:02:56.653826+0000","last_undegraded":"2023-11-28T13:02:56.653826+0000","last_fullsized":"2023-11-28T13:02:56.653826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:49.099601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653780+0000","last_change":"2023-11-28T13:02:54.564097+0000","last_active":"2023-11-28T13:02:56.653780+0000","last_peered":"2023-11-28T13:02:56.653780+0000","last_clean":"2023-11-28T13:02:56.653780+0000","last_became_active":"2023-11-28T13:02:54.563816+0000","last_became_peered":"2023-11-28T13:02:54.563816+0000","last_unstale":"2023-11-28T13:02:56.653780+0000","last_undegraded":"2023-11-28T13:02:56.653780+0000","last_fullsized":"2023-11-28T13:02:56.653780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:52.277147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.979742+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:02:54.979742+0000","last_peered":"2023-11-28T13:02:54.979742+0000","last_clean":"2023-11-28T13:02:54.979742+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:02:54.979742+0000","last_undegraded":"2023-11-28T13:02:54.979742+0000","last_fullsized":"2023-11-28T13:02:54.979742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969476+0000","last_change":"2023-11-28T13:02:54.976703+0000","last_active":"2023-11-28T13:02:55.969476+0000","last_peered":"2023-11-28T13:02:55.969476+0000","last_clean":"2023-11-28T13:02:55.969476+0000","last_became_active":"2023-11-28T13:02:54.976435+0000","last_became_peered":"2023-11-28T13:02:54.976435+0000","last_unstale":"2023-11-28T13:02:55.969476+0000","last_undegraded":"2023-11-28T13:02:55.969476+0000","last_fullsized":"2023-11-28T13:02:55.969476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:17.461214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654331+0000","last_change":"2023-11-28T13:02:54.575909+0000","last_active":"2023-11-28T13:02:56.654331+0000","last_peered":"2023-11-28T13:02:56.654331+0000","last_clean":"2023-11-28T13:02:56.654331+0000","last_became_active":"2023-11-28T13:02:54.575752+0000","last_became_peered":"2023-11-28T13:02:54.575752+0000","last_unstale":"2023-11-28T13:02:56.654331+0000","last_undegraded":"2023-11-28T13:02:56.654331+0000","last_fullsized":"2023-11-28T13:02:56.654331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:50.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.980649+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:02:54.980649+0000","last_peered":"2023-11-28T13:02:54.980649+0000","last_clean":"2023-11-28T13:02:54.980649+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:02:54.980649+0000","last_undegraded":"2023-11-28T13:02:54.980649+0000","last_fullsized":"2023-11-28T13:02:54.980649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654160+0000","last_change":"2023-11-28T13:02:54.978449+0000","last_active":"2023-11-28T13:02:56.654160+0000","last_peered":"2023-11-28T13:02:56.654160+0000","last_clean":"2023-11-28T13:02:56.654160+0000","last_became_active":"2023-11-28T13:02:54.978140+0000","last_became_peered":"2023-11-28T13:02:54.978140+0000","last_unstale":"2023-11-28T13:02:56.654160+0000","last_undegraded":"2023-11-28T13:02:56.654160+0000","last_fullsized":"2023-11-28T13:02:56.654160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:46.948476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654908+0000","last_change":"2023-11-28T13:02:54.574890+0000","last_active":"2023-11-28T13:02:56.654908+0000","last_peered":"2023-11-28T13:02:56.654908+0000","last_clean":"2023-11-28T13:02:56.654908+0000","last_became_active":"2023-11-28T13:02:54.574114+0000","last_became_peered":"2023-11-28T13:02:54.574114+0000","last_unstale":"2023-11-28T13:02:56.654908+0000","last_undegraded":"2023-11-28T13:02:56.654908+0000","last_fullsized":"2023-11-28T13:02:56.654908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:24.555799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.961838+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:02:54.961838+0000","last_peered":"2023-11-28T13:02:54.961838+0000","last_clean":"2023-11-28T13:02:54.961838+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:02:54.961838+0000","last_undegraded":"2023-11-28T13:02:54.961838+0000","last_fullsized":"2023-11-28T13:02:54.961838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969411+0000","last_change":"2023-11-28T13:02:54.978477+0000","last_active":"2023-11-28T13:02:55.969411+0000","last_peered":"2023-11-28T13:02:55.969411+0000","last_clean":"2023-11-28T13:02:55.969411+0000","last_became_active":"2023-11-28T13:02:54.977877+0000","last_became_peered":"2023-11-28T13:02:54.977877+0000","last_unstale":"2023-11-28T13:02:55.969411+0000","last_undegraded":"2023-11-28T13:02:55.969411+0000","last_fullsized":"2023-11-28T13:02:55.969411+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:56.846859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965377+0000","last_change":"2023-11-28T13:02:54.565033+0000","last_active":"2023-11-28T13:02:54.965377+0000","last_peered":"2023-11-28T13:02:54.965377+0000","last_clean":"2023-11-28T13:02:54.965377+0000","last_became_active":"2023-11-28T13:02:54.564748+0000","last_became_peered":"2023-11-28T13:02:54.564748+0000","last_unstale":"2023-11-28T13:02:54.965377+0000","last_undegraded":"2023-11-28T13:02:54.965377+0000","last_fullsized":"2023-11-28T13:02:54.965377+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:58.581060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.967405+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:02:54.967405+0000","last_peered":"2023-11-28T13:02:54.967405+0000","last_clean":"2023-11-28T13:02:54.967405+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:02:54.967405+0000","last_undegraded":"2023-11-28T13:02:54.967405+0000","last_fullsized":"2023-11-28T13:02:54.967405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9764,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362076,"statfs":{"total":96636764160,"available":96626765824,"internally_reserved":0,"allocated":749568,"data_stored":328140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.497,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.63,"15min":0.67800000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.498,"5min":8.218,"15min":8.218},"last":0.36299999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39700000000000002,"15min":0.40400000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.499,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.40400000000000003,"15min":0.41299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.47599999999999998,"5min":1.355,"15min":1.355},"last":0.377}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.871,"15min":0.95199999999999996},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55400000000000005,"5min":13.384,"15min":13.384},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.58999999999999997,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52300000000000002,"5min":5.226,"15min":5.226},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.75900000000000001,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79400000000000004,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.50600000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.85399999999999998,"15min":0.89700000000000002},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81899999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.51300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93799999999999994,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.88300000000000001,"15min":0.92900000000000005},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.78900000000000003,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.56999999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9380,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362460,"statfs":{"total":96636764160,"available":96627159040,"internally_reserved":0,"allocated":749568,"data_stored":328140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.248},{"interface":"front","average":{"1min":0.374,"5min":0.34100000000000003,"15min":0.33500000000000002},"min":{"1min":0.216,"5min":0.187,"15min":0.187},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.37},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.438}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.39400000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.56200000000000006}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.49199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9584,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362256,"statfs":{"total":96636764160,"available":96626950144,"internally_reserved":0,"allocated":761856,"data_stored":329688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.504},{"interface":"front","average":{"1min":0.501,"5min":0.502,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.432}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.753,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46300000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14756,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357084,"statfs":{"total":96636764160,"available":96621654016,"internally_reserved":0,"allocated":1339392,"data_stored":917739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.85299999999999998,"15min":0.93999999999999995},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.85499999999999998,"15min":0.93799999999999994},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61399999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.86199999999999999,"15min":0.93899999999999995},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.98799999999999999,"15min":1.091},"min":{"1min":0.247,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58399999999999996,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.69099999999999995,"15min":0.70899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94299999999999995,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.218,"15min":1.3280000000000001},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68999999999999995,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.204,"15min":1.3109999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76100000000000001,"5min":19.02,"15min":19.02},"last":0.443},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.503}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.748,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.83099999999999996,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.2250000000000001,"15min":1.3320000000000001},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.40999999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10808,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94361032,"statfs":{"total":96636764160,"available":96625696768,"internally_reserved":0,"allocated":1359872,"data_stored":919268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":1.415,"15min":1.415},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":1.867,"15min":1.867},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71399999999999997,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.505},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.375}]},{"osd":3,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2749999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.8440000000000003,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.51200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.528,"15min":1.528},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.625}]}]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9512,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362328,"statfs":{"total":96636764160,"available":96627023872,"internally_reserved":0,"allocated":753664,"data_stored":328162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":1.419,"15min":1.419},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.41699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.40300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79900000000000004,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.68899999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55700000000000005}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47899999999999998},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67100000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40799999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73399999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.748,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52900000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9384,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362456,"statfs":{"total":96636764160,"available":96627154944,"internally_reserved":0,"allocated":753664,"data_stored":328174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.44,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.441,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46800000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.5,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57699999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.622},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.60899999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9472,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362368,"statfs":{"total":96636764160,"available":96627064832,"internally_reserved":0,"allocated":778240,"data_stored":330436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.20999999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.45900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.253},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.33600000000000002,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.441,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52600000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.315},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54900000000000004,"5min":0.752,"15min":0.752},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.51600000000000001,"5min":0.875,"15min":0.875},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":1.044,"15min":1.044},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.499},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75900000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71299999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68400000000000005,"5min":1.089,"15min":1.089},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72499999999999998,"5min":1.202,"15min":1.202},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74399999999999999,"5min":1.02,"15min":1.02},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:03:03.571 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:03:03.703 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.703+0000 7f8307a73700 1 Processor -- start 2023-11-28T13:03:03.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.703+0000 7f8307a73700 1 -- start start 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f8307a73700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 0x7f83000af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f8307a73700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000cc5f0 con 0x7f830015cef0 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f8307a73700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830015d470 con 0x7f830006ba90 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f8307a73700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830015d750 con 0x7f83000cbdf0 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830580f700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f830015cef0 0x7f830006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60446/0 (socket says 172.21.15.153:60446) 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830580f700 1 -- 172.21.15.153:0/3764850022 learned_addr learned my addr 172.21.15.153:0/3764850022 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f8306010700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 0x7f83000af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257732979 0 0) 0x7f83000cc5f0 con 0x7f830015cef0 2023-11-28T13:03:03.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ec0035b0 con 0x7f830015cef0 2023-11-28T13:03:03.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326179435 0 0) 0x7f830015d470 con 0x7f830006ba90 2023-11-28T13:03:03.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83000cc5f0 con 0x7f830006ba90 2023-11-28T13:03:03.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2237990089 0 0) 0x7f82ec0035b0 con 0x7f830015cef0 2023-11-28T13:03:03.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.704+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 msgr2=0x7f83000af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:03.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 0x7f83000af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 >> v1:172.21.15.177:6789/0 conn(0x7f830006ba90 legacy=0x7f83000ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830015da30 con 0x7f830015cef0 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f82fc001670 con 0x7f830015cef0 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 0x7f83000af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 >> v1:172.21.15.153:6789/0 conn(0x7f830015cef0 legacy=0x7f830006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ec003bc0 0x7f82ec003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 0x7f82ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec004d90 0x7f82ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830015d750 con 0x7f82ec004d90 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec00bac0 con 0x7f82ec003bc0 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec00bc00 con 0x7f83000cbdf0 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec004d90 0x7f82ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830580f700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ec003bc0 0x7f82ec003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830500e700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 0x7f82ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 -- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 msgr2=0x7f82ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83000cbdf0 0x7f82ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 -- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ec003bc0 msgr2=0x7f82ec003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ec003bc0 0x7f82ec003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 -- 172.21.15.153:0/3764850022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8009a70 con 0x7f82ec004d90 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f8306010700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec004d90 0x7f82ec005200 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f82f800e350 tx=0x7f82ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf4fda0b29b7457 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800edf0 con 0x7f82ec004d90 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f800f040 con 0x7f82ec004d90 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.705+0000 7f830480d700 1 -- 172.21.15.153:0/3764850022 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80146a0 con 0x7f82ec004d90 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.706+0000 7f8307a73700 1 -- 172.21.15.153:0/3764850022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec004d90 msgr2=0x7f82ec005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.706+0000 7f8307a73700 1 --2- 172.21.15.153:0/3764850022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82ec004d90 0x7f82ec005200 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f82f800e350 tx=0x7f82ec005880 comp rx=0 tx=0).stop 2023-11-28T13:03:03.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.706+0000 7f8306010700 1 -- 172.21.15.153:0/3764850022 reap_dead start 2023-11-28T13:03:03.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.706+0000 7f8307a73700 1 -- 172.21.15.153:0/3764850022 shutdown_connections 2023-11-28T13:03:03.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.706+0000 7f8307a73700 1 -- 172.21.15.153:0/3764850022 >> 172.21.15.153:0/3764850022 conn(0x7f83000d0000 msgr2=0x7f83000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:03.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.706+0000 7f8307a73700 1 -- 172.21.15.153:0/3764850022 shutdown_connections 2023-11-28T13:03:03.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.706+0000 7f8307a73700 1 -- 172.21.15.153:0/3764850022 wait complete. 2023-11-28T13:03:03.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.707+0000 7f8307a73700 1 Processor -- start 2023-11-28T13:03:03.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.707+0000 7f8307a73700 1 -- start start 2023-11-28T13:03:03.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8307a73700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f830014cdf0 0x7f830006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8307a73700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83000c3b10 0x7f830006a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8307a73700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830014f7a0 0x7f830014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8307a73700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83001504d0 con 0x7f83000c3b10 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8307a73700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830014d1c0 con 0x7f830014f7a0 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8307a73700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830014d610 con 0x7f830014cdf0 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83000c3b10 0x7f830006a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83000c3b10 0x7f830006a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42296/0 (socket says 172.21.15.153:42296) 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 -- 172.21.15.153:0/4066867611 learned_addr learned my addr 172.21.15.153:0/4066867611 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8306010700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830014f7a0 0x7f830014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830580f700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f830014cdf0 0x7f830006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 -- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f830014cdf0 msgr2=0x7f830006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f830014cdf0 0x7f830006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 -- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830014f7a0 msgr2=0x7f830014fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830014f7a0 0x7f830014fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f830500e700 1 -- 172.21.15.153:0/4066867611 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830015dff0 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.708+0000 7f8306010700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830014f7a0 0x7f830014fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.709+0000 7f830500e700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83000c3b10 0x7f830006a7b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f82f000f810 tx=0x7f82f000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.709+0000 7f82f67fc700 1 -- 172.21.15.153:0/4066867611 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00052e0 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.709+0000 7f82f67fc700 1 -- 172.21.15.153:0/4066867611 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0005440 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.709+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830015dd10 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.709+0000 7f82f67fc700 1 -- 172.21.15.153:0/4066867611 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00038c0 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.709+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83000af400 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.710+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830009dd80 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.710+0000 7f82f67fc700 1 -- 172.21.15.153:0/4066867611 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f82f0003a20 con 0x7f83000c3b10 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.711+0000 7f82f67fc700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82e40682f0 0x7f82e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.711+0000 7f830580f700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82e40682f0 0x7f82e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.711+0000 7f82f67fc700 1 -- 172.21.15.153:0/4066867611 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82f0086750 con 0x7f83000c3b10 2023-11-28T13:03:03.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.711+0000 7f830580f700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82e40682f0 0x7f82e406a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f82fc000c00 tx=0x7f82fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.715+0000 7f82f67fc700 1 -- 172.21.15.153:0/4066867611 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f82f0053050 con 0x7f83000c3b10 2023-11-28T13:03:03.895 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.895+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83000d0e60 con 0x7f82e40682f0 2023-11-28T13:03:03.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.903+0000 7f82f67fc700 1 -- 172.21.15.153:0/4066867611 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311450 (secure 0 0 0) 0x7f83000d0e60 con 0x7f82e40682f0 2023-11-28T13:03:03.907 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:03.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82e40682f0 msgr2=0x7f82e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82e40682f0 0x7f82e406a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f82fc000c00 tx=0x7f82fc009040 comp rx=0 tx=0).stop 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83000c3b10 msgr2=0x7f830006a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83000c3b10 0x7f830006a7b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f82f000f810 tx=0x7f82f000fbd0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 shutdown_connections 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830014f7a0 0x7f830014fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f830014cdf0 0x7f830006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f82e40682f0 0x7f82e406a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 --2- 172.21.15.153:0/4066867611 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83000c3b10 0x7f830006a7b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 >> 172.21.15.153:0/4066867611 conn(0x7f83000d0000 msgr2=0x7f830015af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 shutdown_connections 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:03.906+0000 7f8307a73700 1 -- 172.21.15.153:0/4066867611 wait complete. 2023-11-28T13:03:03.912 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:03:03.919 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-11-28T13:03:02.183051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82660,"kb_used_data":7076,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":754892060,"statfs":{"total":773094113280,"available":773009469440,"internally_reserved":0,"allocated":7245824,"data_stored":3809747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.628491"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653834+0000","last_change":"2023-11-28T13:02:54.967102+0000","last_active":"2023-11-28T13:02:56.653834+0000","last_peered":"2023-11-28T13:02:56.653834+0000","last_clean":"2023-11-28T13:02:56.653834+0000","last_became_active":"2023-11-28T13:02:54.966804+0000","last_became_peered":"2023-11-28T13:02:54.966804+0000","last_unstale":"2023-11-28T13:02:56.653834+0000","last_undegraded":"2023-11-28T13:02:56.653834+0000","last_fullsized":"2023-11-28T13:02:56.653834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:25.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654114+0000","last_change":"2023-11-28T13:02:54.971255+0000","last_active":"2023-11-28T13:02:56.654114+0000","last_peered":"2023-11-28T13:02:56.654114+0000","last_clean":"2023-11-28T13:02:56.654114+0000","last_became_active":"2023-11-28T13:02:54.970382+0000","last_became_peered":"2023-11-28T13:02:54.970382+0000","last_unstale":"2023-11-28T13:02:56.654114+0000","last_undegraded":"2023-11-28T13:02:56.654114+0000","last_fullsized":"2023-11-28T13:02:56.654114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:13:44.841594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654838+0000","last_change":"2023-11-28T13:02:54.986116+0000","last_active":"2023-11-28T13:02:56.654838+0000","last_peered":"2023-11-28T13:02:56.654838+0000","last_clean":"2023-11-28T13:02:56.654838+0000","last_became_active":"2023-11-28T13:02:54.985837+0000","last_became_peered":"2023-11-28T13:02:54.985837+0000","last_unstale":"2023-11-28T13:02:56.654838+0000","last_undegraded":"2023-11-28T13:02:56.654838+0000","last_fullsized":"2023-11-28T13:02:56.654838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:16.518326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654572+0000","last_change":"2023-11-28T13:02:54.982949+0000","last_active":"2023-11-28T13:02:56.654572+0000","last_peered":"2023-11-28T13:02:56.654572+0000","last_clean":"2023-11-28T13:02:56.654572+0000","last_became_active":"2023-11-28T13:02:54.982553+0000","last_became_peered":"2023-11-28T13:02:54.982553+0000","last_unstale":"2023-11-28T13:02:56.654572+0000","last_undegraded":"2023-11-28T13:02:56.654572+0000","last_fullsized":"2023-11-28T13:02:56.654572+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:39.068694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969244+0000","last_change":"2023-11-28T13:02:54.975631+0000","last_active":"2023-11-28T13:02:55.969244+0000","last_peered":"2023-11-28T13:02:55.969244+0000","last_clean":"2023-11-28T13:02:55.969244+0000","last_became_active":"2023-11-28T13:02:54.975355+0000","last_became_peered":"2023-11-28T13:02:54.975355+0000","last_unstale":"2023-11-28T13:02:55.969244+0000","last_undegraded":"2023-11-28T13:02:55.969244+0000","last_fullsized":"2023-11-28T13:02:55.969244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:51.864773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654061+0000","last_change":"2023-11-28T13:02:54.984256+0000","last_active":"2023-11-28T13:02:56.654061+0000","last_peered":"2023-11-28T13:02:56.654061+0000","last_clean":"2023-11-28T13:02:56.654061+0000","last_became_active":"2023-11-28T13:02:54.984012+0000","last_became_peered":"2023-11-28T13:02:54.984012+0000","last_unstale":"2023-11-28T13:02:56.654061+0000","last_undegraded":"2023-11-28T13:02:56.654061+0000","last_fullsized":"2023-11-28T13:02:56.654061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:58.068458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152743+0000","last_change":"2023-11-28T13:02:54.977048+0000","last_active":"2023-11-28T13:02:56.152743+0000","last_peered":"2023-11-28T13:02:56.152743+0000","last_clean":"2023-11-28T13:02:56.152743+0000","last_became_active":"2023-11-28T13:02:54.976778+0000","last_became_peered":"2023-11-28T13:02:54.976778+0000","last_unstale":"2023-11-28T13:02:56.152743+0000","last_undegraded":"2023-11-28T13:02:56.152743+0000","last_fullsized":"2023-11-28T13:02:56.152743+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:52.444793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968064+0000","last_change":"2023-11-28T13:02:54.980339+0000","last_active":"2023-11-28T13:02:55.968064+0000","last_peered":"2023-11-28T13:02:55.968064+0000","last_clean":"2023-11-28T13:02:55.968064+0000","last_became_active":"2023-11-28T13:02:54.980058+0000","last_became_peered":"2023-11-28T13:02:54.980058+0000","last_unstale":"2023-11-28T13:02:55.968064+0000","last_undegraded":"2023-11-28T13:02:55.968064+0000","last_fullsized":"2023-11-28T13:02:55.968064+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:36.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654535+0000","last_change":"2023-11-28T13:02:54.988713+0000","last_active":"2023-11-28T13:02:56.654535+0000","last_peered":"2023-11-28T13:02:56.654535+0000","last_clean":"2023-11-28T13:02:56.654535+0000","last_became_active":"2023-11-28T13:02:54.988216+0000","last_became_peered":"2023-11-28T13:02:54.988216+0000","last_unstale":"2023-11-28T13:02:56.654535+0000","last_undegraded":"2023-11-28T13:02:56.654535+0000","last_fullsized":"2023-11-28T13:02:56.654535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:46.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654142+0000","last_change":"2023-11-28T13:02:54.987462+0000","last_active":"2023-11-28T13:02:56.654142+0000","last_peered":"2023-11-28T13:02:56.654142+0000","last_clean":"2023-11-28T13:02:56.654142+0000","last_became_active":"2023-11-28T13:02:54.987168+0000","last_became_peered":"2023-11-28T13:02:54.987168+0000","last_unstale":"2023-11-28T13:02:56.654142+0000","last_undegraded":"2023-11-28T13:02:56.654142+0000","last_fullsized":"2023-11-28T13:02:56.654142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:36.972161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654594+0000","last_change":"2023-11-28T13:02:54.974431+0000","last_active":"2023-11-28T13:02:56.654594+0000","last_peered":"2023-11-28T13:02:56.654594+0000","last_clean":"2023-11-28T13:02:56.654594+0000","last_became_active":"2023-11-28T13:02:54.974076+0000","last_became_peered":"2023-11-28T13:02:54.974076+0000","last_unstale":"2023-11-28T13:02:56.654594+0000","last_undegraded":"2023-11-28T13:02:56.654594+0000","last_fullsized":"2023-11-28T13:02:56.654594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:23.744383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968373+0000","last_change":"2023-11-28T13:02:54.977275+0000","last_active":"2023-11-28T13:02:55.968373+0000","last_peered":"2023-11-28T13:02:55.968373+0000","last_clean":"2023-11-28T13:02:55.968373+0000","last_became_active":"2023-11-28T13:02:54.976935+0000","last_became_peered":"2023-11-28T13:02:54.976935+0000","last_unstale":"2023-11-28T13:02:55.968373+0000","last_undegraded":"2023-11-28T13:02:55.968373+0000","last_fullsized":"2023-11-28T13:02:55.968373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:51.741120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968480+0000","last_change":"2023-11-28T13:02:54.974143+0000","last_active":"2023-11-28T13:02:55.968480+0000","last_peered":"2023-11-28T13:02:55.968480+0000","last_clean":"2023-11-28T13:02:55.968480+0000","last_became_active":"2023-11-28T13:02:54.973897+0000","last_became_peered":"2023-11-28T13:02:54.973897+0000","last_unstale":"2023-11-28T13:02:55.968480+0000","last_undegraded":"2023-11-28T13:02:55.968480+0000","last_fullsized":"2023-11-28T13:02:55.968480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:11.038463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654865+0000","last_change":"2023-11-28T13:02:54.986158+0000","last_active":"2023-11-28T13:02:56.654865+0000","last_peered":"2023-11-28T13:02:56.654865+0000","last_clean":"2023-11-28T13:02:56.654865+0000","last_became_active":"2023-11-28T13:02:54.985916+0000","last_became_peered":"2023-11-28T13:02:54.985916+0000","last_unstale":"2023-11-28T13:02:56.654865+0000","last_undegraded":"2023-11-28T13:02:56.654865+0000","last_fullsized":"2023-11-28T13:02:56.654865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:27:44.453808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654732+0000","last_change":"2023-11-28T13:02:54.984129+0000","last_active":"2023-11-28T13:02:56.654732+0000","last_peered":"2023-11-28T13:02:56.654732+0000","last_clean":"2023-11-28T13:02:56.654732+0000","last_became_active":"2023-11-28T13:02:54.983853+0000","last_became_peered":"2023-11-28T13:02:54.983853+0000","last_unstale":"2023-11-28T13:02:56.654732+0000","last_undegraded":"2023-11-28T13:02:56.654732+0000","last_fullsized":"2023-11-28T13:02:56.654732+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.723540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152703+0000","last_change":"2023-11-28T13:02:54.970706+0000","last_active":"2023-11-28T13:02:56.152703+0000","last_peered":"2023-11-28T13:02:56.152703+0000","last_clean":"2023-11-28T13:02:56.152703+0000","last_became_active":"2023-11-28T13:02:54.970422+0000","last_became_peered":"2023-11-28T13:02:54.970422+0000","last_unstale":"2023-11-28T13:02:56.152703+0000","last_undegraded":"2023-11-28T13:02:56.152703+0000","last_fullsized":"2023-11-28T13:02:56.152703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:38:37.749170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653973+0000","last_change":"2023-11-28T13:02:54.977040+0000","last_active":"2023-11-28T13:02:56.653973+0000","last_peered":"2023-11-28T13:02:56.653973+0000","last_clean":"2023-11-28T13:02:56.653973+0000","last_became_active":"2023-11-28T13:02:54.976803+0000","last_became_peered":"2023-11-28T13:02:54.976803+0000","last_unstale":"2023-11-28T13:02:56.653973+0000","last_undegraded":"2023-11-28T13:02:56.653973+0000","last_fullsized":"2023-11-28T13:02:56.653973+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:43:34.185333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654057+0000","last_change":"2023-11-28T13:02:54.979526+0000","last_active":"2023-11-28T13:02:56.654057+0000","last_peered":"2023-11-28T13:02:56.654057+0000","last_clean":"2023-11-28T13:02:56.654057+0000","last_became_active":"2023-11-28T13:02:54.979273+0000","last_became_peered":"2023-11-28T13:02:54.979273+0000","last_unstale":"2023-11-28T13:02:56.654057+0000","last_undegraded":"2023-11-28T13:02:56.654057+0000","last_fullsized":"2023-11-28T13:02:56.654057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:51.025655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969364+0000","last_change":"2023-11-28T13:02:54.974515+0000","last_active":"2023-11-28T13:02:55.969364+0000","last_peered":"2023-11-28T13:02:55.969364+0000","last_clean":"2023-11-28T13:02:55.969364+0000","last_became_active":"2023-11-28T13:02:54.974230+0000","last_became_peered":"2023-11-28T13:02:54.974230+0000","last_unstale":"2023-11-28T13:02:55.969364+0000","last_undegraded":"2023-11-28T13:02:55.969364+0000","last_fullsized":"2023-11-28T13:02:55.969364+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:01.077047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.976273+0000","last_change":"2023-11-28T13:02:54.970059+0000","last_active":"2023-11-28T13:02:54.976273+0000","last_peered":"2023-11-28T13:02:54.976273+0000","last_clean":"2023-11-28T13:02:54.976273+0000","last_became_active":"2023-11-28T13:02:54.969802+0000","last_became_peered":"2023-11-28T13:02:54.969802+0000","last_unstale":"2023-11-28T13:02:54.976273+0000","last_undegraded":"2023-11-28T13:02:54.976273+0000","last_fullsized":"2023-11-28T13:02:54.976273+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:21.818517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968668+0000","last_change":"2023-11-28T13:02:54.964650+0000","last_active":"2023-11-28T13:02:55.968668+0000","last_peered":"2023-11-28T13:02:55.968668+0000","last_clean":"2023-11-28T13:02:55.968668+0000","last_became_active":"2023-11-28T13:02:54.964340+0000","last_became_peered":"2023-11-28T13:02:54.964340+0000","last_unstale":"2023-11-28T13:02:55.968668+0000","last_undegraded":"2023-11-28T13:02:55.968668+0000","last_fullsized":"2023-11-28T13:02:55.968668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:16.518326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.984525+0000","last_change":"2023-11-28T13:02:54.983831+0000","last_active":"2023-11-28T13:02:54.984525+0000","last_peered":"2023-11-28T13:02:54.984525+0000","last_clean":"2023-11-28T13:02:54.984525+0000","last_became_active":"2023-11-28T13:02:54.983570+0000","last_became_peered":"2023-11-28T13:02:54.983570+0000","last_unstale":"2023-11-28T13:02:54.984525+0000","last_undegraded":"2023-11-28T13:02:54.984525+0000","last_fullsized":"2023-11-28T13:02:54.984525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:09.427676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.978254+0000","last_change":"2023-11-28T13:02:54.977435+0000","last_active":"2023-11-28T13:02:54.978254+0000","last_peered":"2023-11-28T13:02:54.978254+0000","last_clean":"2023-11-28T13:02:54.978254+0000","last_became_active":"2023-11-28T13:02:54.977195+0000","last_became_peered":"2023-11-28T13:02:54.977195+0000","last_unstale":"2023-11-28T13:02:54.978254+0000","last_undegraded":"2023-11-28T13:02:54.978254+0000","last_fullsized":"2023-11-28T13:02:54.978254+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:28.957204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654013+0000","last_change":"2023-11-28T13:02:54.984296+0000","last_active":"2023-11-28T13:02:56.654013+0000","last_peered":"2023-11-28T13:02:56.654013+0000","last_clean":"2023-11-28T13:02:56.654013+0000","last_became_active":"2023-11-28T13:02:54.984032+0000","last_became_peered":"2023-11-28T13:02:54.984032+0000","last_unstale":"2023-11-28T13:02:56.654013+0000","last_undegraded":"2023-11-28T13:02:56.654013+0000","last_fullsized":"2023-11-28T13:02:56.654013+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:37.697972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968528+0000","last_change":"2023-11-28T13:02:54.969618+0000","last_active":"2023-11-28T13:02:55.968528+0000","last_peered":"2023-11-28T13:02:55.968528+0000","last_clean":"2023-11-28T13:02:55.968528+0000","last_became_active":"2023-11-28T13:02:54.969252+0000","last_became_peered":"2023-11-28T13:02:54.969252+0000","last_unstale":"2023-11-28T13:02:55.968528+0000","last_undegraded":"2023-11-28T13:02:55.968528+0000","last_fullsized":"2023-11-28T13:02:55.968528+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:37.697972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968474+0000","last_change":"2023-11-28T13:02:54.969033+0000","last_active":"2023-11-28T13:02:55.968474+0000","last_peered":"2023-11-28T13:02:55.968474+0000","last_clean":"2023-11-28T13:02:55.968474+0000","last_became_active":"2023-11-28T13:02:54.967766+0000","last_became_peered":"2023-11-28T13:02:54.967766+0000","last_unstale":"2023-11-28T13:02:55.968474+0000","last_undegraded":"2023-11-28T13:02:55.968474+0000","last_fullsized":"2023-11-28T13:02:55.968474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:46.754083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654277+0000","last_change":"2023-11-28T13:02:54.976309+0000","last_active":"2023-11-28T13:02:56.654277+0000","last_peered":"2023-11-28T13:02:56.654277+0000","last_clean":"2023-11-28T13:02:56.654277+0000","last_became_active":"2023-11-28T13:02:54.975431+0000","last_became_peered":"2023-11-28T13:02:54.975431+0000","last_unstale":"2023-11-28T13:02:56.654277+0000","last_undegraded":"2023-11-28T13:02:56.654277+0000","last_fullsized":"2023-11-28T13:02:56.654277+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:01:38.197496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654243+0000","last_change":"2023-11-28T13:02:54.972357+0000","last_active":"2023-11-28T13:02:56.654243+0000","last_peered":"2023-11-28T13:02:56.654243+0000","last_clean":"2023-11-28T13:02:56.654243+0000","last_became_active":"2023-11-28T13:02:54.972103+0000","last_became_peered":"2023-11-28T13:02:54.972103+0000","last_unstale":"2023-11-28T13:02:56.654243+0000","last_undegraded":"2023-11-28T13:02:56.654243+0000","last_fullsized":"2023-11-28T13:02:56.654243+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:52:44.285754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654212+0000","last_change":"2023-11-28T13:02:54.988699+0000","last_active":"2023-11-28T13:02:56.654212+0000","last_peered":"2023-11-28T13:02:56.654212+0000","last_clean":"2023-11-28T13:02:56.654212+0000","last_became_active":"2023-11-28T13:02:54.988399+0000","last_became_peered":"2023-11-28T13:02:54.988399+0000","last_unstale":"2023-11-28T13:02:56.654212+0000","last_undegraded":"2023-11-28T13:02:56.654212+0000","last_fullsized":"2023-11-28T13:02:56.654212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:46.754083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968182+0000","last_change":"2023-11-28T13:02:54.984161+0000","last_active":"2023-11-28T13:02:55.968182+0000","last_peered":"2023-11-28T13:02:55.968182+0000","last_clean":"2023-11-28T13:02:55.968182+0000","last_became_active":"2023-11-28T13:02:54.983896+0000","last_became_peered":"2023-11-28T13:02:54.983896+0000","last_unstale":"2023-11-28T13:02:55.968182+0000","last_undegraded":"2023-11-28T13:02:55.968182+0000","last_fullsized":"2023-11-28T13:02:55.968182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:53.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654016+0000","last_change":"2023-11-28T13:02:54.975745+0000","last_active":"2023-11-28T13:02:56.654016+0000","last_peered":"2023-11-28T13:02:56.654016+0000","last_clean":"2023-11-28T13:02:56.654016+0000","last_became_active":"2023-11-28T13:02:54.975472+0000","last_became_peered":"2023-11-28T13:02:54.975472+0000","last_unstale":"2023-11-28T13:02:56.654016+0000","last_undegraded":"2023-11-28T13:02:56.654016+0000","last_fullsized":"2023-11-28T13:02:56.654016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:05.196494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653599+0000","last_change":"2023-11-28T13:02:54.973929+0000","last_active":"2023-11-28T13:02:56.653599+0000","last_peered":"2023-11-28T13:02:56.653599+0000","last_clean":"2023-11-28T13:02:56.653599+0000","last_became_active":"2023-11-28T13:02:54.973639+0000","last_became_peered":"2023-11-28T13:02:54.973639+0000","last_unstale":"2023-11-28T13:02:56.653599+0000","last_undegraded":"2023-11-28T13:02:56.653599+0000","last_fullsized":"2023-11-28T13:02:56.653599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:36:28.472469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969354+0000","last_change":"2023-11-28T13:02:54.979291+0000","last_active":"2023-11-28T13:02:55.969354+0000","last_peered":"2023-11-28T13:02:55.969354+0000","last_clean":"2023-11-28T13:02:55.969354+0000","last_became_active":"2023-11-28T13:02:54.977003+0000","last_became_peered":"2023-11-28T13:02:54.977003+0000","last_unstale":"2023-11-28T13:02:55.969354+0000","last_undegraded":"2023-11-28T13:02:55.969354+0000","last_fullsized":"2023-11-28T13:02:55.969354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:59:40.499752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.974611+0000","last_change":"2023-11-28T13:02:54.574619+0000","last_active":"2023-11-28T13:02:54.974611+0000","last_peered":"2023-11-28T13:02:54.974611+0000","last_clean":"2023-11-28T13:02:54.974611+0000","last_became_active":"2023-11-28T13:02:54.574236+0000","last_became_peered":"2023-11-28T13:02:54.574236+0000","last_unstale":"2023-11-28T13:02:54.974611+0000","last_undegraded":"2023-11-28T13:02:54.974611+0000","last_fullsized":"2023-11-28T13:02:54.974611+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:19.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654394+0000","last_change":"2023-11-28T13:02:54.979052+0000","last_active":"2023-11-28T13:02:56.654394+0000","last_peered":"2023-11-28T13:02:56.654394+0000","last_clean":"2023-11-28T13:02:56.654394+0000","last_became_active":"2023-11-28T13:02:54.978662+0000","last_became_peered":"2023-11-28T13:02:54.978662+0000","last_unstale":"2023-11-28T13:02:56.654394+0000","last_undegraded":"2023-11-28T13:02:56.654394+0000","last_fullsized":"2023-11-28T13:02:56.654394+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:14:03.502160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654252+0000","last_change":"2023-11-28T13:02:54.576056+0000","last_active":"2023-11-28T13:02:56.654252+0000","last_peered":"2023-11-28T13:02:56.654252+0000","last_clean":"2023-11-28T13:02:56.654252+0000","last_became_active":"2023-11-28T13:02:54.575718+0000","last_became_peered":"2023-11-28T13:02:54.575718+0000","last_unstale":"2023-11-28T13:02:56.654252+0000","last_undegraded":"2023-11-28T13:02:56.654252+0000","last_fullsized":"2023-11-28T13:02:56.654252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:03:27.738591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654159+0000","last_change":"2023-11-28T13:02:54.969947+0000","last_active":"2023-11-28T13:02:56.654159+0000","last_peered":"2023-11-28T13:02:56.654159+0000","last_clean":"2023-11-28T13:02:56.654159+0000","last_became_active":"2023-11-28T13:02:54.969698+0000","last_became_peered":"2023-11-28T13:02:54.969698+0000","last_unstale":"2023-11-28T13:02:56.654159+0000","last_undegraded":"2023-11-28T13:02:56.654159+0000","last_fullsized":"2023-11-28T13:02:56.654159+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:05.666567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654306+0000","last_change":"2023-11-28T13:02:54.572156+0000","last_active":"2023-11-28T13:02:56.654306+0000","last_peered":"2023-11-28T13:02:56.654306+0000","last_clean":"2023-11-28T13:02:56.654306+0000","last_became_active":"2023-11-28T13:02:54.571876+0000","last_became_peered":"2023-11-28T13:02:54.571876+0000","last_unstale":"2023-11-28T13:02:56.654306+0000","last_undegraded":"2023-11-28T13:02:56.654306+0000","last_fullsized":"2023-11-28T13:02:56.654306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:50.653241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654338+0000","last_change":"2023-11-28T13:02:54.987454+0000","last_active":"2023-11-28T13:02:56.654338+0000","last_peered":"2023-11-28T13:02:56.654338+0000","last_clean":"2023-11-28T13:02:56.654338+0000","last_became_active":"2023-11-28T13:02:54.987193+0000","last_became_peered":"2023-11-28T13:02:54.987193+0000","last_unstale":"2023-11-28T13:02:56.654338+0000","last_undegraded":"2023-11-28T13:02:56.654338+0000","last_fullsized":"2023-11-28T13:02:56.654338+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:51.741120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653781+0000","last_change":"2023-11-28T13:02:54.564777+0000","last_active":"2023-11-28T13:02:56.653781+0000","last_peered":"2023-11-28T13:02:56.653781+0000","last_clean":"2023-11-28T13:02:56.653781+0000","last_became_active":"2023-11-28T13:02:54.564503+0000","last_became_peered":"2023-11-28T13:02:54.564503+0000","last_unstale":"2023-11-28T13:02:56.653781+0000","last_undegraded":"2023-11-28T13:02:56.653781+0000","last_fullsized":"2023-11-28T13:02:56.653781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:39:38.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152777+0000","last_change":"2023-11-28T13:02:54.967759+0000","last_active":"2023-11-28T13:02:56.152777+0000","last_peered":"2023-11-28T13:02:56.152777+0000","last_clean":"2023-11-28T13:02:56.152777+0000","last_became_active":"2023-11-28T13:02:54.967420+0000","last_became_peered":"2023-11-28T13:02:54.967420+0000","last_unstale":"2023-11-28T13:02:56.152777+0000","last_undegraded":"2023-11-28T13:02:56.152777+0000","last_fullsized":"2023-11-28T13:02:56.152777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:19.756048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.968840+0000","last_change":"2023-11-28T13:02:54.565557+0000","last_active":"2023-11-28T13:02:54.968840+0000","last_peered":"2023-11-28T13:02:54.968840+0000","last_clean":"2023-11-28T13:02:54.968840+0000","last_became_active":"2023-11-28T13:02:54.564651+0000","last_became_peered":"2023-11-28T13:02:54.564651+0000","last_unstale":"2023-11-28T13:02:54.968840+0000","last_undegraded":"2023-11-28T13:02:54.968840+0000","last_fullsized":"2023-11-28T13:02:54.968840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:36.062773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654121+0000","last_change":"2023-11-28T13:02:54.975582+0000","last_active":"2023-11-28T13:02:56.654121+0000","last_peered":"2023-11-28T13:02:56.654121+0000","last_clean":"2023-11-28T13:02:56.654121+0000","last_became_active":"2023-11-28T13:02:54.975328+0000","last_became_peered":"2023-11-28T13:02:54.975328+0000","last_unstale":"2023-11-28T13:02:56.654121+0000","last_undegraded":"2023-11-28T13:02:56.654121+0000","last_fullsized":"2023-11-28T13:02:56.654121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:01:36.435453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.968695+0000","last_change":"2023-11-28T13:02:54.565910+0000","last_active":"2023-11-28T13:02:54.968695+0000","last_peered":"2023-11-28T13:02:54.968695+0000","last_clean":"2023-11-28T13:02:54.968695+0000","last_became_active":"2023-11-28T13:02:54.565445+0000","last_became_peered":"2023-11-28T13:02:54.565445+0000","last_unstale":"2023-11-28T13:02:54.968695+0000","last_undegraded":"2023-11-28T13:02:54.968695+0000","last_fullsized":"2023-11-28T13:02:54.968695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:20.321903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968974+0000","last_change":"2023-11-28T13:02:54.979001+0000","last_active":"2023-11-28T13:02:55.968974+0000","last_peered":"2023-11-28T13:02:55.968974+0000","last_clean":"2023-11-28T13:02:55.968974+0000","last_became_active":"2023-11-28T13:02:54.978737+0000","last_became_peered":"2023-11-28T13:02:54.978737+0000","last_unstale":"2023-11-28T13:02:55.968974+0000","last_undegraded":"2023-11-28T13:02:55.968974+0000","last_fullsized":"2023-11-28T13:02:55.968974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:08.589733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965122+0000","last_change":"2023-11-28T13:02:54.563263+0000","last_active":"2023-11-28T13:02:54.965122+0000","last_peered":"2023-11-28T13:02:54.965122+0000","last_clean":"2023-11-28T13:02:54.965122+0000","last_became_active":"2023-11-28T13:02:54.562391+0000","last_became_peered":"2023-11-28T13:02:54.562391+0000","last_unstale":"2023-11-28T13:02:54.965122+0000","last_undegraded":"2023-11-28T13:02:54.965122+0000","last_fullsized":"2023-11-28T13:02:54.965122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:55.820672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.976657+0000","last_change":"2023-11-28T13:02:54.975878+0000","last_active":"2023-11-28T13:02:54.976657+0000","last_peered":"2023-11-28T13:02:54.976657+0000","last_clean":"2023-11-28T13:02:54.976657+0000","last_became_active":"2023-11-28T13:02:54.975567+0000","last_became_peered":"2023-11-28T13:02:54.975567+0000","last_unstale":"2023-11-28T13:02:54.976657+0000","last_undegraded":"2023-11-28T13:02:54.976657+0000","last_fullsized":"2023-11-28T13:02:54.976657+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:11.238594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968897+0000","last_change":"2023-11-28T13:02:54.574400+0000","last_active":"2023-11-28T13:02:55.968897+0000","last_peered":"2023-11-28T13:02:55.968897+0000","last_clean":"2023-11-28T13:02:55.968897+0000","last_became_active":"2023-11-28T13:02:54.574064+0000","last_became_peered":"2023-11-28T13:02:54.574064+0000","last_unstale":"2023-11-28T13:02:55.968897+0000","last_undegraded":"2023-11-28T13:02:55.968897+0000","last_fullsized":"2023-11-28T13:02:55.968897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:55:33.186957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654415+0000","last_change":"2023-11-28T13:02:54.972511+0000","last_active":"2023-11-28T13:02:56.654415+0000","last_peered":"2023-11-28T13:02:56.654415+0000","last_clean":"2023-11-28T13:02:56.654415+0000","last_became_active":"2023-11-28T13:02:54.972263+0000","last_became_peered":"2023-11-28T13:02:54.972263+0000","last_unstale":"2023-11-28T13:02:56.654415+0000","last_undegraded":"2023-11-28T13:02:56.654415+0000","last_fullsized":"2023-11-28T13:02:56.654415+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:36.364613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654429+0000","last_change":"2023-11-28T13:02:54.574976+0000","last_active":"2023-11-28T13:02:56.654429+0000","last_peered":"2023-11-28T13:02:56.654429+0000","last_clean":"2023-11-28T13:02:56.654429+0000","last_became_active":"2023-11-28T13:02:54.574585+0000","last_became_peered":"2023-11-28T13:02:54.574585+0000","last_unstale":"2023-11-28T13:02:56.654429+0000","last_undegraded":"2023-11-28T13:02:56.654429+0000","last_fullsized":"2023-11-28T13:02:56.654429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:14.566824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.976647+0000","last_change":"2023-11-28T13:02:54.973231+0000","last_active":"2023-11-28T13:02:54.976647+0000","last_peered":"2023-11-28T13:02:54.976647+0000","last_clean":"2023-11-28T13:02:54.976647+0000","last_became_active":"2023-11-28T13:02:54.972974+0000","last_became_peered":"2023-11-28T13:02:54.972974+0000","last_unstale":"2023-11-28T13:02:54.976647+0000","last_undegraded":"2023-11-28T13:02:54.976647+0000","last_fullsized":"2023-11-28T13:02:54.976647+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:57:02.456068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654772+0000","last_change":"2023-11-28T13:02:54.569392+0000","last_active":"2023-11-28T13:02:56.654772+0000","last_peered":"2023-11-28T13:02:56.654772+0000","last_clean":"2023-11-28T13:02:56.654772+0000","last_became_active":"2023-11-28T13:02:54.569061+0000","last_became_peered":"2023-11-28T13:02:54.569061+0000","last_unstale":"2023-11-28T13:02:56.654772+0000","last_undegraded":"2023-11-28T13:02:56.654772+0000","last_fullsized":"2023-11-28T13:02:56.654772+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:53:07.582314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969231+0000","last_change":"2023-11-28T13:02:54.973297+0000","last_active":"2023-11-28T13:02:55.969231+0000","last_peered":"2023-11-28T13:02:55.969231+0000","last_clean":"2023-11-28T13:02:55.969231+0000","last_became_active":"2023-11-28T13:02:54.973015+0000","last_became_peered":"2023-11-28T13:02:54.973015+0000","last_unstale":"2023-11-28T13:02:55.969231+0000","last_undegraded":"2023-11-28T13:02:55.969231+0000","last_fullsized":"2023-11-28T13:02:55.969231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:15.574242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.655005+0000","last_change":"2023-11-28T13:02:54.575482+0000","last_active":"2023-11-28T13:02:56.655005+0000","last_peered":"2023-11-28T13:02:56.655005+0000","last_clean":"2023-11-28T13:02:56.655005+0000","last_became_active":"2023-11-28T13:02:54.575196+0000","last_became_peered":"2023-11-28T13:02:54.575196+0000","last_unstale":"2023-11-28T13:02:56.655005+0000","last_undegraded":"2023-11-28T13:02:56.655005+0000","last_fullsized":"2023-11-28T13:02:56.655005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:13.427513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.655078+0000","last_change":"2023-11-28T13:02:54.976848+0000","last_active":"2023-11-28T13:02:56.655078+0000","last_peered":"2023-11-28T13:02:56.655078+0000","last_clean":"2023-11-28T13:02:56.655078+0000","last_became_active":"2023-11-28T13:02:54.976192+0000","last_became_peered":"2023-11-28T13:02:54.976192+0000","last_unstale":"2023-11-28T13:02:56.655078+0000","last_undegraded":"2023-11-28T13:02:56.655078+0000","last_fullsized":"2023-11-28T13:02:56.655078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:05.211009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.983194+0000","last_change":"2023-11-28T13:02:54.571891+0000","last_active":"2023-11-28T13:02:54.983194+0000","last_peered":"2023-11-28T13:02:54.983194+0000","last_clean":"2023-11-28T13:02:54.983194+0000","last_became_active":"2023-11-28T13:02:54.571427+0000","last_became_peered":"2023-11-28T13:02:54.571427+0000","last_unstale":"2023-11-28T13:02:54.983194+0000","last_undegraded":"2023-11-28T13:02:54.983194+0000","last_fullsized":"2023-11-28T13:02:54.983194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:05:54.448798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653846+0000","last_change":"2023-11-28T13:02:54.980145+0000","last_active":"2023-11-28T13:02:56.653846+0000","last_peered":"2023-11-28T13:02:56.653846+0000","last_clean":"2023-11-28T13:02:56.653846+0000","last_became_active":"2023-11-28T13:02:54.979795+0000","last_became_peered":"2023-11-28T13:02:54.979795+0000","last_unstale":"2023-11-28T13:02:56.653846+0000","last_undegraded":"2023-11-28T13:02:56.653846+0000","last_fullsized":"2023-11-28T13:02:56.653846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:46.036607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654319+0000","last_change":"2023-11-28T13:02:54.573939+0000","last_active":"2023-11-28T13:02:56.654319+0000","last_peered":"2023-11-28T13:02:56.654319+0000","last_clean":"2023-11-28T13:02:56.654319+0000","last_became_active":"2023-11-28T13:02:54.573656+0000","last_became_peered":"2023-11-28T13:02:54.573656+0000","last_unstale":"2023-11-28T13:02:56.654319+0000","last_undegraded":"2023-11-28T13:02:56.654319+0000","last_fullsized":"2023-11-28T13:02:56.654319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:34:26.543872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968064+0000","last_change":"2023-11-28T13:02:54.966758+0000","last_active":"2023-11-28T13:02:55.968064+0000","last_peered":"2023-11-28T13:02:55.968064+0000","last_clean":"2023-11-28T13:02:55.968064+0000","last_became_active":"2023-11-28T13:02:54.965319+0000","last_became_peered":"2023-11-28T13:02:54.965319+0000","last_unstale":"2023-11-28T13:02:55.968064+0000","last_undegraded":"2023-11-28T13:02:55.968064+0000","last_fullsized":"2023-11-28T13:02:55.968064+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:48.223379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968120+0000","last_change":"2023-11-28T13:02:54.575183+0000","last_active":"2023-11-28T13:02:55.968120+0000","last_peered":"2023-11-28T13:02:55.968120+0000","last_clean":"2023-11-28T13:02:55.968120+0000","last_became_active":"2023-11-28T13:02:54.574741+0000","last_became_peered":"2023-11-28T13:02:54.574741+0000","last_unstale":"2023-11-28T13:02:55.968120+0000","last_undegraded":"2023-11-28T13:02:55.968120+0000","last_fullsized":"2023-11-28T13:02:55.968120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:13.427513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969101+0000","last_change":"2023-11-28T13:02:54.974494+0000","last_active":"2023-11-28T13:02:55.969101+0000","last_peered":"2023-11-28T13:02:55.969101+0000","last_clean":"2023-11-28T13:02:55.969101+0000","last_became_active":"2023-11-28T13:02:54.974184+0000","last_became_peered":"2023-11-28T13:02:54.974184+0000","last_unstale":"2023-11-28T13:02:55.969101+0000","last_undegraded":"2023-11-28T13:02:55.969101+0000","last_fullsized":"2023-11-28T13:02:55.969101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:49:28.612886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654901+0000","last_change":"2023-11-28T13:02:54.575271+0000","last_active":"2023-11-28T13:02:56.654901+0000","last_peered":"2023-11-28T13:02:56.654901+0000","last_clean":"2023-11-28T13:02:56.654901+0000","last_became_active":"2023-11-28T13:02:54.575026+0000","last_became_peered":"2023-11-28T13:02:54.575026+0000","last_unstale":"2023-11-28T13:02:56.654901+0000","last_undegraded":"2023-11-28T13:02:56.654901+0000","last_fullsized":"2023-11-28T13:02:56.654901+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:50.857354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654027+0000","last_change":"2023-11-28T13:02:54.977663+0000","last_active":"2023-11-28T13:02:56.654027+0000","last_peered":"2023-11-28T13:02:56.654027+0000","last_clean":"2023-11-28T13:02:56.654027+0000","last_became_active":"2023-11-28T13:02:54.976735+0000","last_became_peered":"2023-11-28T13:02:54.976735+0000","last_unstale":"2023-11-28T13:02:56.654027+0000","last_undegraded":"2023-11-28T13:02:56.654027+0000","last_fullsized":"2023-11-28T13:02:56.654027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:35.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653846+0000","last_change":"2023-11-28T13:02:54.576598+0000","last_active":"2023-11-28T13:02:56.653846+0000","last_peered":"2023-11-28T13:02:56.653846+0000","last_clean":"2023-11-28T13:02:56.653846+0000","last_became_active":"2023-11-28T13:02:54.576292+0000","last_became_peered":"2023-11-28T13:02:54.576292+0000","last_unstale":"2023-11-28T13:02:56.653846+0000","last_undegraded":"2023-11-28T13:02:56.653846+0000","last_fullsized":"2023-11-28T13:02:56.653846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:13.993001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653728+0000","last_change":"2023-11-28T13:02:54.979265+0000","last_active":"2023-11-28T13:02:56.653728+0000","last_peered":"2023-11-28T13:02:56.653728+0000","last_clean":"2023-11-28T13:02:56.653728+0000","last_became_active":"2023-11-28T13:02:54.978935+0000","last_became_peered":"2023-11-28T13:02:54.978935+0000","last_unstale":"2023-11-28T13:02:56.653728+0000","last_undegraded":"2023-11-28T13:02:56.653728+0000","last_fullsized":"2023-11-28T13:02:56.653728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:18.297439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968732+0000","last_change":"2023-11-28T13:02:54.574817+0000","last_active":"2023-11-28T13:02:55.968732+0000","last_peered":"2023-11-28T13:02:55.968732+0000","last_clean":"2023-11-28T13:02:55.968732+0000","last_became_active":"2023-11-28T13:02:54.574382+0000","last_became_peered":"2023-11-28T13:02:54.574382+0000","last_unstale":"2023-11-28T13:02:55.968732+0000","last_undegraded":"2023-11-28T13:02:55.968732+0000","last_fullsized":"2023-11-28T13:02:55.968732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:34:26.543872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152942+0000","last_change":"2023-11-28T13:02:54.970164+0000","last_active":"2023-11-28T13:02:56.152942+0000","last_peered":"2023-11-28T13:02:56.152942+0000","last_clean":"2023-11-28T13:02:56.152942+0000","last_became_active":"2023-11-28T13:02:54.969358+0000","last_became_peered":"2023-11-28T13:02:54.969358+0000","last_unstale":"2023-11-28T13:02:56.152942+0000","last_undegraded":"2023-11-28T13:02:56.152942+0000","last_fullsized":"2023-11-28T13:02:56.152942+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:23:29.634193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968648+0000","last_change":"2023-11-28T13:02:54.573909+0000","last_active":"2023-11-28T13:02:55.968648+0000","last_peered":"2023-11-28T13:02:55.968648+0000","last_clean":"2023-11-28T13:02:55.968648+0000","last_became_active":"2023-11-28T13:02:54.573635+0000","last_became_peered":"2023-11-28T13:02:54.573635+0000","last_unstale":"2023-11-28T13:02:55.968648+0000","last_undegraded":"2023-11-28T13:02:55.968648+0000","last_fullsized":"2023-11-28T13:02:55.968648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:14.566824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968511+0000","last_change":"2023-11-28T13:02:54.964265+0000","last_active":"2023-11-28T13:02:55.968511+0000","last_peered":"2023-11-28T13:02:55.968511+0000","last_clean":"2023-11-28T13:02:55.968511+0000","last_became_active":"2023-11-28T13:02:54.963983+0000","last_became_peered":"2023-11-28T13:02:54.963983+0000","last_unstale":"2023-11-28T13:02:55.968511+0000","last_undegraded":"2023-11-28T13:02:55.968511+0000","last_fullsized":"2023-11-28T13:02:55.968511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:21.723540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654399+0000","last_change":"2023-11-28T13:02:54.571685+0000","last_active":"2023-11-28T13:02:56.654399+0000","last_peered":"2023-11-28T13:02:56.654399+0000","last_clean":"2023-11-28T13:02:56.654399+0000","last_became_active":"2023-11-28T13:02:54.571375+0000","last_became_peered":"2023-11-28T13:02:54.571375+0000","last_unstale":"2023-11-28T13:02:56.654399+0000","last_undegraded":"2023-11-28T13:02:56.654399+0000","last_fullsized":"2023-11-28T13:02:56.654399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:45.383406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969336+0000","last_change":"2023-11-28T13:02:54.978652+0000","last_active":"2023-11-28T13:02:55.969336+0000","last_peered":"2023-11-28T13:02:55.969336+0000","last_clean":"2023-11-28T13:02:55.969336+0000","last_became_active":"2023-11-28T13:02:54.978214+0000","last_became_peered":"2023-11-28T13:02:54.978214+0000","last_unstale":"2023-11-28T13:02:55.969336+0000","last_undegraded":"2023-11-28T13:02:55.969336+0000","last_fullsized":"2023-11-28T13:02:55.969336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:31:15.000419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.960507+0000","last_change":"2023-11-28T13:02:54.572070+0000","last_active":"2023-11-28T13:02:54.960507+0000","last_peered":"2023-11-28T13:02:54.960507+0000","last_clean":"2023-11-28T13:02:54.960507+0000","last_became_active":"2023-11-28T13:02:54.571624+0000","last_became_peered":"2023-11-28T13:02:54.571624+0000","last_unstale":"2023-11-28T13:02:54.960507+0000","last_undegraded":"2023-11-28T13:02:54.960507+0000","last_fullsized":"2023-11-28T13:02:54.960507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:24.750211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968891+0000","last_change":"2023-11-28T13:02:54.978080+0000","last_active":"2023-11-28T13:02:55.968891+0000","last_peered":"2023-11-28T13:02:55.968891+0000","last_clean":"2023-11-28T13:02:55.968891+0000","last_became_active":"2023-11-28T13:02:54.977364+0000","last_became_peered":"2023-11-28T13:02:54.977364+0000","last_unstale":"2023-11-28T13:02:55.968891+0000","last_undegraded":"2023-11-28T13:02:55.968891+0000","last_fullsized":"2023-11-28T13:02:55.968891+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:31.551434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654411+0000","last_change":"2023-11-28T13:02:54.574565+0000","last_active":"2023-11-28T13:02:56.654411+0000","last_peered":"2023-11-28T13:02:56.654411+0000","last_clean":"2023-11-28T13:02:56.654411+0000","last_became_active":"2023-11-28T13:02:54.574145+0000","last_became_peered":"2023-11-28T13:02:54.574145+0000","last_unstale":"2023-11-28T13:02:56.654411+0000","last_undegraded":"2023-11-28T13:02:56.654411+0000","last_fullsized":"2023-11-28T13:02:56.654411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:16.453796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654003+0000","last_change":"2023-11-28T13:02:54.978426+0000","last_active":"2023-11-28T13:02:56.654003+0000","last_peered":"2023-11-28T13:02:56.654003+0000","last_clean":"2023-11-28T13:02:56.654003+0000","last_became_active":"2023-11-28T13:02:54.978156+0000","last_became_peered":"2023-11-28T13:02:54.978156+0000","last_unstale":"2023-11-28T13:02:56.654003+0000","last_undegraded":"2023-11-28T13:02:56.654003+0000","last_fullsized":"2023-11-28T13:02:56.654003+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:45:41.711431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965515+0000","last_change":"2023-11-28T13:02:54.565731+0000","last_active":"2023-11-28T13:02:54.965515+0000","last_peered":"2023-11-28T13:02:54.965515+0000","last_clean":"2023-11-28T13:02:54.965515+0000","last_became_active":"2023-11-28T13:02:54.565059+0000","last_became_peered":"2023-11-28T13:02:54.565059+0000","last_unstale":"2023-11-28T13:02:54.965515+0000","last_undegraded":"2023-11-28T13:02:54.965515+0000","last_fullsized":"2023-11-28T13:02:54.965515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:36.858578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653664+0000","last_change":"2023-11-28T13:02:54.975476+0000","last_active":"2023-11-28T13:02:56.653664+0000","last_peered":"2023-11-28T13:02:56.653664+0000","last_clean":"2023-11-28T13:02:56.653664+0000","last_became_active":"2023-11-28T13:02:54.975273+0000","last_became_peered":"2023-11-28T13:02:54.975273+0000","last_unstale":"2023-11-28T13:02:56.653664+0000","last_undegraded":"2023-11-28T13:02:56.653664+0000","last_fullsized":"2023-11-28T13:02:56.653664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:41.576824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965223+0000","last_change":"2023-11-28T13:02:54.565314+0000","last_active":"2023-11-28T13:02:54.965223+0000","last_peered":"2023-11-28T13:02:54.965223+0000","last_clean":"2023-11-28T13:02:54.965223+0000","last_became_active":"2023-11-28T13:02:54.563242+0000","last_became_peered":"2023-11-28T13:02:54.563242+0000","last_unstale":"2023-11-28T13:02:54.965223+0000","last_undegraded":"2023-11-28T13:02:54.965223+0000","last_fullsized":"2023-11-28T13:02:54.965223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:36.187142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.152895+0000","last_change":"2023-11-28T13:02:54.970716+0000","last_active":"2023-11-28T13:02:56.152895+0000","last_peered":"2023-11-28T13:02:56.152895+0000","last_clean":"2023-11-28T13:02:56.152895+0000","last_became_active":"2023-11-28T13:02:54.970465+0000","last_became_peered":"2023-11-28T13:02:54.970465+0000","last_unstale":"2023-11-28T13:02:56.152895+0000","last_undegraded":"2023-11-28T13:02:56.152895+0000","last_fullsized":"2023-11-28T13:02:56.152895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:12.376175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.959325+0000","last_change":"2023-11-28T13:02:54.568037+0000","last_active":"2023-11-28T13:02:54.959325+0000","last_peered":"2023-11-28T13:02:54.959325+0000","last_clean":"2023-11-28T13:02:54.959325+0000","last_became_active":"2023-11-28T13:02:54.566895+0000","last_became_peered":"2023-11-28T13:02:54.566895+0000","last_unstale":"2023-11-28T13:02:54.959325+0000","last_undegraded":"2023-11-28T13:02:54.959325+0000","last_fullsized":"2023-11-28T13:02:54.959325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:34:44.950034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654632+0000","last_change":"2023-11-28T13:02:54.990785+0000","last_active":"2023-11-28T13:02:56.654632+0000","last_peered":"2023-11-28T13:02:56.654632+0000","last_clean":"2023-11-28T13:02:56.654632+0000","last_became_active":"2023-11-28T13:02:54.990473+0000","last_became_peered":"2023-11-28T13:02:54.990473+0000","last_unstale":"2023-11-28T13:02:56.654632+0000","last_undegraded":"2023-11-28T13:02:56.654632+0000","last_fullsized":"2023-11-28T13:02:56.654632+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:48.223379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968647+0000","last_change":"2023-11-28T13:02:54.575460+0000","last_active":"2023-11-28T13:02:55.968647+0000","last_peered":"2023-11-28T13:02:55.968647+0000","last_clean":"2023-11-28T13:02:55.968647+0000","last_became_active":"2023-11-28T13:02:54.575157+0000","last_became_peered":"2023-11-28T13:02:54.575157+0000","last_unstale":"2023-11-28T13:02:55.968647+0000","last_undegraded":"2023-11-28T13:02:55.968647+0000","last_fullsized":"2023-11-28T13:02:55.968647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:24.555799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.960024+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:02:54.960024+0000","last_peered":"2023-11-28T13:02:54.960024+0000","last_clean":"2023-11-28T13:02:54.960024+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:02:54.960024+0000","last_undegraded":"2023-11-28T13:02:54.960024+0000","last_fullsized":"2023-11-28T13:02:54.960024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969101+0000","last_change":"2023-11-28T13:02:54.978861+0000","last_active":"2023-11-28T13:02:55.969101+0000","last_peered":"2023-11-28T13:02:55.969101+0000","last_clean":"2023-11-28T13:02:55.969101+0000","last_became_active":"2023-11-28T13:02:54.978374+0000","last_became_peered":"2023-11-28T13:02:54.978374+0000","last_unstale":"2023-11-28T13:02:55.969101+0000","last_undegraded":"2023-11-28T13:02:55.969101+0000","last_fullsized":"2023-11-28T13:02:55.969101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:10.520996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653875+0000","last_change":"2023-11-28T13:02:54.572150+0000","last_active":"2023-11-28T13:02:56.653875+0000","last_peered":"2023-11-28T13:02:56.653875+0000","last_clean":"2023-11-28T13:02:56.653875+0000","last_became_active":"2023-11-28T13:02:54.571846+0000","last_became_peered":"2023-11-28T13:02:54.571846+0000","last_unstale":"2023-11-28T13:02:56.653875+0000","last_undegraded":"2023-11-28T13:02:56.653875+0000","last_fullsized":"2023-11-28T13:02:56.653875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.733701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.959518+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:02:54.959518+0000","last_peered":"2023-11-28T13:02:54.959518+0000","last_clean":"2023-11-28T13:02:54.959518+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:02:54.959518+0000","last_undegraded":"2023-11-28T13:02:54.959518+0000","last_fullsized":"2023-11-28T13:02:54.959518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968178+0000","last_change":"2023-11-28T13:02:54.969468+0000","last_active":"2023-11-28T13:02:55.968178+0000","last_peered":"2023-11-28T13:02:55.968178+0000","last_clean":"2023-11-28T13:02:55.968178+0000","last_became_active":"2023-11-28T13:02:54.969194+0000","last_became_peered":"2023-11-28T13:02:54.969194+0000","last_unstale":"2023-11-28T13:02:55.968178+0000","last_undegraded":"2023-11-28T13:02:55.968178+0000","last_fullsized":"2023-11-28T13:02:55.968178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:33.775943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.968213+0000","last_change":"2023-11-28T13:02:54.575006+0000","last_active":"2023-11-28T13:02:55.968213+0000","last_peered":"2023-11-28T13:02:55.968213+0000","last_clean":"2023-11-28T13:02:55.968213+0000","last_became_active":"2023-11-28T13:02:54.574370+0000","last_became_peered":"2023-11-28T13:02:54.574370+0000","last_unstale":"2023-11-28T13:02:55.968213+0000","last_undegraded":"2023-11-28T13:02:55.968213+0000","last_fullsized":"2023-11-28T13:02:55.968213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:11:50.857354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.966385+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:02:54.966385+0000","last_peered":"2023-11-28T13:02:54.966385+0000","last_clean":"2023-11-28T13:02:54.966385+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:02:54.966385+0000","last_undegraded":"2023-11-28T13:02:54.966385+0000","last_fullsized":"2023-11-28T13:02:54.966385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969566+0000","last_change":"2023-11-28T13:02:54.977949+0000","last_active":"2023-11-28T13:02:55.969566+0000","last_peered":"2023-11-28T13:02:55.969566+0000","last_clean":"2023-11-28T13:02:55.969566+0000","last_became_active":"2023-11-28T13:02:54.977680+0000","last_became_peered":"2023-11-28T13:02:54.977680+0000","last_unstale":"2023-11-28T13:02:55.969566+0000","last_undegraded":"2023-11-28T13:02:55.969566+0000","last_fullsized":"2023-11-28T13:02:55.969566+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:29.898926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.964771+0000","last_change":"2023-11-28T13:02:54.564196+0000","last_active":"2023-11-28T13:02:54.964771+0000","last_peered":"2023-11-28T13:02:54.964771+0000","last_clean":"2023-11-28T13:02:54.964771+0000","last_became_active":"2023-11-28T13:02:54.563920+0000","last_became_peered":"2023-11-28T13:02:54.563920+0000","last_unstale":"2023-11-28T13:02:54.964771+0000","last_undegraded":"2023-11-28T13:02:54.964771+0000","last_fullsized":"2023-11-28T13:02:54.964771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:57.576385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.959297+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:02:54.959297+0000","last_peered":"2023-11-28T13:02:54.959297+0000","last_clean":"2023-11-28T13:02:54.959297+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:02:54.959297+0000","last_undegraded":"2023-11-28T13:02:54.959297+0000","last_fullsized":"2023-11-28T13:02:54.959297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.979828+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:02:54.979828+0000","last_peered":"2023-11-28T13:02:54.979828+0000","last_clean":"2023-11-28T13:02:54.979828+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:02:54.979828+0000","last_undegraded":"2023-11-28T13:02:54.979828+0000","last_fullsized":"2023-11-28T13:02:54.979828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653826+0000","last_change":"2023-11-28T13:02:54.974128+0000","last_active":"2023-11-28T13:02:56.653826+0000","last_peered":"2023-11-28T13:02:56.653826+0000","last_clean":"2023-11-28T13:02:56.653826+0000","last_became_active":"2023-11-28T13:02:54.973776+0000","last_became_peered":"2023-11-28T13:02:54.973776+0000","last_unstale":"2023-11-28T13:02:56.653826+0000","last_undegraded":"2023-11-28T13:02:56.653826+0000","last_fullsized":"2023-11-28T13:02:56.653826+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:22:49.099601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.653780+0000","last_change":"2023-11-28T13:02:54.564097+0000","last_active":"2023-11-28T13:02:56.653780+0000","last_peered":"2023-11-28T13:02:56.653780+0000","last_clean":"2023-11-28T13:02:56.653780+0000","last_became_active":"2023-11-28T13:02:54.563816+0000","last_became_peered":"2023-11-28T13:02:54.563816+0000","last_unstale":"2023-11-28T13:02:56.653780+0000","last_undegraded":"2023-11-28T13:02:56.653780+0000","last_fullsized":"2023-11-28T13:02:56.653780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:52.277147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.979742+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:02:54.979742+0000","last_peered":"2023-11-28T13:02:54.979742+0000","last_clean":"2023-11-28T13:02:54.979742+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:02:54.979742+0000","last_undegraded":"2023-11-28T13:02:54.979742+0000","last_fullsized":"2023-11-28T13:02:54.979742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969476+0000","last_change":"2023-11-28T13:02:54.976703+0000","last_active":"2023-11-28T13:02:55.969476+0000","last_peered":"2023-11-28T13:02:55.969476+0000","last_clean":"2023-11-28T13:02:55.969476+0000","last_became_active":"2023-11-28T13:02:54.976435+0000","last_became_peered":"2023-11-28T13:02:54.976435+0000","last_unstale":"2023-11-28T13:02:55.969476+0000","last_undegraded":"2023-11-28T13:02:55.969476+0000","last_fullsized":"2023-11-28T13:02:55.969476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:17.461214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654331+0000","last_change":"2023-11-28T13:02:54.575909+0000","last_active":"2023-11-28T13:02:56.654331+0000","last_peered":"2023-11-28T13:02:56.654331+0000","last_clean":"2023-11-28T13:02:56.654331+0000","last_became_active":"2023-11-28T13:02:54.575752+0000","last_became_peered":"2023-11-28T13:02:54.575752+0000","last_unstale":"2023-11-28T13:02:56.654331+0000","last_undegraded":"2023-11-28T13:02:56.654331+0000","last_fullsized":"2023-11-28T13:02:56.654331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:50.018236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.980649+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:02:54.980649+0000","last_peered":"2023-11-28T13:02:54.980649+0000","last_clean":"2023-11-28T13:02:54.980649+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:02:54.980649+0000","last_undegraded":"2023-11-28T13:02:54.980649+0000","last_fullsized":"2023-11-28T13:02:54.980649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654160+0000","last_change":"2023-11-28T13:02:54.978449+0000","last_active":"2023-11-28T13:02:56.654160+0000","last_peered":"2023-11-28T13:02:56.654160+0000","last_clean":"2023-11-28T13:02:56.654160+0000","last_became_active":"2023-11-28T13:02:54.978140+0000","last_became_peered":"2023-11-28T13:02:54.978140+0000","last_unstale":"2023-11-28T13:02:56.654160+0000","last_undegraded":"2023-11-28T13:02:56.654160+0000","last_fullsized":"2023-11-28T13:02:56.654160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:46.948476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:56.654908+0000","last_change":"2023-11-28T13:02:54.574890+0000","last_active":"2023-11-28T13:02:56.654908+0000","last_peered":"2023-11-28T13:02:56.654908+0000","last_clean":"2023-11-28T13:02:56.654908+0000","last_became_active":"2023-11-28T13:02:54.574114+0000","last_became_peered":"2023-11-28T13:02:54.574114+0000","last_unstale":"2023-11-28T13:02:56.654908+0000","last_undegraded":"2023-11-28T13:02:56.654908+0000","last_fullsized":"2023-11-28T13:02:56.654908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:24.555799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.961838+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:02:54.961838+0000","last_peered":"2023-11-28T13:02:54.961838+0000","last_clean":"2023-11-28T13:02:54.961838+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:02:54.961838+0000","last_undegraded":"2023-11-28T13:02:54.961838+0000","last_fullsized":"2023-11-28T13:02:54.961838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-28T13:02:55.969411+0000","last_change":"2023-11-28T13:02:54.978477+0000","last_active":"2023-11-28T13:02:55.969411+0000","last_peered":"2023-11-28T13:02:55.969411+0000","last_clean":"2023-11-28T13:02:55.969411+0000","last_became_active":"2023-11-28T13:02:54.977877+0000","last_became_peered":"2023-11-28T13:02:54.977877+0000","last_unstale":"2023-11-28T13:02:55.969411+0000","last_undegraded":"2023-11-28T13:02:55.969411+0000","last_fullsized":"2023-11-28T13:02:55.969411+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:54.546564+0000","last_clean_scrub_stamp":"2023-11-28T13:02:54.546564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:56.846859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.965377+0000","last_change":"2023-11-28T13:02:54.565033+0000","last_active":"2023-11-28T13:02:54.965377+0000","last_peered":"2023-11-28T13:02:54.965377+0000","last_clean":"2023-11-28T13:02:54.965377+0000","last_became_active":"2023-11-28T13:02:54.564748+0000","last_became_peered":"2023-11-28T13:02:54.564748+0000","last_unstale":"2023-11-28T13:02:54.965377+0000","last_undegraded":"2023-11-28T13:02:54.965377+0000","last_fullsized":"2023-11-28T13:02:54.965377+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:53.539146+0000","last_clean_scrub_stamp":"2023-11-28T13:02:53.539146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:58.581060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T13:02:54.967405+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:02:54.967405+0000","last_peered":"2023-11-28T13:02:54.967405+0000","last_clean":"2023-11-28T13:02:54.967405+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:02:54.967405+0000","last_undegraded":"2023-11-28T13:02:54.967405+0000","last_fullsized":"2023-11-28T13:02:54.967405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9764,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362076,"statfs":{"total":96636764160,"available":96626765824,"internally_reserved":0,"allocated":749568,"data_stored":328140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.497,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.63,"15min":0.67800000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.498,"5min":8.218,"15min":8.218},"last":0.36299999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39700000000000002,"15min":0.40400000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.499,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.40400000000000003,"15min":0.41299999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.47599999999999998,"5min":1.355,"15min":1.355},"last":0.377}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.871,"15min":0.95199999999999996},"min":{"1min":0.27300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55400000000000005,"5min":13.384,"15min":13.384},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.58999999999999997,"15min":0.62},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52300000000000002,"5min":5.226,"15min":5.226},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84099999999999997,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.75900000000000001,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.79400000000000004,"5min":5.3609999999999998,"15min":5.3609999999999998},"last":0.50600000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.63400000000000001,"15min":0.64100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.85399999999999998,"15min":0.89700000000000002},"min":{"1min":0.44500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81899999999999995,"5min":8.1579999999999995,"15min":8.1579999999999995},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.51300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":5.3209999999999997,"15min":5.3209999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.67000000000000004,"15min":0.67700000000000005},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93799999999999994,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.88300000000000001,"15min":0.92900000000000005},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.78900000000000003,"5min":8.1029999999999998,"15min":8.1029999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.79000000000000004,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81799999999999995,"5min":5.3849999999999998,"15min":5.3849999999999998},"last":0.56999999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9380,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362460,"statfs":{"total":96636764160,"available":96627159040,"internally_reserved":0,"allocated":749568,"data_stored":328140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.248},{"interface":"front","average":{"1min":0.374,"5min":0.34100000000000003,"15min":0.33500000000000002},"min":{"1min":0.216,"5min":0.187,"15min":0.187},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.37},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.438}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.39400000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.59199999999999997,"15min":0.57899999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.48399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.621,"15min":0.60699999999999998},"min":{"1min":0.41699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.56200000000000006}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.42899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.49199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9584,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362256,"statfs":{"total":96636764160,"available":96626950144,"internally_reserved":0,"allocated":761856,"data_stored":329688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.45000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.40400000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.504},{"interface":"front","average":{"1min":0.501,"5min":0.502,"15min":0.502},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.432}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.753,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46300000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14756,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":13440,"kb_avail":94357084,"statfs":{"total":96636764160,"available":96621654016,"internally_reserved":0,"allocated":1339392,"data_stored":917739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13762560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32200000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.49199999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.221},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.85299999999999998,"15min":0.93999999999999995},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.54200000000000004,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.85499999999999998,"15min":0.93799999999999994},"min":{"1min":0.251,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61399999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.86199999999999999,"15min":0.93899999999999995},"min":{"1min":0.27700000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":14.997999999999999,"15min":14.997999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.98799999999999999,"15min":1.091},"min":{"1min":0.247,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.58399999999999996,"5min":19.045999999999999,"15min":19.045999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39400000000000002,"15min":0.39200000000000002},"min":{"1min":0.28299999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.69099999999999995,"15min":0.70899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94299999999999995,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.379},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.218,"15min":1.3280000000000001},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68999999999999995,"5min":18.995000000000001,"15min":18.995000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.204,"15min":1.3109999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76100000000000001,"5min":19.02,"15min":19.02},"last":0.443},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.503}]},{"osd":6,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":4.2359999999999998,"15min":4.2359999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.748,"5min":4.1150000000000002,"15min":4.1150000000000002},"last":0.41899999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.83099999999999996,"5min":4.1539999999999999,"15min":4.1539999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.2250000000000001,"15min":1.3320000000000001},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.81799999999999995,"5min":18.957999999999998,"15min":18.957999999999998},"last":0.40999999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10808,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94361032,"statfs":{"total":96636764160,"available":96625696768,"internally_reserved":0,"allocated":1359872,"data_stored":919268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":1.415,"15min":1.415},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":1.867,"15min":1.867},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.63100000000000001,"15min":0.60299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.7830000000000004,"5min":4.7830000000000004,"15min":4.7830000000000004},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58799999999999997,"15min":0.59699999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71399999999999997,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.505},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.375}]},{"osd":3,"last update":"Tue Nov 28 13:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2749999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.8440000000000003,"5min":4.8440000000000003,"15min":4.8440000000000003},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.53600000000000003,"15min":0.51200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.87,"5min":1.528,"15min":1.528},"last":0.56100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68200000000000005,"15min":0.67000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.625}]}]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9512,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362328,"statfs":{"total":96636764160,"available":96627023872,"internally_reserved":0,"allocated":753664,"data_stored":328162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51700000000000002,"15min":0.51400000000000001},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76300000000000001,"5min":1.419,"15min":1.419},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66500000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.41699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72099999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.40300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.79900000000000004,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.68899999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55700000000000005}]},{"osd":4,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47899999999999998},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65800000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.67100000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40799999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73399999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:02:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70799999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.748,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.52900000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9384,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362456,"statfs":{"total":96636764160,"available":96627154944,"internally_reserved":0,"allocated":753664,"data_stored":328174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.44,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.56399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.504},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.441,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.45100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46800000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48999999999999999,"15min":0.48399999999999999},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.5,"15min":0.49299999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.57699999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.622},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.60899999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9472,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362368,"statfs":{"total":96636764160,"available":96627064832,"internally_reserved":0,"allocated":778240,"data_stored":330436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.20999999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.45900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.253},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.33600000000000002,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.441,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52600000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.315},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.46800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39200000000000002,"15min":0.39600000000000002},"min":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54900000000000004,"5min":0.752,"15min":0.752},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.51600000000000001,"5min":0.875,"15min":0.875},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73099999999999998,"5min":1.044,"15min":1.044},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73899999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.499},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75900000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46999999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71299999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.68400000000000005,"5min":1.089,"15min":1.089},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72499999999999998,"5min":1.202,"15min":1.202},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61799999999999999,"15min":0.628},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.74399999999999999,"5min":1.02,"15min":1.02},"last":0.46500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:03:03.924 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:03:03.925 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:03:04.087 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.086+0000 7ff3932d0700 1 Processor -- start 2023-11-28T13:03:04.087 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.086+0000 7ff3932d0700 1 -- start start 2023-11-28T13:03:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff3932d0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff38c06dee0 0x7ff38c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff3932d0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c160540 0x7ff38c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff3932d0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c15d060 con 0x7ff38c071040 2023-11-28T13:03:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff3932d0700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c15d340 con 0x7ff38c06dee0 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff3932d0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c15d620 con 0x7ff38c160540 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff39086b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff38c06dee0 0x7ff38c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff39106c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff38c071040 0x7ff38c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60466/0 (socket says 172.21.15.153:60466) 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff39106c700 1 -- 172.21.15.153:0/4020221825 learned_addr learned my addr 172.21.15.153:0/4020221825 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff39086b700 1 -- 172.21.15.153:0/4020221825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c160540 msgr2=0x7ff38c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff39086b700 1 --2- 172.21.15.153:0/4020221825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c160540 0x7ff38c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff39086b700 1 -- 172.21.15.153:0/4020221825 >> v1:172.21.15.153:6789/0 conn(0x7ff38c071040 legacy=0x7ff38c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.090 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff39086b700 1 -- 172.21.15.153:0/4020221825 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c15d900 con 0x7ff38c06dee0 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.087+0000 7ff383fff700 1 -- 172.21.15.153:0/4020221825 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846671206 0 0) 0x7ff38c15d060 con 0x7ff38c071040 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff39086b700 1 --2- 172.21.15.153:0/4020221825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c06dee0 0x7ff38c06e330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff388009840 tx=0x7ff388009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a955fcb6732f2b24 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff383fff700 1 -- 172.21.15.153:0/4020221825 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388011830 con 0x7ff38c06dee0 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff383fff700 1 -- 172.21.15.153:0/4020221825 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff388011990 con 0x7ff38c06dee0 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff383fff700 1 -- 172.21.15.153:0/4020221825 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388011d20 con 0x7ff38c06dee0 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 -- 172.21.15.153:0/4020221825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c06dee0 msgr2=0x7ff38c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 --2- 172.21.15.153:0/4020221825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c06dee0 0x7ff38c06e330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff388009840 tx=0x7ff388009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 -- 172.21.15.153:0/4020221825 shutdown_connections 2023-11-28T13:03:04.091 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 --2- 172.21.15.153:0/4020221825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c160540 0x7ff38c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 --2- 172.21.15.153:0/4020221825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c06dee0 0x7ff38c06e330 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 -- 172.21.15.153:0/4020221825 >> 172.21.15.153:0/4020221825 conn(0x7ff38c0d0000 msgr2=0x7ff38c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:04.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 -- 172.21.15.153:0/4020221825 shutdown_connections 2023-11-28T13:03:04.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.088+0000 7ff3932d0700 1 -- 172.21.15.153:0/4020221825 wait complete. 2023-11-28T13:03:04.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.089+0000 7ff3932d0700 1 Processor -- start 2023-11-28T13:03:04.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.089+0000 7ff3932d0700 1 -- start start 2023-11-28T13:03:04.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff3932d0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c06dee0 0x7ff38c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff3932d0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c071040 0x7ff38c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff3932d0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c160540 0x7ff38c0c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff3932d0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0c4860 con 0x7ff38c160540 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff3932d0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0c4b40 con 0x7ff38c071040 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff3932d0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0c4e20 con 0x7ff38c06dee0 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff39186d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c160540 0x7ff38c0c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff39186d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c160540 0x7ff38c0c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42312/0 (socket says 172.21.15.153:42312) 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff39186d700 1 -- 172.21.15.153:0/1632063017 learned_addr learned my addr 172.21.15.153:0/1632063017 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff39086b700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c071040 0x7ff38c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.090+0000 7ff39106c700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c06dee0 0x7ff38c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff39186d700 1 -- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c06dee0 msgr2=0x7ff38c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff39186d700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c06dee0 0x7ff38c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff39186d700 1 -- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c071040 msgr2=0x7ff38c0b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff39186d700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c071040 0x7ff38c0b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff39186d700 1 -- 172.21.15.153:0/1632063017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3880094a0 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff39186d700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c160540 0x7ff38c0c3b10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff38400b970 tx=0x7ff38400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff381ffb700 1 -- 172.21.15.153:0/1632063017 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38400c9b0 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff381ffb700 1 -- 172.21.15.153:0/1632063017 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38400cb30 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c15ccd0 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff381ffb700 1 -- 172.21.15.153:0/1632063017 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384014440 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.091+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c06e870 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.092+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c06a270 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.092+0000 7ff381ffb700 1 -- 172.21.15.153:0/1632063017 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff38401b030 con 0x7ff38c160540 2023-11-28T13:03:04.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.093+0000 7ff381ffb700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3700682f0 0x7ff37006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.093+0000 7ff381ffb700 1 -- 172.21.15.153:0/1632063017 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff384085910 con 0x7ff38c160540 2023-11-28T13:03:04.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.093+0000 7ff39106c700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3700682f0 0x7ff37006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.093+0000 7ff39106c700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3700682f0 0x7ff37006a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000f60 tx=0x7ff37c007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.097+0000 7ff381ffb700 1 -- 172.21.15.153:0/1632063017 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff384052170 con 0x7ff38c160540 2023-11-28T13:03:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.355+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7ff38c0720a0 con 0x7ff38c160540 2023-11-28T13:03:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.359+0000 7ff381ffb700 1 -- 172.21.15.153:0/1632063017 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff3840190e0 con 0x7ff38c160540 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3700682f0 msgr2=0x7ff37006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3700682f0 0x7ff37006a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000f60 tx=0x7ff37c007c90 comp rx=0 tx=0).stop 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c160540 msgr2=0x7ff38c0c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c160540 0x7ff38c0c3b10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff38400b970 tx=0x7ff38400bd30 comp rx=0 tx=0).stop 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 shutdown_connections 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3700682f0 0x7ff37006a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff38c160540 0x7ff38c0c3b10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c06dee0 0x7ff38c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 --2- 172.21.15.153:0/1632063017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c071040 0x7ff38c0b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 >> 172.21.15.153:0/1632063017 conn(0x7ff38c0d0000 msgr2=0x7ff38c0af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:04.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 shutdown_connections 2023-11-28T13:03:04.363 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.361+0000 7ff3932d0700 1 -- 172.21.15.153:0/1632063017 wait complete. 2023-11-28T13:03:04.363 INFO:teuthology.orchestra.run.smithi153.stderr:updated caps for client.0 2023-11-28T13:03:04.374 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:04.494 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.493+0000 7f2bde102700 1 Processor -- start 2023-11-28T13:03:04.495 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bde102700 1 -- start start 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bde102700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2bd806f850 0x7f2bd806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bde102700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd813ff00 con 0x7f2bd8140950 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bde102700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd81404c0 con 0x7f2bd80cd8c0 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bde102700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd8140dc0 con 0x7f2bd806f850 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bd7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2bd806f850 0x7f2bd806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bd6ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2bd8140950 0x7f2bd80b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60482/0 (socket says 172.21.15.153:60482) 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bd7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2bd806f850 0x7f2bd806fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42320/0 (socket says 172.21.15.153:42320) 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.494+0000 7f2bd6ffd700 1 -- 172.21.15.153:0/2577864741 learned_addr learned my addr 172.21.15.153:0/2577864741 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.495+0000 7f2bd7fff700 1 -- 172.21.15.153:0/2577864741 >> v1:172.21.15.177:6789/0 conn(0x7f2bd80cd8c0 legacy=0x7f2bd80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:04.497 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.495+0000 7f2bd7fff700 1 -- 172.21.15.153:0/2577864741 >> v1:172.21.15.153:6789/0 conn(0x7f2bd8140950 legacy=0x7f2bd80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.495+0000 7f2bd7fff700 1 -- 172.21.15.153:0/2577864741 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd815aa80 con 0x7f2bd806f850 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.495+0000 7f2bd7fff700 1 --2- 172.21.15.153:0/2577864741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd806f850 0x7f2bd806fcf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009d90 tx=0x7f2bd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8d9159bc4edb0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.495+0000 7f2bd67fc700 1 -- 172.21.15.153:0/2577864741 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000e070 con 0x7f2bd806f850 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.495+0000 7f2bd67fc700 1 -- 172.21.15.153:0/2577864741 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd000bb90 con 0x7f2bd806f850 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.495+0000 7f2bd67fc700 1 -- 172.21.15.153:0/2577864741 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0003a20 con 0x7f2bd806f850 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.496+0000 7f2bde102700 1 -- 172.21.15.153:0/2577864741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd806f850 msgr2=0x7f2bd806fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.496+0000 7f2bde102700 1 --2- 172.21.15.153:0/2577864741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd806f850 0x7f2bd806fcf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009d90 tx=0x7f2bd000b330 comp rx=0 tx=0).stop 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.496+0000 7f2bde102700 1 -- 172.21.15.153:0/2577864741 shutdown_connections 2023-11-28T13:03:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.496+0000 7f2bde102700 1 --2- 172.21.15.153:0/2577864741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd806f850 0x7f2bd806fcf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.496+0000 7f2bde102700 1 -- 172.21.15.153:0/2577864741 >> 172.21.15.153:0/2577864741 conn(0x7f2bd806aed0 msgr2=0x7f2bd806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:04.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.496+0000 7f2bde102700 1 -- 172.21.15.153:0/2577864741 shutdown_connections 2023-11-28T13:03:04.499 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.496+0000 7f2bde102700 1 -- 172.21.15.153:0/2577864741 wait complete. 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.497+0000 7f2bde102700 1 Processor -- start 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.497+0000 7f2bde102700 1 -- start start 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bde102700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806f850 0x7f2bd80b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bde102700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd80cd8c0 0x7f2bd80b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bde102700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd8140950 0x7f2bd813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bde102700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80b6af0 con 0x7f2bd8140950 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bde102700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80b6530 con 0x7f2bd80cd8c0 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bde102700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80b6810 con 0x7f2bd806f850 2023-11-28T13:03:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd8140950 0x7f2bd813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd8140950 0x7f2bd813ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42326/0 (socket says 172.21.15.153:42326) 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 -- 172.21.15.153:0/4100155368 learned_addr learned my addr 172.21.15.153:0/4100155368 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd6ffd700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd80cd8c0 0x7f2bd80b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd77fe700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806f850 0x7f2bd80b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 -- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806f850 msgr2=0x7f2bd80b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806f850 0x7f2bd80b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 -- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd80cd8c0 msgr2=0x7f2bd80b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd80cd8c0 0x7f2bd80b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 -- 172.21.15.153:0/4100155368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0009a70 con 0x7f2bd8140950 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd6ffd700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd80cd8c0 0x7f2bd80b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bd7fff700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd8140950 0x7f2bd813ccf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80b3aa0 tx=0x7f2bd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.503 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bbffff700 1 -- 172.21.15.153:0/4100155368 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0003a20 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bbffff700 1 -- 172.21.15.153:0/4100155368 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd0012670 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bbffff700 1 -- 172.21.15.153:0/4100155368 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0012a20 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.498+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd80b3310 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.499+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80b3040 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.500+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd80b35f0 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.500+0000 7f2bbffff700 1 -- 172.21.15.153:0/4100155368 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2bd002c070 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.500+0000 7f2bbffff700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2bc40682f0 0x7f2bc406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.500+0000 7f2bbffff700 1 -- 172.21.15.153:0/4100155368 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2bd0087e30 con 0x7f2bd8140950 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.500+0000 7f2bd77fe700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2bc40682f0 0x7f2bc406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.504 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.501+0000 7f2bd77fe700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2bc40682f0 0x7f2bc406a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80016a0 tx=0x7f2bc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.505 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.504+0000 7f2bbffff700 1 -- 172.21.15.153:0/4100155368 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2bd0054690 con 0x7f2bd8140950 2023-11-28T13:03:04.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.729+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd8068690 con 0x7f2bd8140950 2023-11-28T13:03:04.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.729+0000 7f2bbffff700 1 -- 172.21.15.153:0/4100155368 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f2bd002a090 con 0x7f2bd8140950 2023-11-28T13:03:04.730 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:04.730 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:02:57.970702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:03:04.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2bc40682f0 msgr2=0x7f2bc406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2bc40682f0 0x7f2bc406a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80016a0 tx=0x7f2bc8009040 comp rx=0 tx=0).stop 2023-11-28T13:03:04.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd8140950 msgr2=0x7f2bd813ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd8140950 0x7f2bd813ccf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80b3aa0 tx=0x7f2bd0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 shutdown_connections 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2bc40682f0 0x7f2bc406a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2bd8140950 0x7f2bd813ccf0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bd806f850 0x7f2bd80b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 --2- 172.21.15.153:0/4100155368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd80cd8c0 0x7f2bd80b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.731+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 >> 172.21.15.153:0/4100155368 conn(0x7f2bd806aed0 msgr2=0x7f2bd80cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.732+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 shutdown_connections 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:04.732+0000 7f2bde102700 1 -- 172.21.15.153:0/4100155368 wait complete. 2023-11-28T13:03:04.733 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 31 2023-11-28T13:03:05.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:02:54.953387+0000', 'modified': '2023-11-28T13:02:57.970702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5991}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5991': {'gid': 5991, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.153:6837/3815439165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.153:6835', 'nonce': 3815439165}, {'type': 'v1', 'addr': '172.21.15.153:6837', 'nonce': 3815439165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:03:05.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:03:05.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T13:03:05.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 Processor -- start 2023-11-28T13:03:05.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 -- start start 2023-11-28T13:03:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe4c8161340 0x7fe4c80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe4c80d2320 0x7fe4c80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c80b5800 con 0x7fe4c8161340 2023-11-28T13:03:05.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c80d1950 con 0x7fe4c80d2320 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.276+0000 7fe4ce902700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80b5520 con 0x7fe4c815c9b0 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe4c8161340 0x7fe4c80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe4c8161340 0x7fe4c80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42332/0 (socket says 172.21.15.153:42332) 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2500308627 learned_addr learned my addr 172.21.15.153:0/2500308627 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/2500308627 >> v2:172.21.15.177:3300/0 conn(0x7fe4c80d2320 0x7fe4c80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/2500308627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2500308627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 msgr2=0x7fe4c80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/2500308627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2500308627 >> v2:172.21.15.177:3300/0 conn(0x7fe4c80d2320 msgr2=0x7fe4c80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/2500308627 >> v2:172.21.15.177:3300/0 conn(0x7fe4c80d2320 0x7fe4c80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2500308627 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80d1300 con 0x7fe4c8161340 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/2500308627 >> v2:172.21.15.177:3300/0 conn(0x7fe4c80d2320 0x7fe4c80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:05.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/2500308627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8161340 0x7fe4c80d1de0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4c808c4d0 tx=0x7fe4bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e36d4f40adadf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c6ffd700 1 -- 172.21.15.153:0/2500308627 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00b870 con 0x7fe4c8161340 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c6ffd700 1 -- 172.21.15.153:0/2500308627 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc00b9d0 con 0x7fe4c8161340 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.277+0000 7fe4c6ffd700 1 -- 172.21.15.153:0/2500308627 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00bd60 con 0x7fe4c8161340 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 -- 172.21.15.153:0/2500308627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8161340 msgr2=0x7fe4c80d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 --2- 172.21.15.153:0/2500308627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8161340 0x7fe4c80d1de0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4c808c4d0 tx=0x7fe4bc009bd0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 -- 172.21.15.153:0/2500308627 shutdown_connections 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 --2- 172.21.15.153:0/2500308627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 --2- 172.21.15.153:0/2500308627 >> v2:172.21.15.177:3300/0 conn(0x7fe4c80d2320 0x7fe4c80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 --2- 172.21.15.153:0/2500308627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c8161340 0x7fe4c80d1de0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 -- 172.21.15.153:0/2500308627 >> 172.21.15.153:0/2500308627 conn(0x7fe4c8072600 msgr2=0x7fe4c80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 -- 172.21.15.153:0/2500308627 shutdown_connections 2023-11-28T13:03:05.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.278+0000 7fe4ce902700 1 -- 172.21.15.153:0/2500308627 wait complete. 2023-11-28T13:03:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.279+0000 7fe4ce902700 1 Processor -- start 2023-11-28T13:03:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.279+0000 7fe4ce902700 1 -- start start 2023-11-28T13:03:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4ce902700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c80d2320 0x7fe4c814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4ce902700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c81e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4ce902700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4c8161340 0x7fe4c81501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4ce902700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81e88d0 con 0x7fe4c80d2320 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4ce902700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81e7cc0 con 0x7fe4c8161340 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4ce902700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81e7fa0 con 0x7fe4c815c9b0 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c80d2320 0x7fe4c814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c80d2320 0x7fe4c814e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42336/0 (socket says 172.21.15.153:42336) 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 -- 172.21.15.153:0/3205583301 learned_addr learned my addr 172.21.15.153:0/3205583301 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c81e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 -- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 msgr2=0x7fe4c81e73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c81e73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4c8161340 0x7fe4c81501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 -- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4c8161340 msgr2=0x7fe4c81501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4c8161340 0x7fe4c81501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 -- 172.21.15.153:0/3205583301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc009550 con 0x7fe4c80d2320 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.280+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c80d2320 0x7fe4c814e2b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c7f0 tx=0x7fe4b800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.281+0000 7fe4c4ff9700 1 -- 172.21.15.153:0/3205583301 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8011800 con 0x7fe4c80d2320 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.281+0000 7fe4c4ff9700 1 -- 172.21.15.153:0/3205583301 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b8011960 con 0x7fe4c80d2320 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.281+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c80b5130 con 0x7fe4c80d2320 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.281+0000 7fe4c4ff9700 1 -- 172.21.15.153:0/3205583301 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8011cf0 con 0x7fe4c80d2320 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.281+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c80d15e0 con 0x7fe4c80d2320 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.282+0000 7fe4c4ff9700 1 -- 172.21.15.153:0/3205583301 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4b8021070 con 0x7fe4c80d2320 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.282+0000 7fe4c4ff9700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe4b40682f0 0x7fe4b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.282+0000 7fe4c4ff9700 1 -- 172.21.15.153:0/3205583301 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4b8085ab0 con 0x7fe4c80d2320 2023-11-28T13:03:05.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.282+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe4b40682f0 0x7fe4b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.283+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe4b40682f0 0x7fe4b406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe4c814ee70 tx=0x7fe4bc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.284+0000 7fe4daa8bb80 1 Processor -- start 2023-11-28T13:03:05.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.284+0000 7fe4daa8bb80 1 -- start start 2023-11-28T13:03:05.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d06a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d30be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d4c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712bf0bc0 con 0x55f712d30810 2023-11-28T13:03:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712c5f350 con 0x55f712d31120 2023-11-28T13:03:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f712c58880 con 0x55f712d2e130 2023-11-28T13:03:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d06a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d06a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42344/0 (socket says 172.21.15.153:42344) 2023-11-28T13:03:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.285+0000 7fe4c7fff700 1 -- 172.21.15.153:0/788636071 learned_addr learned my addr 172.21.15.153:0/788636071 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d4c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d30be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4c7fff700 1 -- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 msgr2=0x55f712d30be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d30be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4c7fff700 1 -- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 msgr2=0x55f712d4c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d4c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4c7fff700 1 -- 172.21.15.153:0/788636071 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f712ada5e0 con 0x55f712d30810 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d06a20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8055b90 tx=0x7fe4b805dff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f87d6fba0498807 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4adffb700 1 -- 172.21.15.153:0/788636071 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b805dce0 con 0x55f712d30810 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4adffb700 1 -- 172.21.15.153:0/788636071 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b806d1d0 con 0x55f712d30810 2023-11-28T13:03:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.286+0000 7fe4adffb700 1 -- 172.21.15.153:0/788636071 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8051ef0 con 0x55f712d30810 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/788636071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 msgr2=0x55f712d06a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d06a20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8055b90 tx=0x7fe4b805dff0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/788636071 shutdown_connections 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d06a20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d4c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/788636071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d30be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/788636071 >> 172.21.15.153:0/788636071 conn(0x55f712d03d60 msgr2=0x55f712d046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/788636071 shutdown_connections 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/788636071 wait complete. 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.287+0000 7fe4daa8bb80 1 -- start start 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.288+0000 7fe4daa8bb80 1 Processor -- start 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d2fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d26dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d27300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712adc8d0 con 0x55f712d30810 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712d50700 con 0x55f712d31120 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f712d27840 con 0x55f712d2e130 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d26dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 10 client.0 _ll_register_callbacks cb 0x55f712afe240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d26dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42352/0 (socket says 172.21.15.153:42352) 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 -- 172.21.15.153:0/4271233674 learned_addr learned my addr 172.21.15.153:0/4271233674 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d2fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d27300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 -- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 msgr2=0x55f712d2fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d2fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 -- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 msgr2=0x55f712d27300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d27300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 -- 172.21.15.153:0/4271233674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c80d15e0 con 0x55f712d30810 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d27300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d26dc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009fd0 tx=0x7fe4bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6b4a79d42366d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/4271233674 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003d30 con 0x55f712d30810 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/4271233674 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4bc032470 con 0x55f712d30810 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.289+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f712c26140 con 0x55f712d30810 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/4271233674 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4bc003e90 con 0x55f712d30810 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 10 client.6165 fetch_fsmap learned FSMap version 31 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55f712a86d90 con 0x55f712d30810 2023-11-28T13:03:05.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/4271233674 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc039070 con 0x55f712d30810 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/4271233674 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe4bc032630 con 0x55f712d30810 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 10 client.6165 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 10 client.6165 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 20 client.6165 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d280c0 0x55f712d28490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 4 client.6165 mds_command: new command op to 5991 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f712adcf10 con 0x55f712d280c0 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d280c0 0x55f712d28490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4a6ffd700 10 client.6165 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.290+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d280c0 0x55f712d28490 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb5debf718e95df6 server_cookie=74cec70793a01eba in_seq=0 out_seq=0 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/4271233674 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55f712adcf10 con 0x55f712d280c0 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4a6ffd700 10 client.6165 handle_command_reply: tid=0 2023-11-28T13:03:05.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 client.6165 shutdown 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 msgr2=0x55f712d26dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d26dc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009fd0 tx=0x7fe4bc0047c0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 shutdown_connections 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d280c0 0x55f712d28490 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d31120 0x55f712d27300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d2e130 0x55f712d2fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4a6ffd700 0 client.6165 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/4271233674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d26dc0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 >> 172.21.15.153:0/4271233674 conn(0x55f712d03d60 msgr2=0x55f712d06990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 shutdown_connections 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/4271233674 wait complete. 2023-11-28T13:03:05.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.291+0000 7fe4daa8bb80 20 client.6165 trim_cache size 0 max 16384 2023-11-28T13:03:05.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.308+0000 7fe4daa8bb80 1 Processor -- start 2023-11-28T13:03:05.309 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4daa8bb80 1 -- start start 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d2e130 0x55f712d34370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d52430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712d52970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712d450f0 con 0x55f712d30810 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712d28cd0 con 0x55f712d2e130 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f712d07d50 con 0x55f712d31120 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4c77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d52430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4c77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d52430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42360/0 (socket says 172.21.15.153:42360) 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2365210513 learned_addr learned my addr 172.21.15.153:0/2365210513 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d2e130 0x55f712d34370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.309+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712d52970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 msgr2=0x55f712d52970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712d52970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d2e130 msgr2=0x55f712d34370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d2e130 0x55f712d34370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4c77fe700 1 -- 172.21.15.153:0/2365210513 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f712c1f140 con 0x55f712d30810 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d2e130 0x55f712d34370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712d52970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d52430 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b010 tx=0x7fe4bc02b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4791924c2f711a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4ae7fc700 1 -- 172.21.15.153:0/2365210513 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc032670 con 0x55f712d30810 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4ae7fc700 1 -- 172.21.15.153:0/2365210513 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc039070 con 0x55f712d30810 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4ae7fc700 1 -- 172.21.15.153:0/2365210513 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003b80 con 0x55f712d30810 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/2365210513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 msgr2=0x55f712d52430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d52430 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b010 tx=0x7fe4bc02b070 comp rx=0 tx=0).stop 2023-11-28T13:03:05.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/2365210513 shutdown_connections 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712d52970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d2e130 0x55f712d34370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/2365210513 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d30810 0x55f712d52430 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.310+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/2365210513 >> 172.21.15.153:0/2365210513 conn(0x55f712d03d60 msgr2=0x55f712d28490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.311+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/2365210513 shutdown_connections 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.311+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/2365210513 wait complete. 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.311+0000 7fe4daa8bb80 1 -- start start 2023-11-28T13:03:05.314 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.311+0000 7fe4daa8bb80 1 Processor -- start 2023-11-28T13:03:05.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d2e130 0x55f712d40fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d30810 0x55f712db0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4daa8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712db0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712d42170 con 0x55f712d2e130 2023-11-28T13:03:05.319 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f712d45350 con 0x55f712d30810 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4daa8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f712db11a0 con 0x55f712d31120 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4daa8bb80 10 client.0 _ll_register_callbacks cb 0x55f712afe240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d2e130 0x55f712d40fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d2e130 0x55f712d40fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42364/0 (socket says 172.21.15.153:42364) 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 -- 172.21.15.153:0/1004225328 learned_addr learned my addr 172.21.15.153:0/1004225328 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d30810 0x55f712db0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4cce9f700 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712db0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 -- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 msgr2=0x55f712db0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712db0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 -- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d30810 msgr2=0x55f712db0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d30810 0x55f712db0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 -- 172.21.15.153:0/1004225328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f712a86d90 con 0x55f712d2e130 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4c7fff700 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d2e130 0x55f712d40fb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe4b805b010 tx=0x7fe4b801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c98890ad267493 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.312+0000 7fe4a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/1004225328 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8067060 con 0x55f712d2e130 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/1004225328 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4b8059e40 con 0x55f712d2e130 2023-11-28T13:03:05.321 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f712c16160 con 0x55f712d2e130 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/1004225328 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4b805dd70 con 0x55f712d2e130 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 10 client.6171 fetch_fsmap learned FSMap version 31 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55f712aad200 con 0x55f712d2e130 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/1004225328 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f712c16160 con 0x55f712d2e130 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/1004225328 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe4b8051ed0 con 0x55f712d2e130 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 10 client.6171 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 10 client.6171 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 20 client.6171 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d64550 0x55f712d64920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 4 client.6171 mds_command: new command op to 5991 tid=0[{"prefix": "status"}] 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f712c00170 con 0x55f712d64550 2023-11-28T13:03:05.322 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d64550 0x55f712d64920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4a6ffd700 10 client.6171 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.313+0000 7fe4c77fe700 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d64550 0x55f712d64920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955bc4faa25584a8 server_cookie=916a2219d8238a38 in_seq=0 out_seq=0 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4a6ffd700 1 -- 172.21.15.153:0/1004225328 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f712c00170 con 0x55f712d64550 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4a6ffd700 10 client.6171 handle_command_reply: tid=0 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 client.6171 shutdown 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d2e130 msgr2=0x55f712d40fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d2e130 0x55f712d40fb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe4b805b010 tx=0x7fe4b801f000 comp rx=0 tx=0).stop 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 shutdown_connections 2023-11-28T13:03:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55f712d64550 0x55f712d64920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55f712d2e130 0x55f712d40fb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f712d31120 0x55f712db0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 --2- 172.21.15.153:0/1004225328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f712d30810 0x55f712db0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4a6ffd700 0 client.6171 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 >> 172.21.15.153:0/1004225328 conn(0x55f712d03d60 msgr2=0x55f712d34160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 shutdown_connections 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 1 -- 172.21.15.153:0/1004225328 wait complete. 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4daa8bb80 20 client.6171 trim_cache size 0 max 16384 2023-11-28T13:03:05.324 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "cluster_fsid": "47bfaa00-a151-4537-aa4c-32b0866eb248", 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "whoami": 0, 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 5991, 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "want_state": "up:active", 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "state": "up:active", 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs", 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "rank_uptime": 10.359357911, 2023-11-28T13:03:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: "mdsmap_epoch": 31, 2023-11-28T13:03:05.326 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch": 63, 2023-11-28T13:03:05.326 INFO:teuthology.orchestra.run.smithi153.stdout: "osdmap_epoch_barrier": 63, 2023-11-28T13:03:05.326 INFO:teuthology.orchestra.run.smithi153.stdout: "uptime": 15.707238776000001 2023-11-28T13:03:05.326 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:03:05.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe4b40682f0 msgr2=0x7fe4b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.314+0000 7fe4ce902700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe4b40682f0 0x7fe4b406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe4c814ee70 tx=0x7fe4bc008030 comp rx=0 tx=0).stop 2023-11-28T13:03:05.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c80d2320 msgr2=0x7fe4c814e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c80d2320 0x7fe4c814e2b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c7f0 tx=0x7fe4b800cbb0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 shutdown_connections 2023-11-28T13:03:05.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe4b40682f0 0x7fe4b406a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4c80d2320 0x7fe4c814e2b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4c8161340 0x7fe4c81501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 --2- 172.21.15.153:0/3205583301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4c815c9b0 0x7fe4c81e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 >> 172.21.15.153:0/3205583301 conn(0x7fe4c8072600 msgr2=0x7fe4c815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 shutdown_connections 2023-11-28T13:03:05.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.315+0000 7fe4ce902700 1 -- 172.21.15.153:0/3205583301 wait complete. 2023-11-28T13:03:05.328 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:03:05.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.450+0000 7f4d4abc2700 1 Processor -- start 2023-11-28T13:03:05.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.450+0000 7f4d4abc2700 1 -- start start 2023-11-28T13:03:05.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4abc2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d4406ba90 0x7f4d440af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4abc2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4d440cbdf0 0x7f4d440ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4abc2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d440cc5f0 con 0x7f4d4415cef0 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4abc2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4415d470 con 0x7f4d4406ba90 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4abc2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4415d750 con 0x7f4d440cbdf0 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d43fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d4406ba90 0x7f4d440af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4895e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4d4415cef0 0x7f4d4406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60536/0 (socket says 172.21.15.153:60536) 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4895e700 1 -- 172.21.15.153:0/4292021771 learned_addr learned my addr 172.21.15.153:0/4292021771 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d437fe700 1 -- 172.21.15.153:0/4292021771 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306246512 0 0) 0x7f4d440cc5f0 con 0x7f4d4415cef0 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d437fe700 1 -- 172.21.15.153:0/4292021771 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d340035b0 con 0x7f4d4415cef0 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d4915f700 1 --2- 172.21.15.153:0/4292021771 >> v2:172.21.15.177:3300/0 conn(0x7f4d440cbdf0 0x7f4d440ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d43fff700 1 -- 172.21.15.153:0/4292021771 >> v2:172.21.15.177:3300/0 conn(0x7f4d440cbdf0 msgr2=0x7f4d440ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d43fff700 1 --2- 172.21.15.153:0/4292021771 >> v2:172.21.15.177:3300/0 conn(0x7f4d440cbdf0 0x7f4d440ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d43fff700 1 -- 172.21.15.153:0/4292021771 >> v1:172.21.15.153:6789/0 conn(0x7f4d4415cef0 legacy=0x7f4d4406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d43fff700 1 -- 172.21.15.153:0/4292021771 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4415da30 con 0x7f4d4406ba90 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d43fff700 1 --2- 172.21.15.153:0/4292021771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4406ba90 0x7f4d440af400 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4d44073b10 tx=0x7f4d30009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2148e1dceaa5c4ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.451+0000 7f4d437fe700 1 -- 172.21.15.153:0/4292021771 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30011830 con 0x7f4d4406ba90 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d437fe700 1 -- 172.21.15.153:0/4292021771 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d30011990 con 0x7f4d4406ba90 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d437fe700 1 -- 172.21.15.153:0/4292021771 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30011d20 con 0x7f4d4406ba90 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 -- 172.21.15.153:0/4292021771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4406ba90 msgr2=0x7f4d440af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/4292021771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4406ba90 0x7f4d440af400 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4d44073b10 tx=0x7f4d30009b20 comp rx=0 tx=0).stop 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 -- 172.21.15.153:0/4292021771 shutdown_connections 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/4292021771 >> v2:172.21.15.177:3300/0 conn(0x7f4d440cbdf0 0x7f4d440ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/4292021771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4406ba90 0x7f4d440af400 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 -- 172.21.15.153:0/4292021771 >> 172.21.15.153:0/4292021771 conn(0x7f4d440d0000 msgr2=0x7f4d440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 -- 172.21.15.153:0/4292021771 shutdown_connections 2023-11-28T13:03:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.452+0000 7f4d4abc2700 1 -- 172.21.15.153:0/4292021771 wait complete. 2023-11-28T13:03:05.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.453+0000 7f4d4abc2700 1 Processor -- start 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.453+0000 7f4d4abc2700 1 -- start start 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4abc2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d4406ba90 0x7f4d441e3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4abc2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d440cbdf0 0x7f4d440b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4abc2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d4415cef0 0x7f4d44149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4abc2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44147070 con 0x7f4d440cbdf0 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4abc2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440b5c80 con 0x7f4d4406ba90 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4abc2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44149bb0 con 0x7f4d4415cef0 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d440cbdf0 0x7f4d440b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d43fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d440cbdf0 0x7f4d440b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42380/0 (socket says 172.21.15.153:42380) 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d43fff700 1 -- 172.21.15.153:0/3987979000 learned_addr learned my addr 172.21.15.153:0/3987979000 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:05.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4895e700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d4406ba90 0x7f4d441e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.454+0000 7f4d4915f700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d4415cef0 0x7f4d44149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d43fff700 1 -- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d4415cef0 msgr2=0x7f4d44149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d43fff700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d4415cef0 0x7f4d44149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d43fff700 1 -- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d4406ba90 msgr2=0x7f4d441e3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d43fff700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d4406ba90 0x7f4d441e3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d43fff700 1 -- 172.21.15.153:0/3987979000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300094a0 con 0x7f4d440cbdf0 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d43fff700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d440cbdf0 0x7f4d440b5740 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009b60 tx=0x7f4d30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d417fa700 1 -- 172.21.15.153:0/3987979000 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d300036a0 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d417fa700 1 -- 172.21.15.153:0/3987979000 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3001f070 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4415dff0 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d417fa700 1 -- 172.21.15.153:0/3987979000 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30003a40 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.455+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4415dd10 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.456+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d440cab90 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.456+0000 7f4d417fa700 1 -- 172.21.15.153:0/3987979000 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d3000f050 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.457+0000 7f4d417fa700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d280682f0 0x7f4d2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.457+0000 7f4d417fa700 1 -- 172.21.15.153:0/3987979000 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d30021070 con 0x7f4d440cbdf0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.457+0000 7f4d4895e700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d280682f0 0x7f4d2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.457+0000 7f4d4895e700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d280682f0 0x7f4d2806a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d380011e0 tx=0x7f4d38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.461+0000 7f4d417fa700 1 -- 172.21.15.153:0/3987979000 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d30053e50 con 0x7f4d440cbdf0 2023-11-28T13:03:05.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.687+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d440ceb00 con 0x7f4d440cbdf0 2023-11-28T13:03:05.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.687+0000 7f4d417fa700 1 -- 172.21.15.153:0/3987979000 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d30015070 con 0x7f4d440cbdf0 2023-11-28T13:03:05.688 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:03:05.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d280682f0 msgr2=0x7f4d2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d280682f0 0x7f4d2806a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d380011e0 tx=0x7f4d38009040 comp rx=0 tx=0).stop 2023-11-28T13:03:05.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d440cbdf0 msgr2=0x7f4d440b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d440cbdf0 0x7f4d440b5740 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009b60 tx=0x7f4d30004750 comp rx=0 tx=0).stop 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 shutdown_connections 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d4415cef0 0x7f4d44149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d4406ba90 0x7f4d441e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d280682f0 0x7f4d2806a7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 --2- 172.21.15.153:0/3987979000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d440cbdf0 0x7f4d440b5740 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 >> 172.21.15.153:0/3987979000 conn(0x7f4d440d0000 msgr2=0x7f4d4415c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 shutdown_connections 2023-11-28T13:03:05.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:05.689+0000 7f4d4abc2700 1 -- 172.21.15.153:0/3987979000 wait complete. 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:03:05.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:03:05.703 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:03:05.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:05.703 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:05.720 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42195sec preferred_lft 42195sec 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ad7d:b13e:f07:e265/64 scope link noprefixroute 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:05.721 INFO:teuthology.orchestra.run.smithi153.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 4e:45:7e:fd:cf:4e brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::6cbb:9eff:fe64:da81/64 scope link 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether a2:10:9d:7d:8b:89 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::a010:9dff:fe7d:8b89/64 scope link 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 4e:45:7e:fd:cf:4e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:03:05.722 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::4c45:7eff:fefd:cf4e/64 scope link 2023-11-28T13:03:05.723 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:05.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:05.723 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:03:05.786 INFO:teuthology.orchestra.run.smithi153.stdout:1 2023-11-28T13:03:05.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:05.788 DEBUG:teuthology.orchestra.run.smithi153:> route 2023-11-28T13:03:05.846 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2023-11-28T13:03:05.846 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:03:05.847 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:03:05.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:03:05.847 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:03:05.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:05.847 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2023-11-28T13:03:05.847 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2023-11-28T13:03:05.847 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:03:05.847 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:03:05.847 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:03:05.848 DEBUG:teuthology.orchestra.run.smithi153:> ') 2023-11-28T13:03:05.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:05.951 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:03:05.968 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:03:05.968 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:03:05.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:03:05.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:05.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2023-11-28T13:03:06.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:03:06.035 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:06.092 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:03:06.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:03:06.093 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:06.149 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T13:03:06.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:06.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:03:06.235 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:03:06.235 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:03:06.235 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:03:06.244 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:03:06.245 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:03:06.245 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:03:06.245 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:03:06.246 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:03:06.318 INFO:teuthology.orchestra.run.smithi153.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T13:03:06.318 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T13:03:06.319 INFO:teuthology.orchestra.run.smithi153.stdout:invalid new device string format 2023-11-28T13:03:06.319 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T13:03:06.319 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T13:03:06.319 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: resolved to: "172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T13:03:06.319 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.153:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T13:03:06.319 INFO:teuthology.orchestra.run.smithi153.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=47bfaa00-a151-4537-aa4c-32b0866eb248" will pass to kernel 2023-11-28T13:03:06.321 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T13:03:06.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:06.321 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:06.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:06.348 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T13:03:06.433 INFO:teuthology.orchestra.run.smithi153.stdout:client5393 2023-11-28T13:03:06.433 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:03:06.434 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:03:06.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:06.451 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T13:03:06.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:06.513 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T13:03:06.573 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:03:06.574 DEBUG:teuthology.orchestra.run.smithi153:> import os 2023-11-28T13:03:06.574 DEBUG:teuthology.orchestra.run.smithi153:> import stat 2023-11-28T13:03:06.574 DEBUG:teuthology.orchestra.run.smithi153:> 2023-11-28T13:03:06.574 DEBUG:teuthology.orchestra.run.smithi153:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T13:03:06.574 DEBUG:teuthology.orchestra.run.smithi153:> ' 2023-11-28T13:03:06.679 INFO:teuthology.orchestra.run.smithi153.stdout:1099511627777 2023-11-28T13:03:06.868 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:07.029 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.029+0000 7f47cd369700 1 Processor -- start 2023-11-28T13:03:07.030 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.029+0000 7f47cd369700 1 -- start start 2023-11-28T13:03:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.029+0000 7f47cd369700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f47c8154e70 0x7f47c80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.029+0000 7f47cd369700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c8071750 0x7f47c815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.029+0000 7f47cd369700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c8155240 con 0x7f47c80cf360 2023-11-28T13:03:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.029+0000 7f47cd369700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c815b100 con 0x7f47c8154e70 2023-11-28T13:03:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.029+0000 7f47cd369700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c815b3e0 con 0x7f47c8071750 2023-11-28T13:03:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f47c8154e70 0x7f47c80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f47c8154e70 0x7f47c80cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42390/0 (socket says 172.21.15.153:42390) 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 -- 172.21.15.153:0/719946883 learned_addr learned my addr 172.21.15.153:0/719946883 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c77fe700 1 --2- 172.21.15.153:0/719946883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c8071750 0x7f47c815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 -- 172.21.15.153:0/719946883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c8071750 msgr2=0x7f47c815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 --2- 172.21.15.153:0/719946883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c8071750 0x7f47c815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 -- 172.21.15.153:0/719946883 >> v1:172.21.15.177:6789/0 conn(0x7f47c80cf360 legacy=0x7f47c8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 -- 172.21.15.153:0/719946883 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c815b6c0 con 0x7f47c8154e70 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c77fe700 1 --2- 172.21.15.153:0/719946883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c8071750 0x7f47c815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c6ffd700 1 --2- 172.21.15.153:0/719946883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8154e70 0x7f47c80cdbd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009e50 tx=0x7f47b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88dcb52c80bbe06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c5ffb700 1 -- 172.21.15.153:0/719946883 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b800e070 con 0x7f47c8154e70 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c5ffb700 1 -- 172.21.15.153:0/719946883 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b8003a20 con 0x7f47c8154e70 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.030+0000 7f47c5ffb700 1 -- 172.21.15.153:0/719946883 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8003dd0 con 0x7f47c8154e70 2023-11-28T13:03:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 -- 172.21.15.153:0/719946883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8154e70 msgr2=0x7f47c80cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 --2- 172.21.15.153:0/719946883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8154e70 0x7f47c80cdbd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009e50 tx=0x7f47b800b330 comp rx=0 tx=0).stop 2023-11-28T13:03:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 -- 172.21.15.153:0/719946883 shutdown_connections 2023-11-28T13:03:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 --2- 172.21.15.153:0/719946883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c8071750 0x7f47c815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 --2- 172.21.15.153:0/719946883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8154e70 0x7f47c80cdbd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 -- 172.21.15.153:0/719946883 >> 172.21.15.153:0/719946883 conn(0x7f47c80d0000 msgr2=0x7f47c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 -- 172.21.15.153:0/719946883 shutdown_connections 2023-11-28T13:03:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.031+0000 7f47cd369700 1 -- 172.21.15.153:0/719946883 wait complete. 2023-11-28T13:03:07.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.032+0000 7f47cd369700 1 Processor -- start 2023-11-28T13:03:07.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.032+0000 7f47cd369700 1 -- start start 2023-11-28T13:03:07.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47cd369700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8071750 0x7f47c80c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47cd369700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c80cf360 0x7f47c80b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47cd369700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47c8154e70 0x7f47c815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47cd369700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c806edf0 con 0x7f47c8071750 2023-11-28T13:03:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47cd369700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80b4150 con 0x7f47c8154e70 2023-11-28T13:03:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47cd369700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80b43d0 con 0x7f47c80cf360 2023-11-28T13:03:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8071750 0x7f47c80c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8071750 0x7f47c80c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42396/0 (socket says 172.21.15.153:42396) 2023-11-28T13:03:07.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 -- 172.21.15.153:0/1438074682 learned_addr learned my addr 172.21.15.153:0/1438074682 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c67fc700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c80cf360 0x7f47c80b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 -- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c80cf360 msgr2=0x7f47c80b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c80cf360 0x7f47c80b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.040 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 -- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47c8154e70 msgr2=0x7f47c815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c77fe700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47c8154e70 0x7f47c815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47c8154e70 0x7f47c815d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.033+0000 7f47c6ffd700 1 -- 172.21.15.153:0/1438074682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8009b30 con 0x7f47c8071750 2023-11-28T13:03:07.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.034+0000 7f47c6ffd700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8071750 0x7f47c80c2d70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f47b8015040 tx=0x7f47b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.041 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.034+0000 7f47bf7fe700 1 -- 172.21.15.153:0/1438074682 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004440 con 0x7f47c8071750 2023-11-28T13:03:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.034+0000 7f47bf7fe700 1 -- 172.21.15.153:0/1438074682 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b80045c0 con 0x7f47c8071750 2023-11-28T13:03:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.034+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c80af520 con 0x7f47c8071750 2023-11-28T13:03:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.034+0000 7f47bf7fe700 1 -- 172.21.15.153:0/1438074682 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004970 con 0x7f47c8071750 2023-11-28T13:03:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.034+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c815b9a0 con 0x7f47c8071750 2023-11-28T13:03:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.035+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c80cd460 con 0x7f47c8071750 2023-11-28T13:03:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.035+0000 7f47bf7fe700 1 -- 172.21.15.153:0/1438074682 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47b8022030 con 0x7f47c8071750 2023-11-28T13:03:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.035+0000 7f47bf7fe700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a80682f0 0x7f47a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.036+0000 7f47bf7fe700 1 -- 172.21.15.153:0/1438074682 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47b8087170 con 0x7f47c8071750 2023-11-28T13:03:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.036+0000 7f47c67fc700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a80682f0 0x7f47a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.036+0000 7f47c67fc700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a80682f0 0x7f47a806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f47c80c2a60 tx=0x7f47b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.040+0000 7f47bf7fe700 1 -- 172.21.15.153:0/1438074682 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47b8053950 con 0x7f47c8071750 2023-11-28T13:03:07.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.262+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47c815ad50 con 0x7f47c8071750 2023-11-28T13:03:07.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.262+0000 7f47bf7fe700 1 -- 172.21.15.153:0/1438074682 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f47b8056f70 con 0x7f47c8071750 2023-11-28T13:03:07.263 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:07.264 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:02:57.970702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:03:07.266 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a80682f0 msgr2=0x7f47a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a80682f0 0x7f47a806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f47c80c2a60 tx=0x7f47b0009040 comp rx=0 tx=0).stop 2023-11-28T13:03:07.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8071750 msgr2=0x7f47c80c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8071750 0x7f47c80c2d70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f47b8015040 tx=0x7f47b8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 shutdown_connections 2023-11-28T13:03:07.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f47a80682f0 0x7f47a806a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47c8071750 0x7f47c80c2d70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47c8154e70 0x7f47c815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 --2- 172.21.15.153:0/1438074682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47c80cf360 0x7f47c80b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.264+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 >> 172.21.15.153:0/1438074682 conn(0x7f47c80d0000 msgr2=0x7f47c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.265+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 shutdown_connections 2023-11-28T13:03:07.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.265+0000 7f47cd369700 1 -- 172.21.15.153:0/1438074682 wait complete. 2023-11-28T13:03:07.268 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 31 2023-11-28T13:03:07.277 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T13:03:07.430 INFO:teuthology.orchestra.run.smithi153.stdout:{"message":"","return_code":0} 2023-11-28T13:03:07.431 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:03:07.431 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:03:07.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.475+0000 7f25a07f6080 1 Processor -- start 2023-11-28T13:03:07.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.475+0000 7f25a07f6080 1 -- start start 2023-11-28T13:03:07.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25a07f6080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6cc80 0x5654970d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25a07f6080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x5654970d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25a07f6080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6d050 0x5654970d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25a07f6080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496ea7fe0 con 0x565496f6d050 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25a07f6080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549703b3b0 con 0x565496fb3390 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25a07f6080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x565496f9c050 con 0x565496f6cc80 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6d050 0x5654970d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6d050 0x5654970d7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42420/0 (socket says 172.21.15.153:42420) 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 -- 172.21.15.153:0/3093175596 learned_addr learned my addr 172.21.15.153:0/3093175596 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25960f8700 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x5654970d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25968f9700 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6cc80 0x5654970d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 -- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6cc80 msgr2=0x5654970d7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6cc80 0x5654970d7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 -- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 msgr2=0x5654970d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x5654970d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25970fa700 1 -- 172.21.15.153:0/3093175596 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565496eb3830 con 0x565496f6d050 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25960f8700 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x5654970d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.476+0000 7f25968f9700 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6cc80 0x5654970d7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25970fa700 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6d050 0x5654970d7da0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2590007010 tx=0x7f2590010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe0412b982dd413 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25950f6700 1 -- 172.21.15.153:0/3093175596 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590012450 con 0x565496f6d050 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25950f6700 1 -- 172.21.15.153:0/3093175596 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590012940 con 0x565496f6d050 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25950f6700 1 -- 172.21.15.153:0/3093175596 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590012cf0 con 0x565496f6d050 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 -- 172.21.15.153:0/3093175596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6d050 msgr2=0x5654970d7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6d050 0x5654970d7da0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2590007010 tx=0x7f2590010000 comp rx=0 tx=0).stop 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 -- 172.21.15.153:0/3093175596 shutdown_connections 2023-11-28T13:03:07.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6d050 0x5654970d7da0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6cc80 0x5654970d7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 --2- 172.21.15.153:0/3093175596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x5654970d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 -- 172.21.15.153:0/3093175596 >> 172.21.15.153:0/3093175596 conn(0x565496f99b60 msgr2=0x565496fae250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 -- 172.21.15.153:0/3093175596 shutdown_connections 2023-11-28T13:03:07.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.477+0000 7f25a07f6080 1 -- 172.21.15.153:0/3093175596 wait complete. 2023-11-28T13:03:07.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.478+0000 7f25a07f6080 1 Processor -- start 2023-11-28T13:03:07.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.478+0000 7f25a07f6080 1 -- start start 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.478+0000 7f25a07f6080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6cc80 0x565496faf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.478+0000 7f25a07f6080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6d050 0x56549703a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25a07f6080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x56549703a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25a07f6080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496f9b690 con 0x565496f6cc80 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25a07f6080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549703b1b0 con 0x565496fb3390 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25a07f6080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654970cf5d0 con 0x565496f6d050 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6cc80 0x565496faf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6cc80 0x565496faf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42424/0 (socket says 172.21.15.153:42424) 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 -- 172.21.15.153:0/1667631346 learned_addr learned my addr 172.21.15.153:0/1667631346 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25960f8700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6d050 0x56549703a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25970fa700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x56549703a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 -- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6d050 msgr2=0x56549703a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565496f6d050 0x56549703a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 -- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 msgr2=0x56549703a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565496fb3390 0x56549703a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 -- 172.21.15.153:0/1667631346 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580001e10 con 0x565496f6cc80 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25968f9700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6cc80 0x565496faf320 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c00f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f257ffff700 1 -- 172.21.15.153:0/1667631346 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00d810 con 0x565496f6cc80 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f257ffff700 1 -- 172.21.15.153:0/1667631346 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c00d970 con 0x565496f6cc80 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x565496ec0360 con 0x565496f6cc80 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f257ffff700 1 -- 172.21.15.153:0/1667631346 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00dce0 con 0x565496f6cc80 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.479+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565496ec0120 con 0x565496f6cc80 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f257ffff700 1 -- 172.21.15.153:0/1667631346 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f258c020070 con 0x565496f6cc80 2023-11-28T13:03:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f257ffff700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2584066c10 0x7f25840690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f257ffff700 1 -- 172.21.15.153:0/1667631346 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f258c004040 con 0x565496f6cc80 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f25960f8700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2584066c10 0x7f25840690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f25a07f6080 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x565496faf860 0x565496fb1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 --> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63) v8 -- 0x5654970496d0 con 0x565496faf860 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f25960f8700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2584066c10 0x7f25840690c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x565496f996a0 tx=0x7f258800c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.481+0000 7f25970fa700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x565496faf860 0x565496fb1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25970fa700 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x565496faf860 0x565496fb1f10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25970fa700 1 -- 172.21.15.153:0/1667631346 <== osd.6 v2:172.21.15.177:6824/103787 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f259002d040 con 0x565496faf860 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x565496faf860 msgr2=0x565496fb1f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x565496faf860 0x565496fb1f10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2584066c10 msgr2=0x7f25840690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2584066c10 0x7f25840690c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x565496f996a0 tx=0x7f258800c040 comp rx=0 tx=0).stop 2023-11-28T13:03:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6cc80 msgr2=0x565496faf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 --2- 172.21.15.153:0/1667631346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x565496f6cc80 0x565496faf320 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c00f6d0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25970fa700 1 -- 172.21.15.153:0/1667631346 reap_dead start 2023-11-28T13:03:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 shutdown_connections 2023-11-28T13:03:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 >> 172.21.15.153:0/1667631346 conn(0x565496f99b60 msgr2=0x5654970ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 shutdown_connections 2023-11-28T13:03:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.482+0000 7f25a07f6080 1 -- 172.21.15.153:0/1667631346 wait complete. 2023-11-28T13:03:07.489 INFO:teuthology.orchestra.run.smithi153.stdout:Xfiledir_test 2023-11-28T13:03:07.489 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:07.489 DEBUG:teuthology.orchestra.run.smithi153:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:03:07.544 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:03:07.544 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 1099511627777, 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "ancestors": [ 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "dirino": 1099511627776, 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "dname": "file", 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "version": 2 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "dirino": 1, 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "dname": "dir_test", 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "version": 4 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2023-11-28T13:03:07.545 INFO:teuthology.orchestra.run.smithi153.stdout: "pool": 10, 2023-11-28T13:03:07.546 INFO:teuthology.orchestra.run.smithi153.stdout: "old_pools": [] 2023-11-28T13:03:07.546 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:03:07.546 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:07.547 DEBUG:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T13:03:07.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.631+0000 7f36575e6080 1 Processor -- start 2023-11-28T13:03:07.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.631+0000 7f36575e6080 1 -- start start 2023-11-28T13:03:07.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.631+0000 7f36575e6080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be75f3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f36575e6080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7685460 0x555be7684960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f36575e6080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7689df0 0x555be76880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f36575e6080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be7555000 con 0x555be75af7f0 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f36575e6080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be74e7fe0 con 0x555be7685460 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f36575e6080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x555be75dd6a0 con 0x555be7689df0 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be75f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be75f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42440/0 (socket says 172.21.15.153:42440) 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 -- 172.21.15.153:0/2582206949 learned_addr learned my addr 172.21.15.153:0/2582206949 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364cee8700 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7685460 0x555be7684960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364deea700 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7689df0 0x555be76880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 -- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7689df0 msgr2=0x555be76880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7689df0 0x555be76880b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 -- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7685460 msgr2=0x555be7684960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7685460 0x555be7684960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 -- 172.21.15.153:0/2582206949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364800a100 con 0x555be75af7f0 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364deea700 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7689df0 0x555be76880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f364d6e9700 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be75f3970 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3644002de0 tx=0x7f364400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bc59c26a613424 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f363f7fe700 1 -- 172.21.15.153:0/2582206949 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400be10 con 0x555be75af7f0 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f363f7fe700 1 -- 172.21.15.153:0/2582206949 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644008030 con 0x555be75af7f0 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.632+0000 7f363f7fe700 1 -- 172.21.15.153:0/2582206949 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364400f640 con 0x555be75af7f0 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 -- 172.21.15.153:0/2582206949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 msgr2=0x555be75f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be75f3970 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3644002de0 tx=0x7f364400b4c0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 -- 172.21.15.153:0/2582206949 shutdown_connections 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be75f3970 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7689df0 0x555be76880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 --2- 172.21.15.153:0/2582206949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7685460 0x555be7684960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 -- 172.21.15.153:0/2582206949 >> 172.21.15.153:0/2582206949 conn(0x555be75d9b70 msgr2=0x555be75ee0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 -- 172.21.15.153:0/2582206949 shutdown_connections 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 -- 172.21.15.153:0/2582206949 wait complete. 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 Processor -- start 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.633+0000 7f36575e6080 1 -- start start 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.634+0000 7f36575e6080 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be7677030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.634+0000 7f36575e6080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7685460 0x555be7682e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.634+0000 7f36575e6080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7689df0 0x555be75db450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.634+0000 7f36575e6080 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be75f3eb0 con 0x555be75af7f0 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.634+0000 7f36575e6080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555be767dc50 con 0x555be7689df0 2023-11-28T13:03:07.639 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.634+0000 7f36575e6080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x555be75ee9d0 con 0x555be7685460 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.634+0000 7f364d6e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be7677030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be7677030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42446/0 (socket says 172.21.15.153:42446) 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 -- 172.21.15.153:0/203848227 learned_addr learned my addr 172.21.15.153:0/203848227 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364cee8700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7685460 0x555be7682e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364deea700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7689df0 0x555be75db450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 -- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7685460 msgr2=0x555be7682e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555be7685460 0x555be7682e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 -- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7689df0 msgr2=0x555be75db450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555be7689df0 0x555be75db450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 -- 172.21.15.153:0/203848227 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364400a040 con 0x555be75af7f0 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f364d6e9700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be7677030 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f364400b500 tx=0x7f3644001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f363e7fc700 1 -- 172.21.15.153:0/203848227 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644002a20 con 0x555be75af7f0 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f363e7fc700 1 -- 172.21.15.153:0/203848227 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644002150 con 0x555be75af7f0 2023-11-28T13:03:07.640 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555be74f3830 con 0x555be75af7f0 2023-11-28T13:03:07.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f363e7fc700 1 -- 172.21.15.153:0/203848227 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644019480 con 0x555be75af7f0 2023-11-28T13:03:07.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.635+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555be7500360 con 0x555be75af7f0 2023-11-28T13:03:07.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.636+0000 7f363e7fc700 1 -- 172.21.15.153:0/203848227 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f364402a030 con 0x555be75af7f0 2023-11-28T13:03:07.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.637+0000 7f363e7fc700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3630066c10 0x7f36300690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.637+0000 7f363e7fc700 1 -- 172.21.15.153:0/203848227 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3644085f30 con 0x555be75af7f0 2023-11-28T13:03:07.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.637+0000 7f364cee8700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3630066c10 0x7f36300690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.648 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.637+0000 7f36575e6080 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x555be7684230 0x555be7727f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.637+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 --> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63) v8 -- 0x555be767f0d0 con 0x555be7684230 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.637+0000 7f364deea700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x555be7684230 0x555be7727f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.637+0000 7f364cee8700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3630066c10 0x7f36300690c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f36400017e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.638+0000 7f364deea700 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x555be7684230 0x555be7727f00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f364deea700 1 -- 172.21.15.153:0/203848227 <== osd.6 v2:172.21.15.177:6824/103787 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v63'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f364800fa60 con 0x555be7684230 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x555be7684230 msgr2=0x555be7727f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x555be7684230 0x555be7727f00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3630066c10 msgr2=0x7f36300690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3630066c10 0x7f36300690c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f36400017e0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 msgr2=0x555be7677030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.649 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 --2- 172.21.15.153:0/203848227 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x555be75af7f0 0x555be7677030 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f364400b500 tx=0x7f3644001fe0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f364deea700 1 -- 172.21.15.153:0/203848227 reap_dead start 2023-11-28T13:03:07.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 shutdown_connections 2023-11-28T13:03:07.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 >> 172.21.15.153:0/203848227 conn(0x555be75d9b70 msgr2=0x555be7686bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 shutdown_connections 2023-11-28T13:03:07.650 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.639+0000 7f36575e6080 1 -- 172.21.15.153:0/203848227 wait complete. 2023-11-28T13:03:07.650 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T13:03:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.773+0000 7f1ebc11e700 1 Processor -- start 2023-11-28T13:03:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.774+0000 7f1ebc11e700 1 -- start start 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.774+0000 7f1ebc11e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1eb40af430 0x7f1eb4070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.774+0000 7f1ebc11e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb40713c0 con 0x7f1eb40af060 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.774+0000 7f1ebc11e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb40716a0 con 0x7f1eb406a980 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.774+0000 7f1ebc11e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb4071980 con 0x7f1eb40af430 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb9eba700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1eb40af060 0x7f1eb406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60620/0 (socket says 172.21.15.153:60620) 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb9eba700 1 -- 172.21.15.153:0/2181475638 learned_addr learned my addr 172.21.15.153:0/2181475638 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/2181475638 >> v2:172.21.15.177:3300/0 conn(0x7f1eb40af430 0x7f1eb4070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302313936 0 0) 0x7f1eb40713c0 con 0x7f1eb40af060 2023-11-28T13:03:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea00035b0 con 0x7f1eb40af060 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980973191 0 0) 0x7f1eb40716a0 con 0x7f1eb406a980 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb40713c0 con 0x7f1eb406a980 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eba6bb700 1 -- 172.21.15.153:0/2181475638 >> v1:172.21.15.177:6789/0 conn(0x7f1eb406a980 legacy=0x7f1eb40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3365584374 0 0) 0x7f1ea00035b0 con 0x7f1eb40af060 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eba6bb700 1 -- 172.21.15.153:0/2181475638 >> v1:172.21.15.153:6789/0 conn(0x7f1eb40af060 legacy=0x7f1eb406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eba6bb700 1 -- 172.21.15.153:0/2181475638 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4071c60 con 0x7f1eb40af430 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/2181475638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb40af430 0x7f1eb4070d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009d90 tx=0x7f1eac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=631b8715bc16ba9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00e070 con 0x7f1eb40af430 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac003a20 con 0x7f1eb40af430 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.775+0000 7f1eb8eb8700 1 -- 172.21.15.153:0/2181475638 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4001670 con 0x7f1eb40af430 2023-11-28T13:03:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.776+0000 7f1ebc11e700 1 -- 172.21.15.153:0/2181475638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb40af430 msgr2=0x7f1eb4070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.776+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/2181475638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb40af430 0x7f1eb4070d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009d90 tx=0x7f1eac00b330 comp rx=0 tx=0).stop 2023-11-28T13:03:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.776+0000 7f1ebc11e700 1 -- 172.21.15.153:0/2181475638 shutdown_connections 2023-11-28T13:03:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.776+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/2181475638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb40af430 0x7f1eb4070d50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.776+0000 7f1ebc11e700 1 -- 172.21.15.153:0/2181475638 >> 172.21.15.153:0/2181475638 conn(0x7f1eb406b380 msgr2=0x7f1eb406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.776+0000 7f1ebc11e700 1 -- 172.21.15.153:0/2181475638 shutdown_connections 2023-11-28T13:03:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.776+0000 7f1ebc11e700 1 -- 172.21.15.153:0/2181475638 wait complete. 2023-11-28T13:03:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.777+0000 7f1ebc11e700 1 Processor -- start 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.777+0000 7f1ebc11e700 1 -- start start 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1ebc11e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb406a980 0x7f1eb41557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1ebc11e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1eb40af060 0x7f1eb415e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1ebc11e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eb40af430 0x7f1eb40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1ebc11e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb415d460 con 0x7f1eb40af060 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1ebc11e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb415d740 con 0x7f1eb406a980 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1ebc11e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb415da20 con 0x7f1eb40af430 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1eb40af060 0x7f1eb415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1eb40af060 0x7f1eb415e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42464/0 (socket says 172.21.15.153:42464) 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1eb96b9700 1 -- 172.21.15.153:0/96185156 learned_addr learned my addr 172.21.15.153:0/96185156 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eb40af430 0x7f1eb40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.778+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb406a980 0x7f1eb41557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eb96b9700 1 -- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eb40af430 msgr2=0x7f1eb40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eb40af430 0x7f1eb40c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eb96b9700 1 -- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb406a980 msgr2=0x7f1eb41557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb406a980 0x7f1eb41557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eb96b9700 1 -- 172.21.15.153:0/96185156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac009a70 con 0x7f1eb40af060 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1eb40af060 0x7f1eb415e500 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c430 tx=0x7f1eb000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eaa7fc700 1 -- 172.21.15.153:0/96185156 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00052d0 con 0x7f1eb40af060 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eaa7fc700 1 -- 172.21.15.153:0/96185156 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0005450 con 0x7f1eb40af060 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb406d9a0 con 0x7f1eb40af060 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1eaa7fc700 1 -- 172.21.15.153:0/96185156 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00038c0 con 0x7f1eb40af060 2023-11-28T13:03:07.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.779+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb4071f40 con 0x7f1eb40af060 2023-11-28T13:03:07.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.780+0000 7f1eaa7fc700 1 -- 172.21.15.153:0/96185156 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1eb0019030 con 0x7f1eb40af060 2023-11-28T13:03:07.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.781+0000 7f1eaa7fc700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e980682f0 0x7f1e9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.781+0000 7f1eaa7fc700 1 -- 172.21.15.153:0/96185156 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1eb0084f90 con 0x7f1eb40af060 2023-11-28T13:03:07.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.781+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e980682f0 0x7f1e9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.781+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e980682f0 0x7f1e9806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40011e0 tx=0x7f1ea4001be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.783+0000 7f1ec82a7b80 1 Processor -- start 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.783+0000 7f1ec82a7b80 1 -- start start 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e931ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e9591e0 0x558c5e95bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e977820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e81bc70 con 0x558c5e9591e0 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e88a400 con 0x558c5e95c1d0 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c5e883930 con 0x558c5e95b8c0 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e9591e0 0x558c5e95bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e9591e0 0x558c5e95bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42472/0 (socket says 172.21.15.153:42472) 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1613083634 learned_addr learned my addr 172.21.15.153:0/1613083634 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.787 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e931ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e977820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 msgr2=0x558c5e931ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e931ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 msgr2=0x558c5e977820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e977820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1613083634 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c5e705690 con 0x558c5e9591e0 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e9591e0 0x558c5e95bc90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0054e30 tx=0x7f1eb0055350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c65eddefe461ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1e977fe700 1 -- 172.21.15.153:0/1613083634 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00537f0 con 0x558c5e9591e0 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1e977fe700 1 -- 172.21.15.153:0/1613083634 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb005d0b0 con 0x558c5e9591e0 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.784+0000 7f1e977fe700 1 -- 172.21.15.153:0/1613083634 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0051240 con 0x558c5e9591e0 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1613083634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e9591e0 msgr2=0x558c5e95bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e9591e0 0x558c5e95bc90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0054e30 tx=0x7f1eb0055350 comp rx=0 tx=0).stop 2023-11-28T13:03:07.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1613083634 shutdown_connections 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e977820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e931ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1613083634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e9591e0 0x558c5e95bc90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1613083634 >> 172.21.15.153:0/1613083634 conn(0x558c5e92ee10 msgr2=0x558c5e92f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1613083634 shutdown_connections 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.785+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1613083634 wait complete. 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.786+0000 7f1ec82a7b80 1 -- start start 2023-11-28T13:03:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.786+0000 7f1ec82a7b80 1 Processor -- start 2023-11-28T13:03:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e97d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e97da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95c1d0 0x558c5e952970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e707980 con 0x558c5e95c1d0 2023-11-28T13:03:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e95eea0 con 0x558c5e9591e0 2023-11-28T13:03:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c5e97d3a0 con 0x558c5e95b8c0 2023-11-28T13:03:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 10 client.0 _ll_register_callbacks cb 0x558c5e7292f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95c1d0 0x558c5e952970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95c1d0 0x558c5e952970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42480/0 (socket says 172.21.15.153:42480) 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 -- 172.21.15.153:0/3188786474 learned_addr learned my addr 172.21.15.153:0/3188786474 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e97da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e97d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 -- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 msgr2=0x558c5e97da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e97da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 -- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 msgr2=0x558c5e97d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e97d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 -- 172.21.15.153:0/3188786474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb4071f40 con 0x558c5e95c1d0 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95c1d0 0x558c5e952970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00b680 tx=0x7f1eac004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa07ed6b34dc0a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.794 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1e94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1e94ff9700 1 -- 172.21.15.153:0/3188786474 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00bd20 con 0x558c5e95c1d0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1e94ff9700 1 -- 172.21.15.153:0/3188786474 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1eac028c40 con 0x558c5e95c1d0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.787+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c5e8511f0 con 0x558c5e95c1d0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1e94ff9700 1 -- 172.21.15.153:0/3188786474 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eac00be80 con 0x558c5e95c1d0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 10 client.6219 fetch_fsmap learned FSMap version 31 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x558c5e6b1e40 con 0x558c5e95c1d0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1e94ff9700 1 -- 172.21.15.153:0/3188786474 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c5e8511f0 con 0x558c5e95c1d0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1e94ff9700 1 -- 172.21.15.153:0/3188786474 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1eac004030 con 0x558c5e95c1d0 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 10 client.6219 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 10 client.6219 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 20 client.6219 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:07.795 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e952eb0 0x558c5e9532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 4 client.6219 mds_command: new command op to 5991 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c5e707fc0 con 0x558c5e952eb0 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e952eb0 0x558c5e9532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1e94ff9700 10 client.6219 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.788+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e952eb0 0x558c5e9532b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33b1db46f4c66a7 server_cookie=e643ad373138c0d in_seq=0 out_seq=0 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1e94ff9700 1 -- 172.21.15.153:0/3188786474 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x558c5e707fc0 con 0x558c5e952eb0 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1e94ff9700 10 client.6219 handle_command_reply: tid=0 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 client.6219 shutdown 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95c1d0 msgr2=0x558c5e952970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95c1d0 0x558c5e952970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00b680 tx=0x7f1eac004b40 comp rx=0 tx=0).stop 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 shutdown_connections 2023-11-28T13:03:07.796 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e952eb0 0x558c5e9532b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95c1d0 0x558c5e952970 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e97d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1e94ff9700 0 client.6219 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/3188786474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95b8c0 0x558c5e97da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 >> 172.21.15.153:0/3188786474 conn(0x558c5e92ee10 msgr2=0x558c5e92f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 shutdown_connections 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/3188786474 wait complete. 2023-11-28T13:03:07.797 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.789+0000 7f1ec82a7b80 20 client.6219 trim_cache size 0 max 16384 2023-11-28T13:03:07.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 Processor -- start 2023-11-28T13:03:07.806 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 -- start start 2023-11-28T13:03:07.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e970750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e97aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95c1d0 0x558c5e97afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e92e620 con 0x558c5e95b8c0 2023-11-28T13:03:07.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e95b310 con 0x558c5e9591e0 2023-11-28T13:03:07.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.806+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c5e97a630 con 0x558c5e95c1d0 2023-11-28T13:03:07.809 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e97aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e97aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42486/0 (socket says 172.21.15.153:42486) 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1163679996 learned_addr learned my addr 172.21.15.153:0/1163679996 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95c1d0 0x558c5e97afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e970750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95c1d0 msgr2=0x558c5e97afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95c1d0 0x558c5e97afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 msgr2=0x558c5e970750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e970750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 -- 172.21.15.153:0/1163679996 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c5e95b540 con 0x558c5e95b8c0 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e97aaa0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cbf0 tx=0x7f1eb001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7350192dd20b8fd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1e97fff700 1 -- 172.21.15.153:0/1163679996 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00832b0 con 0x558c5e95b8c0 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1e97fff700 1 -- 172.21.15.153:0/1163679996 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb00537f0 con 0x558c5e95b8c0 2023-11-28T13:03:07.810 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.807+0000 7f1e97fff700 1 -- 172.21.15.153:0/1163679996 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb005d2f0 con 0x558c5e95b8c0 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1163679996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 msgr2=0x558c5e97aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e97aaa0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cbf0 tx=0x7f1eb001d000 comp rx=0 tx=0).stop 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1163679996 shutdown_connections 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e95c1d0 0x558c5e97afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e9591e0 0x558c5e970750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/1163679996 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e97aaa0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1163679996 >> 172.21.15.153:0/1163679996 conn(0x558c5e92ee10 msgr2=0x558c5e95f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1163679996 shutdown_connections 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/1163679996 wait complete. 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 -- start start 2023-11-28T13:03:07.811 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.808+0000 7f1ec82a7b80 1 Processor -- start 2023-11-28T13:03:07.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e9591e0 0x558c5e9dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e98efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1ec82a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e98f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.815 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e978d20 con 0x558c5e95b8c0 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c5e9dc7b0 con 0x558c5e95c1d0 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1ec82a7b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c5e97b2e0 con 0x558c5e9591e0 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1ec82a7b80 10 client.0 _ll_register_callbacks cb 0x558c5e7292f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e98efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1eb96b9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e98efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42492/0 (socket says 172.21.15.153:42492) 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.809+0000 7f1eb96b9700 1 -- 172.21.15.153:0/2226699902 learned_addr learned my addr 172.21.15.153:0/2226699902 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eba6bb700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e98f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e9591e0 0x558c5e9dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb96b9700 1 -- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e9591e0 msgr2=0x558c5e9dbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e9591e0 0x558c5e9dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb96b9700 1 -- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 msgr2=0x558c5e98f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e98f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.816 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb96b9700 1 -- 172.21.15.153:0/2226699902 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c5e705690 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e9591e0 0x558c5e9dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb96b9700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e98efd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00553e0 tx=0x7f1eb0058070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ad40163420d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1e94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1e94ff9700 1 -- 172.21.15.153:0/2226699902 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00832b0 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1e94ff9700 1 -- 172.21.15.153:0/2226699902 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1eb0082070 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c5e841210 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1e94ff9700 1 -- 172.21.15.153:0/2226699902 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eb0051240 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 10 client.6225 fetch_fsmap learned FSMap version 31 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x558c5e69acc0 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1e94ff9700 1 -- 172.21.15.153:0/2226699902 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c5e841210 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1e94ff9700 1 -- 172.21.15.153:0/2226699902 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1eb00513a0 con 0x558c5e95b8c0 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 10 client.6225 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:03:07.817 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 10 client.6225 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 20 client.6225 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e97cb20 0x558c5e979170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 4 client.6225 mds_command: new command op to 5991 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x558c5e82b220 con 0x558c5e97cb20 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.810+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e97cb20 0x558c5e979170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.811+0000 7f1e94ff9700 10 client.6225 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.811+0000 7f1eb9eba700 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e97cb20 0x558c5e979170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d726ddf714d708d server_cookie=e2e820875fc236e8 in_seq=0 out_seq=0 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.811+0000 7f1e94ff9700 1 -- 172.21.15.153:0/2226699902 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558c5e82b220 con 0x558c5e97cb20 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.811+0000 7f1e94ff9700 10 client.6225 handle_command_reply: tid=0 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.811+0000 7f1ec82a7b80 1 client.6225 shutdown 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 msgr2=0x558c5e98efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e98efd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00553e0 tx=0x7f1eb0058070 comp rx=0 tx=0).stop 2023-11-28T13:03:07.818 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 shutdown_connections 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x558c5e97cb20 0x558c5e979170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c5e95c1d0 0x558c5e98f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1e94ff9700 0 client.6225 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c5e9591e0 0x558c5e9dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 --2- 172.21.15.153:0/2226699902 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x558c5e95b8c0 0x558c5e98efd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 >> 172.21.15.153:0/2226699902 conn(0x558c5e92ee10 msgr2=0x558c5e95efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 shutdown_connections 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 1 -- 172.21.15.153:0/2226699902 wait complete. 2023-11-28T13:03:07.819 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ec82a7b80 20 client.6225 trim_cache size 0 max 16384 2023-11-28T13:03:07.819 INFO:tasks.ceph.mds.c.smithi153.stderr:2023-11-28T13:03:07.812+0000 7fe70ec80700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5393=0-4194304@1} caps={5393=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5393 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x555c33c4bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:03:07.820 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:03:07.820 INFO:teuthology.orchestra.run.smithi153.stdout: "return_code": 0, 2023-11-28T13:03:07.820 INFO:teuthology.orchestra.run.smithi153.stdout: "scrub_tag": "7df5a9be-9538-4cce-ab33-e01a954dd78e", 2023-11-28T13:03:07.820 INFO:teuthology.orchestra.run.smithi153.stdout: "mode": "asynchronous" 2023-11-28T13:03:07.820 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e980682f0 msgr2=0x7f1e9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e980682f0 0x7f1e9806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40011e0 tx=0x7f1ea4001be0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1eb40af060 msgr2=0x7f1eb415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.812+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1eb40af060 0x7f1eb415e500 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c430 tx=0x7f1eb000c7f0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 shutdown_connections 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eb40af430 0x7f1eb40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eb406a980 0x7f1eb41557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1e980682f0 0x7f1e9806a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 --2- 172.21.15.153:0/96185156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1eb40af060 0x7f1eb415e500 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.821 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 >> 172.21.15.153:0/96185156 conn(0x7f1eb406b380 msgr2=0x7f1eb40cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 shutdown_connections 2023-11-28T13:03:07.822 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.813+0000 7f1ebc11e700 1 -- 172.21.15.153:0/96185156 wait complete. 2023-11-28T13:03:07.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T13:03:07.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.945+0000 7fb245af4700 1 Processor -- start 2023-11-28T13:03:07.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.945+0000 7fb245af4700 1 -- start start 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb245af4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb245af4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb2400ae9d0 0x7fb24006e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb245af4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb24015e540 0x7fb24015e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb245af4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24015bf10 con 0x7fb2400ae9d0 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb245af4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24006ee10 con 0x7fb24015e540 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb245af4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400aee40 con 0x7fb24006e500 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb23effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb2400ae9d0 0x7fb24006e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb23effd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb2400ae9d0 0x7fb24006e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42500/0 (socket says 172.21.15.153:42500) 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.946+0000 7fb23effd700 1 -- 172.21.15.153:0/3355836524 learned_addr learned my addr 172.21.15.153:0/3355836524 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3355836524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23ffff700 1 --2- 172.21.15.153:0/3355836524 >> v2:172.21.15.177:3300/0 conn(0x7fb24015e540 0x7fb24015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23effd700 1 -- 172.21.15.153:0/3355836524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 msgr2=0x7fb24015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23effd700 1 --2- 172.21.15.153:0/3355836524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23effd700 1 -- 172.21.15.153:0/3355836524 >> v2:172.21.15.177:3300/0 conn(0x7fb24015e540 msgr2=0x7fb24015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23effd700 1 --2- 172.21.15.153:0/3355836524 >> v2:172.21.15.177:3300/0 conn(0x7fb24015e540 0x7fb24015e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23effd700 1 -- 172.21.15.153:0/3355836524 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24006c930 con 0x7fb2400ae9d0 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23ffff700 1 --2- 172.21.15.153:0/3355836524 >> v2:172.21.15.177:3300/0 conn(0x7fb24015e540 0x7fb24015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23effd700 1 --2- 172.21.15.153:0/3355836524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb24006e8d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb2400b2170 tx=0x7fb230009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b1df3fd2a9590b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23e7fc700 1 -- 172.21.15.153:0/3355836524 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000b870 con 0x7fb2400ae9d0 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23e7fc700 1 -- 172.21.15.153:0/3355836524 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23000b9d0 con 0x7fb2400ae9d0 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.947+0000 7fb23e7fc700 1 -- 172.21.15.153:0/3355836524 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000bd60 con 0x7fb2400ae9d0 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 -- 172.21.15.153:0/3355836524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 msgr2=0x7fb24006e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 --2- 172.21.15.153:0/3355836524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb24006e8d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb2400b2170 tx=0x7fb230009bd0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 -- 172.21.15.153:0/3355836524 shutdown_connections 2023-11-28T13:03:07.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 --2- 172.21.15.153:0/3355836524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 --2- 172.21.15.153:0/3355836524 >> v2:172.21.15.177:3300/0 conn(0x7fb24015e540 0x7fb24015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 --2- 172.21.15.153:0/3355836524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb24006e8d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 -- 172.21.15.153:0/3355836524 >> 172.21.15.153:0/3355836524 conn(0x7fb2400b12b0 msgr2=0x7fb2400b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 -- 172.21.15.153:0/3355836524 shutdown_connections 2023-11-28T13:03:07.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.948+0000 7fb245af4700 1 -- 172.21.15.153:0/3355836524 wait complete. 2023-11-28T13:03:07.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.949+0000 7fb245af4700 1 Processor -- start 2023-11-28T13:03:07.953 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.949+0000 7fb245af4700 1 -- start start 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb245af4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb245af4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb240158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb245af4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb24015e540 0x7fb240159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb245af4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb240150fe0 con 0x7fb2400ae9d0 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb245af4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401e5400 con 0x7fb24015e540 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb245af4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401e6fe0 con 0x7fb24006e500 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb240158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb240158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42506/0 (socket says 172.21.15.153:42506) 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 -- 172.21.15.153:0/2947219110 learned_addr learned my addr 172.21.15.153:0/2947219110 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23ffff700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb24015e540 0x7fb240159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.954 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 -- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 msgr2=0x7fb24015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 -- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb24015e540 msgr2=0x7fb240159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb24015e540 0x7fb240159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 -- 172.21.15.153:0/2947219110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230009550 con 0x7fb2400ae9d0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23ffff700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb24015e540 0x7fb240159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb23effd700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb240158fd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb230000c00 tx=0x7fb230004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb227fff700 1 -- 172.21.15.153:0/2947219110 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230004540 con 0x7fb2400ae9d0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2400664c0 con 0x7fb2400ae9d0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.950+0000 7fb227fff700 1 -- 172.21.15.153:0/2947219110 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2300036a0 con 0x7fb2400ae9d0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.951+0000 7fb227fff700 1 -- 172.21.15.153:0/2947219110 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230003a50 con 0x7fb2400ae9d0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.951+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24006cc10 con 0x7fb2400ae9d0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.952+0000 7fb227fff700 1 -- 172.21.15.153:0/2947219110 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb23000f050 con 0x7fb2400ae9d0 2023-11-28T13:03:07.955 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.952+0000 7fb227fff700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb22c0682f0 0x7fb22c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.952+0000 7fb227fff700 1 -- 172.21.15.153:0/2947219110 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2300875b0 con 0x7fb2400ae9d0 2023-11-28T13:03:07.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.952+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb22c0682f0 0x7fb22c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.956 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.953+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb22c0682f0 0x7fb22c06a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb2340096f0 tx=0x7fb234008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.954+0000 7fb251c7db80 1 Processor -- start 2023-11-28T13:03:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.954+0000 7fb251c7db80 1 -- start start 2023-11-28T13:03:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df65860 0x55d02df3ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df63180 0x55d02df65c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb251c7db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02de25c10 con 0x55d02df65860 2023-11-28T13:03:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02de943a0 con 0x55d02df66170 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02de8d8d0 con 0x55d02df63180 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df65860 0x55d02df3ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df65860 0x55d02df3ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42512/0 (socket says 172.21.15.153:42512) 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb23f7fe700 1 -- 172.21.15.153:0/524091359 learned_addr learned my addr 172.21.15.153:0/524091359 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb23ffff700 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb23effd700 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df63180 0x55d02df65c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb23f7fe700 1 -- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df63180 msgr2=0x55d02df65c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.959 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.955+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df63180 0x55d02df65c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb23f7fe700 1 -- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 msgr2=0x55d02df817c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df817c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb23f7fe700 1 -- 172.21.15.153:0/524091359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02dd0f630 con 0x55d02df65860 2023-11-28T13:03:07.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df65860 0x55d02df3ba70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb234015960 tx=0x7fb234011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d08439bdefe2797 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.960 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb2257fa700 1 -- 172.21.15.153:0/524091359 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2340137a0 con 0x55d02df65860 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb2257fa700 1 -- 172.21.15.153:0/524091359 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb234013900 con 0x55d02df65860 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb2257fa700 1 -- 172.21.15.153:0/524091359 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234013cb0 con 0x55d02df65860 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb251c7db80 1 -- 172.21.15.153:0/524091359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df65860 msgr2=0x55d02df3ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.956+0000 7fb251c7db80 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df65860 0x55d02df3ba70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb234015960 tx=0x7fb234011000 comp rx=0 tx=0).stop 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 -- 172.21.15.153:0/524091359 shutdown_connections 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df65860 0x55d02df3ba70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df817c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 --2- 172.21.15.153:0/524091359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df63180 0x55d02df65c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 -- 172.21.15.153:0/524091359 >> 172.21.15.153:0/524091359 conn(0x55d02df38db0 msgr2=0x55d02df39740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 -- 172.21.15.153:0/524091359 shutdown_connections 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 -- 172.21.15.153:0/524091359 wait complete. 2023-11-28T13:03:07.961 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 -- start start 2023-11-28T13:03:07.962 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.957+0000 7fb251c7db80 1 Processor -- start 2023-11-28T13:03:07.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df57ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.965 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df58ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df59010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02dd11920 con 0x55d02df63180 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02df68690 con 0x55d02df66170 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02df64220 con 0x55d02df65860 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 10 client.0 _ll_register_callbacks cb 0x55d02dd33290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df57ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df57ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42518/0 (socket says 172.21.15.153:42518) 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3432437625 learned_addr learned my addr 172.21.15.153:0/3432437625 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23effd700 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df58ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23ffff700 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df59010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 msgr2=0x55d02df58ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df58ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 msgr2=0x55d02df59010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df59010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3432437625 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2340155f0 con 0x55d02df63180 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df57ac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb234023040 tx=0x7fb2340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93d71b85cf7039f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb21e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3432437625 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234020640 con 0x55d02df63180 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3432437625 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2340207a0 con 0x55d02df63180 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d02de5b190 con 0x55d02df63180 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3432437625 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb234020960 con 0x55d02df63180 2023-11-28T13:03:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 10 client.6237 fetch_fsmap learned FSMap version 31 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.959+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55d02dcbbde0 con 0x55d02df63180 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3432437625 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d02de5b190 con 0x55d02df63180 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3432437625 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb234020ac0 con 0x55d02df63180 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb251c7db80 10 client.6237 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb251c7db80 10 client.6237 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb251c7db80 20 client.6237 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df58030 0x55d02df67840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb251c7db80 4 client.6237 mds_command: new command op to 5991 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:07.968 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d02dd11f60 con 0x55d02df58030 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb23effd700 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df58030 0x55d02df67840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb21e7fc700 10 client.6237 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.960+0000 7fb23effd700 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df58030 0x55d02df67840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc679881f900b25 server_cookie=6c852fc705f45c15 in_seq=0 out_seq=0 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3432437625 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55d02dd11f60 con 0x55d02df58030 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb21e7fc700 10 client.6237 handle_command_reply: tid=0 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 client.6237 shutdown 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 msgr2=0x55d02df57ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df57ac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb234023040 tx=0x7fb2340047c0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 shutdown_connections 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df58030 0x55d02df67840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.969 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df57ac0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb21e7fc700 0 client.6237 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df66170 0x55d02df59010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3432437625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df58ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 >> 172.21.15.153:0/3432437625 conn(0x55d02df38db0 msgr2=0x55d02df34f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 shutdown_connections 2023-11-28T13:03:07.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 1 -- 172.21.15.153:0/3432437625 wait complete. 2023-11-28T13:03:07.970 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.961+0000 7fb251c7db80 20 client.6237 trim_cache size 0 max 16384 2023-11-28T13:03:07.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.977+0000 7fb251c7db80 1 Processor -- start 2023-11-28T13:03:07.978 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.977+0000 7fb251c7db80 1 -- start start 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df83d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df65860 0x55d02df67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df66170 0x55d02df63550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb251c7db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02df30ad0 con 0x55d02df63180 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02df84440 con 0x55d02df65860 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02df65fb0 con 0x55d02df66170 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df83d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb23f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df83d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42526/0 (socket says 172.21.15.153:42526) 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3299740631 learned_addr learned my addr 172.21.15.153:0/3299740631 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3299740631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df66170 msgr2=0x55d02df63550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.978+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df66170 0x55d02df63550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3299740631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df65860 msgr2=0x55d02df67a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb23effd700 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df65860 0x55d02df67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df65860 0x55d02df67a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb23f7fe700 1 -- 172.21.15.153:0/3299740631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02de54190 con 0x55d02df63180 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df83d10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb234004840 tx=0x7fb23402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7854e14837d2fc22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb225ffb700 1 -- 172.21.15.153:0/3299740631 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234034640 con 0x55d02df63180 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb225ffb700 1 -- 172.21.15.153:0/3299740631 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb234019040 con 0x55d02df63180 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb225ffb700 1 -- 172.21.15.153:0/3299740631 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234013ba0 con 0x55d02df63180 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb251c7db80 1 -- 172.21.15.153:0/3299740631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 msgr2=0x55d02df83d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.979+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df83d10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb234004840 tx=0x7fb23402c000 comp rx=0 tx=0).stop 2023-11-28T13:03:07.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 -- 172.21.15.153:0/3299740631 shutdown_connections 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df63180 0x55d02df83d10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df66170 0x55d02df63550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3299740631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df65860 0x55d02df67a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 -- 172.21.15.153:0/3299740631 >> 172.21.15.153:0/3299740631 conn(0x55d02df38db0 msgr2=0x55d02df578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 -- 172.21.15.153:0/3299740631 shutdown_connections 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 -- 172.21.15.153:0/3299740631 wait complete. 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 -- start start 2023-11-28T13:03:07.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.980+0000 7fb251c7db80 1 Processor -- start 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df63180 0x55d02df76000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb251c7db80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df66170 0x55d02df63550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb251c7db80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02df76af0 con 0x55d02df66170 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02df9a0d0 con 0x55d02df63180 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb251c7db80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02df592b0 con 0x55d02df65860 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb251c7db80 10 client.0 _ll_register_callbacks cb 0x55d02dd33290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df66170 0x55d02df63550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df66170 0x55d02df63550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42536/0 (socket says 172.21.15.153:42536) 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 -- 172.21.15.153:0/3440171469 learned_addr learned my addr 172.21.15.153:0/3440171469 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:07.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df63180 0x55d02df76000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23effd700 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 -- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 msgr2=0x55d02df585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 -- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df63180 msgr2=0x55d02df76000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df63180 0x55d02df76000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.981+0000 7fb23ffff700 1 -- 172.21.15.153:0/3440171469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d02dcbbde0 con 0x55d02df66170 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb23ffff700 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df66170 0x55d02df63550 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb238007a20 tx=0x7fb238007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9868d22db2176cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb21e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3440171469 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23800e070 con 0x55d02df66170 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3440171469 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb238015660 con 0x55d02df66170 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d02de4b1b0 con 0x55d02df66170 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3440171469 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb238015820 con 0x55d02df66170 2023-11-28T13:03:07.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 10 client.6243 fetch_fsmap learned FSMap version 31 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55d02dce2250 con 0x55d02df66170 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3440171469 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d02de4b1b0 con 0x55d02df66170 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3440171469 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb238015980 con 0x55d02df66170 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 10 client.6243 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 10 client.6243 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 20 client.6243 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df62ad0 0x55d02df63a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 4 client.6243 mds_command: new command op to 5991 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d02de351c0 con 0x55d02df62ad0 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df62ad0 0x55d02df63a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb21e7fc700 10 client.6243 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.982+0000 7fb23f7fe700 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df62ad0 0x55d02df63a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6044e38494583ccb server_cookie=c40b00944c7d405c in_seq=0 out_seq=0 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb21e7fc700 1 -- 172.21.15.153:0/3440171469 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55d02de351c0 con 0x55d02df62ad0 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb21e7fc700 10 client.6243 handle_command_reply: tid=0 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 client.6243 shutdown 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df66170 msgr2=0x55d02df63550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df66170 0x55d02df63550 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb238007a20 tx=0x7fb238007a50 comp rx=0 tx=0).stop 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 shutdown_connections 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x55d02df62ad0 0x55d02df63a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55d02df66170 0x55d02df63550 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d02df63180 0x55d02df76000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb21e7fc700 0 client.6243 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 --2- 172.21.15.153:0/3440171469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d02df65860 0x55d02df585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 >> 172.21.15.153:0/3440171469 conn(0x55d02df38db0 msgr2=0x55d02df3c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 shutdown_connections 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 1 -- 172.21.15.153:0/3440171469 wait complete. 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb251c7db80 20 client.6243 trim_cache size 0 max 16384 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stdout: "scrubs": { 2023-11-28T13:03:07.991 INFO:teuthology.orchestra.run.smithi153.stdout: "7df5a9be-9538-4cce-ab33-e01a954dd78e": { 2023-11-28T13:03:07.992 INFO:teuthology.orchestra.run.smithi153.stdout: "path": "/", 2023-11-28T13:03:07.992 INFO:teuthology.orchestra.run.smithi153.stdout: "tag": "7df5a9be-9538-4cce-ab33-e01a954dd78e", 2023-11-28T13:03:07.992 INFO:teuthology.orchestra.run.smithi153.stdout: "options": "recursive" 2023-11-28T13:03:07.992 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:03:07.992 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:03:07.992 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb22c0682f0 msgr2=0x7fb22c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb22c0682f0 0x7fb22c06a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb2340096f0 tx=0x7fb234008040 comp rx=0 tx=0).stop 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 msgr2=0x7fb240158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb240158fd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb230000c00 tx=0x7fb230004750 comp rx=0 tx=0).stop 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 shutdown_connections 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb24015e540 0x7fb240159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb24006e500 0x7fb24015de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb22c0682f0 0x7fb22c06a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 --2- 172.21.15.153:0/2947219110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2400ae9d0 0x7fb240158fd0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.983+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 >> 172.21.15.153:0/2947219110 conn(0x7fb2400b12b0 msgr2=0x7fb2400b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.984+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 shutdown_connections 2023-11-28T13:03:07.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:07.984+0000 7fb245af4700 1 -- 172.21.15.153:0/2947219110 wait complete. 2023-11-28T13:03:07.996 INFO:tasks.cephfs.filesystem:scrub status for tag:7df5a9be-9538-4cce-ab33-e01a954dd78e - {'path': '/', 'tag': '7df5a9be-9538-4cce-ab33-e01a954dd78e', 'options': 'recursive'} 2023-11-28T13:03:11.406 INFO:tasks.ceph.mon.a.smithi153.stderr:2023-11-28T13:03:11.404+0000 7feb34d4f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T13:03:16.517 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:16.521 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:37.998 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T13:03:38.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.132+0000 7f248c36c700 1 Processor -- start 2023-11-28T13:03:38.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.133+0000 7f248c36c700 1 -- start start 2023-11-28T13:03:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.133+0000 7f248c36c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248406c990 0x7f248406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.133+0000 7f248c36c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f24840ca240 0x7f24840ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.133+0000 7f248c36c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248406d320 con 0x7f248406e9f0 2023-11-28T13:03:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.133+0000 7f248c36c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248406ae60 con 0x7f24840ca240 2023-11-28T13:03:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.133+0000 7f248c36c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248406a270 con 0x7f248406c990 2023-11-28T13:03:38.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f24840ca240 0x7f24840ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f24840ca240 0x7f24840ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42544/0 (socket says 172.21.15.153:42544) 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 -- 172.21.15.153:0/1927005894 learned_addr learned my addr 172.21.15.153:0/1927005894 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f2489907700 1 --2- 172.21.15.153:0/1927005894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248406c990 0x7f248406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 -- 172.21.15.153:0/1927005894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248406c990 msgr2=0x7f248406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 --2- 172.21.15.153:0/1927005894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248406c990 0x7f248406cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 -- 172.21.15.153:0/1927005894 >> v1:172.21.15.177:6789/0 conn(0x7f248406e9f0 legacy=0x7f248406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 -- 172.21.15.153:0/1927005894 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24840cabe0 con 0x7f24840ca240 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f2489907700 1 --2- 172.21.15.153:0/1927005894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248406c990 0x7f248406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f248a909700 1 --2- 172.21.15.153:0/1927005894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24840ca240 0x7f24840ca6a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f247c009e50 tx=0x7f247c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b233e216136313 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f2489106700 1 -- 172.21.15.153:0/1927005894 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00e070 con 0x7f24840ca240 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f2489106700 1 -- 172.21.15.153:0/1927005894 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c003a20 con 0x7f24840ca240 2023-11-28T13:03:38.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.134+0000 7f2489106700 1 -- 172.21.15.153:0/1927005894 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c003dd0 con 0x7f24840ca240 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 -- 172.21.15.153:0/1927005894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24840ca240 msgr2=0x7f24840ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 --2- 172.21.15.153:0/1927005894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24840ca240 0x7f24840ca6a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f247c009e50 tx=0x7f247c00b330 comp rx=0 tx=0).stop 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 -- 172.21.15.153:0/1927005894 shutdown_connections 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 --2- 172.21.15.153:0/1927005894 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f24840ca240 0x7f24840ca6a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 --2- 172.21.15.153:0/1927005894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248406c990 0x7f248406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 -- 172.21.15.153:0/1927005894 >> 172.21.15.153:0/1927005894 conn(0x7f24840b12b0 msgr2=0x7f24840b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 -- 172.21.15.153:0/1927005894 shutdown_connections 2023-11-28T13:03:38.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.135+0000 7f248c36c700 1 -- 172.21.15.153:0/1927005894 wait complete. 2023-11-28T13:03:38.150 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.136+0000 7f248c36c700 1 Processor -- start 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.136+0000 7f248c36c700 1 -- start start 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248c36c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248406c990 0x7f248415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248c36c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f248406e9f0 0x7f24841f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248c36c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24840ca240 0x7f24841f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248c36c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840c59d0 con 0x7f248406e9f0 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248c36c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484165e50 con 0x7f248406c990 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248c36c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840b4ff0 con 0x7f24840ca240 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f248406e9f0 0x7f24841f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f248406e9f0 0x7f24841f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42548/0 (socket says 172.21.15.153:42548) 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 -- 172.21.15.153:0/353488848 learned_addr learned my addr 172.21.15.153:0/353488848 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248a909700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24840ca240 0x7f24841f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.151 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 -- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24840ca240 msgr2=0x7f24841f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248a108700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248406c990 0x7f248415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24840ca240 0x7f24841f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 -- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248406c990 msgr2=0x7f248415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248406c990 0x7f248415bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 -- 172.21.15.153:0/353488848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c009b30 con 0x7f248406e9f0 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248a909700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24840ca240 0x7f24841f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f248a108700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248406c990 0x7f248415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f2489907700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f248406e9f0 0x7f24841f4190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f248000c780 tx=0x7f248000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.137+0000 7f247affd700 1 -- 172.21.15.153:0/353488848 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480011800 con 0x7f248406e9f0 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.138+0000 7f247affd700 1 -- 172.21.15.153:0/353488848 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480011960 con 0x7f248406e9f0 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.138+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248406a810 con 0x7f248406e9f0 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.138+0000 7f247affd700 1 -- 172.21.15.153:0/353488848 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480011cd0 con 0x7f248406e9f0 2023-11-28T13:03:38.152 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.138+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248406a530 con 0x7f248406e9f0 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.139+0000 7f247affd700 1 -- 172.21.15.153:0/353488848 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2480020070 con 0x7f248406e9f0 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.139+0000 7f247affd700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f24680682f0 0x7f246806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.139+0000 7f247affd700 1 -- 172.21.15.153:0/353488848 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2480085bb0 con 0x7f248406e9f0 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.140+0000 7f248a108700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f24680682f0 0x7f246806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.140+0000 7f248a108700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f24680682f0 0x7f246806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2484156f30 tx=0x7f2474009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.141+0000 7f24984f5b80 1 Processor -- start 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f24984f5b80 1 -- start start 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe087860 0x5626fe05da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe087c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe088170 0x5626fe0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f24984f5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fdf47c10 con 0x5626fe087860 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fdfb63a0 con 0x5626fe088170 2023-11-28T13:03:38.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fdfaf8d0 con 0x5626fe085180 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f248a108700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe087860 0x5626fe05da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f248a108700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe087860 0x5626fe05da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42554/0 (socket says 172.21.15.153:42554) 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.142+0000 7f248a108700 1 -- 172.21.15.153:0/422118351 learned_addr learned my addr 172.21.15.153:0/422118351 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f248a909700 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe088170 0x5626fe0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f2489907700 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe087c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f248a108700 1 -- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 msgr2=0x5626fe087c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f248a108700 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe087c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f248a108700 1 -- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe088170 msgr2=0x5626fe0a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f248a108700 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe088170 0x5626fe0a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f248a108700 1 -- 172.21.15.153:0/422118351 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626fde31630 con 0x5626fe087860 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f248a108700 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe087860 0x5626fe05da70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f24740055a0 tx=0x7f2474010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebfef5c7af9e435 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f2467fff700 1 -- 172.21.15.153:0/422118351 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474003ea0 con 0x5626fe087860 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f2467fff700 1 -- 172.21.15.153:0/422118351 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2474012820 con 0x5626fe087860 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.143+0000 7f2467fff700 1 -- 172.21.15.153:0/422118351 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474005180 con 0x5626fe087860 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 -- 172.21.15.153:0/422118351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe087860 msgr2=0x5626fe05da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe087860 0x5626fe05da70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f24740055a0 tx=0x7f2474010000 comp rx=0 tx=0).stop 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 -- 172.21.15.153:0/422118351 shutdown_connections 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe087860 0x5626fe05da70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe088170 0x5626fe0a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 --2- 172.21.15.153:0/422118351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe087c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 -- 172.21.15.153:0/422118351 >> 172.21.15.153:0/422118351 conn(0x5626fe05adb0 msgr2=0x5626fe05b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 -- 172.21.15.153:0/422118351 shutdown_connections 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 -- 172.21.15.153:0/422118351 wait complete. 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 -- start start 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.144+0000 7f24984f5b80 1 Processor -- start 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe0762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe0869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe076840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f24984f5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fde33920 con 0x5626fe088170 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fe079cd0 con 0x5626fe087860 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fe0856c0 con 0x5626fe085180 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f24984f5b80 10 client.0 _ll_register_callbacks cb 0x5626fde55290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f248a909700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe076840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f248a909700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe076840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42562/0 (socket says 172.21.15.153:42562) 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.145+0000 7f248a909700 1 -- 172.21.15.153:0/2267179891 learned_addr learned my addr 172.21.15.153:0/2267179891 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f248a108700 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe0762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f2489907700 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe0869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f248a909700 1 -- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 msgr2=0x5626fe0762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f248a909700 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe0762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f248a909700 1 -- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 msgr2=0x5626fe0869b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f248a909700 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe0869b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f248a909700 1 -- 172.21.15.153:0/2267179891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24740052e0 con 0x5626fe088170 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f248a909700 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe076840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f24840704f0 tx=0x7f247c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef23f82488642f7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24657fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24657fa700 1 -- 172.21.15.153:0/2267179891 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c004230 con 0x5626fe088170 2023-11-28T13:03:38.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24657fa700 1 -- 172.21.15.153:0/2267179891 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f247c004390 con 0x5626fe088170 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626fdf7d190 con 0x5626fe088170 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24657fa700 1 -- 172.21.15.153:0/2267179891 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f247c004550 con 0x5626fe088170 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 10 client.6255 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5626fddddde0 con 0x5626fe088170 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24657fa700 1 -- 172.21.15.153:0/2267179891 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626fdf7d190 con 0x5626fe088170 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24657fa700 1 -- 172.21.15.153:0/2267179891 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f247c0046b0 con 0x5626fe088170 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 10 client.6255 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 10 client.6255 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 20 client.6255 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe076de0 0x5626fe079150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 4 client.6255 mds_command: new command op to 5991 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:38.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.146+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626fde33f60 con 0x5626fe076de0 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.147+0000 7f248a108700 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe076de0 0x5626fe079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.147+0000 7f24657fa700 10 client.6255 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.147+0000 7f248a108700 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe076de0 0x5626fe079150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734c6456eb6f44a9 server_cookie=94beaa449040a96b in_seq=0 out_seq=0 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.147+0000 7f24657fa700 1 -- 172.21.15.153:0/2267179891 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5626fde33f60 con 0x5626fe076de0 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.147+0000 7f24657fa700 10 client.6255 handle_command_reply: tid=0 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.147+0000 7f24984f5b80 1 client.6255 shutdown 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.147+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 msgr2=0x5626fe076840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe076840 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f24840704f0 tx=0x7f247c004b40 comp rx=0 tx=0).stop 2023-11-28T13:03:38.159 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 shutdown_connections 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe076de0 0x5626fe079150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe076840 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24657fa700 0 client.6255 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe0762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 --2- 172.21.15.153:0/2267179891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe0869b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 >> 172.21.15.153:0/2267179891 conn(0x5626fe05adb0 msgr2=0x5626fe05d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 shutdown_connections 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 1 -- 172.21.15.153:0/2267179891 wait complete. 2023-11-28T13:03:38.160 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.148+0000 7f24984f5b80 20 client.6255 trim_cache size 0 max 16384 2023-11-28T13:03:38.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.164+0000 7f24984f5b80 1 Processor -- start 2023-11-28T13:03:38.165 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.164+0000 7f24984f5b80 1 -- start start 2023-11-28T13:03:38.167 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe07a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe07f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe07fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f24984f5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fe077bc0 con 0x5626fe088170 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fe0807b0 con 0x5626fe087860 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fe05b6b0 con 0x5626fe085180 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe07fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe07fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42572/0 (socket says 172.21.15.153:42572) 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 -- 172.21.15.153:0/1029768618 learned_addr learned my addr 172.21.15.153:0/1029768618 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a108700 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe07a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 -- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 msgr2=0x5626fe07a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe07a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.168 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 -- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 msgr2=0x5626fe07f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f2489907700 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe07f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe07f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 -- 172.21.15.153:0/1029768618 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626fdf76190 con 0x5626fe088170 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f248a909700 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe07fa30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f247c002230 tx=0x7f247c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8553c76a694536f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.165+0000 7f2478dad700 1 -- 172.21.15.153:0/1029768618 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c009490 con 0x5626fe088170 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f2478dad700 1 -- 172.21.15.153:0/1029768618 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c026660 con 0x5626fe088170 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f2478dad700 1 -- 172.21.15.153:0/1029768618 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c026a10 con 0x5626fe088170 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 -- 172.21.15.153:0/1029768618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 msgr2=0x5626fe07fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe07fa30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f247c002230 tx=0x7f247c01e000 comp rx=0 tx=0).stop 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 -- 172.21.15.153:0/1029768618 shutdown_connections 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe07fa30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe085180 0x5626fe07a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.169 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1029768618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe087860 0x5626fe07f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 -- 172.21.15.153:0/1029768618 >> 172.21.15.153:0/1029768618 conn(0x5626fe05adb0 msgr2=0x5626fe05ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 -- 172.21.15.153:0/1029768618 shutdown_connections 2023-11-28T13:03:38.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.166+0000 7f24984f5b80 1 -- 172.21.15.153:0/1029768618 wait complete. 2023-11-28T13:03:38.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.167+0000 7f24984f5b80 1 -- start start 2023-11-28T13:03:38.170 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.167+0000 7f24984f5b80 1 Processor -- start 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe085180 0x5626fe0f4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe087860 0x5626fe056740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe056c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fe0f5260 con 0x5626fe088170 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626fe05c280 con 0x5626fe085180 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626fe0f4ea0 con 0x5626fe087860 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 10 client.0 _ll_register_callbacks cb 0x5626fde55290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe056c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe056c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42578/0 (socket says 172.21.15.153:42578) 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 -- 172.21.15.153:0/1887667008 learned_addr learned my addr 172.21.15.153:0/1887667008 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f2489907700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe087860 0x5626fe056740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a108700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe085180 0x5626fe0f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 -- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe087860 msgr2=0x5626fe056740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe087860 0x5626fe056740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 -- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe085180 msgr2=0x5626fe0f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe085180 0x5626fe0f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 -- 172.21.15.153:0/1887667008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626fddddde0 con 0x5626fe088170 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a108700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe085180 0x5626fe0f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f248a909700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe056c80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f247c00d010 tx=0x7f247c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850983f8534a61b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24657fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24657fa700 1 -- 172.21.15.153:0/1887667008 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00e030 con 0x5626fe088170 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24657fa700 1 -- 172.21.15.153:0/1887667008 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f247c004410 con 0x5626fe088170 2023-11-28T13:03:38.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.168+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626fdf6d1b0 con 0x5626fe088170 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24657fa700 1 -- 172.21.15.153:0/1887667008 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f247c0264e0 con 0x5626fe088170 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 10 client.6261 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5626fde04250 con 0x5626fe088170 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24657fa700 1 -- 172.21.15.153:0/1887667008 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626fdf6d1b0 con 0x5626fe088170 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24657fa700 1 -- 172.21.15.153:0/1887667008 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f247c0045d0 con 0x5626fe088170 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 10 client.6261 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 10 client.6261 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 20 client.6261 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe0f53f0 0x5626fe10fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 4 client.6261 mds_command: new command op to 5991 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5626fdf571c0 con 0x5626fe0f53f0 2023-11-28T13:03:38.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f248a108700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe0f53f0 0x5626fe10fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24657fa700 10 client.6261 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f248a108700 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe0f53f0 0x5626fe10fad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b246700593b1f625 server_cookie=99e3282db93b90b1 in_seq=0 out_seq=0 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24657fa700 1 -- 172.21.15.153:0/1887667008 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5626fdf571c0 con 0x5626fe0f53f0 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24657fa700 10 client.6261 handle_command_reply: tid=0 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.169+0000 7f24984f5b80 1 client.6261 shutdown 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 msgr2=0x5626fe056c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe056c80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f247c00d010 tx=0x7f247c009810 comp rx=0 tx=0).stop 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 shutdown_connections 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x5626fe0f53f0 0x5626fe10fad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5626fe088170 0x5626fe056c80 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626fe085180 0x5626fe0f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.177 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24657fa700 0 client.6261 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 --2- 172.21.15.153:0/1887667008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626fe087860 0x5626fe056740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 >> 172.21.15.153:0/1887667008 conn(0x5626fe05adb0 msgr2=0x5626fe0b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 shutdown_connections 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 1 -- 172.21.15.153:0/1887667008 wait complete. 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f24984f5b80 20 client.6261 trim_cache size 0 max 16384 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stdout: "status": "no active scrubs running", 2023-11-28T13:03:38.178 INFO:teuthology.orchestra.run.smithi153.stdout: "scrubs": {} 2023-11-28T13:03:38.179 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:03:38.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f24680682f0 msgr2=0x7f246806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.179 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f24680682f0 0x7f246806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2484156f30 tx=0x7f2474009040 comp rx=0 tx=0).stop 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f248406e9f0 msgr2=0x7f24841f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f248406e9f0 0x7f24841f4190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f248000c780 tx=0x7f248000cb40 comp rx=0 tx=0).stop 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 shutdown_connections 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24840ca240 0x7f24841f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f248406c990 0x7f248415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f24680682f0 0x7f246806a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 --2- 172.21.15.153:0/353488848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f248406e9f0 0x7f24841f4190 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 >> 172.21.15.153:0/353488848 conn(0x7f24840b12b0 msgr2=0x7f24840b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 shutdown_connections 2023-11-28T13:03:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.170+0000 7f248c36c700 1 -- 172.21.15.153:0/353488848 wait complete. 2023-11-28T13:03:38.183 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T13:03:38.183 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:38.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.303+0000 7fb9d383e700 1 Processor -- start 2023-11-28T13:03:38.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.303+0000 7fb9d383e700 1 -- start start 2023-11-28T13:03:38.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d383e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb9cc06a4e0 0x7fb9cc071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d383e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc154670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d383e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc06df90 con 0x7fb9cc13fd10 2023-11-28T13:03:38.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d383e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc06eb70 con 0x7fb9cc06a4e0 2023-11-28T13:03:38.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d383e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc15e8d0 con 0x7fb9cc06a8b0 2023-11-28T13:03:38.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d15da700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb9cc06a4e0 0x7fb9cc071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d0dd9700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fb9cc13fd10 0x7fb9cc15e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60746/0 (socket says 172.21.15.153:60746) 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d0dd9700 1 -- 172.21.15.153:0/2765891282 learned_addr learned my addr 172.21.15.153:0/2765891282 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d1ddb700 1 --2- 172.21.15.153:0/2765891282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d15da700 1 -- 172.21.15.153:0/2765891282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 msgr2=0x7fb9cc154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d15da700 1 --2- 172.21.15.153:0/2765891282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d15da700 1 -- 172.21.15.153:0/2765891282 >> v1:172.21.15.153:6789/0 conn(0x7fb9cc13fd10 legacy=0x7fb9cc15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9c3fff700 1 -- 172.21.15.153:0/2765891282 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704269647 0 0) 0x7fb9cc06df90 con 0x7fb9cc13fd10 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d15da700 1 -- 172.21.15.153:0/2765891282 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc140210 con 0x7fb9cc06a4e0 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9d15da700 1 --2- 172.21.15.153:0/2765891282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc06a4e0 0x7fb9cc071b30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009870 tx=0x7fb9c8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7541fb90a8cf69ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9c3fff700 1 -- 172.21.15.153:0/2765891282 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8011830 con 0x7fb9cc06a4e0 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9c3fff700 1 -- 172.21.15.153:0/2765891282 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c8011990 con 0x7fb9cc06a4e0 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.304+0000 7fb9c3fff700 1 -- 172.21.15.153:0/2765891282 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8011d20 con 0x7fb9cc06a4e0 2023-11-28T13:03:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 -- 172.21.15.153:0/2765891282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc06a4e0 msgr2=0x7fb9cc071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 --2- 172.21.15.153:0/2765891282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc06a4e0 0x7fb9cc071b30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009870 tx=0x7fb9c8009b80 comp rx=0 tx=0).stop 2023-11-28T13:03:38.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 -- 172.21.15.153:0/2765891282 shutdown_connections 2023-11-28T13:03:38.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 --2- 172.21.15.153:0/2765891282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 --2- 172.21.15.153:0/2765891282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc06a4e0 0x7fb9cc071b30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 -- 172.21.15.153:0/2765891282 >> 172.21.15.153:0/2765891282 conn(0x7fb9cc0b19f0 msgr2=0x7fb9cc0b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 -- 172.21.15.153:0/2765891282 shutdown_connections 2023-11-28T13:03:38.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.305+0000 7fb9d383e700 1 -- 172.21.15.153:0/2765891282 wait complete. 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.306+0000 7fb9d383e700 1 Processor -- start 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.306+0000 7fb9d383e700 1 -- start start 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d383e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9cc06a4e0 0x7fb9cc0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d383e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d383e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc13fd10 0x7fb9cc132ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d383e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc136150 con 0x7fb9cc13fd10 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d383e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc133220 con 0x7fb9cc06a4e0 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d383e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0b45f0 con 0x7fb9cc06a8b0 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc13fd10 0x7fb9cc132ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc13fd10 0x7fb9cc132ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42592/0 (socket says 172.21.15.153:42592) 2023-11-28T13:03:38.311 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 -- 172.21.15.153:0/4252844325 learned_addr learned my addr 172.21.15.153:0/4252844325 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d0dd9700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d15da700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9cc06a4e0 0x7fb9cc0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 -- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 msgr2=0x7fb9cc155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 -- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9cc06a4e0 msgr2=0x7fb9cc0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9cc06a4e0 0x7fb9cc0b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.307+0000 7fb9d1ddb700 1 -- 172.21.15.153:0/4252844325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80094a0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.308+0000 7fb9d1ddb700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc13fd10 0x7fb9cc132ce0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400efc0 tx=0x7fb9c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.308+0000 7fb9c1ffb700 1 -- 172.21.15.153:0/4252844325 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4009ea0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.308+0000 7fb9c1ffb700 1 -- 172.21.15.153:0/4252844325 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c400f040 con 0x7fb9cc13fd10 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.308+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc1407d0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.312 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.308+0000 7fb9c1ffb700 1 -- 172.21.15.153:0/4252844325 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400ced0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.308+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc1404f0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.309+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc140ab0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.309+0000 7fb9c1ffb700 1 -- 172.21.15.153:0/4252844325 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9c4014480 con 0x7fb9cc13fd10 2023-11-28T13:03:38.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.310+0000 7fb9c1ffb700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9b00682f0 0x7fb9b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.310+0000 7fb9d15da700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9b00682f0 0x7fb9b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.310+0000 7fb9c1ffb700 1 -- 172.21.15.153:0/4252844325 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb9c40865b0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.313 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.310+0000 7fb9d15da700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9b00682f0 0x7fb9b006a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80097c0 tx=0x7fb9c8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.315 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.314+0000 7fb9c1ffb700 1 -- 172.21.15.153:0/4252844325 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9c4052db0 con 0x7fb9cc13fd10 2023-11-28T13:03:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.538+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9cc068690 con 0x7fb9cc13fd10 2023-11-28T13:03:38.539 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.538+0000 7fb9c1ffb700 1 -- 172.21.15.153:0/4252844325 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7fb9c4057e60 con 0x7fb9cc13fd10 2023-11-28T13:03:38.539 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:38.539 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:03:11.406430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:03:38.541 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9b00682f0 msgr2=0x7fb9b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.541 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9b00682f0 0x7fb9b006a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80097c0 tx=0x7fb9c8008030 comp rx=0 tx=0).stop 2023-11-28T13:03:38.541 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc13fd10 msgr2=0x7fb9cc132ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.541 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc13fd10 0x7fb9cc132ce0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400efc0 tx=0x7fb9c400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 shutdown_connections 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb9b00682f0 0x7fb9b006a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9cc13fd10 0x7fb9cc132ce0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9cc06a4e0 0x7fb9cc0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 --2- 172.21.15.153:0/4252844325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9cc06a8b0 0x7fb9cc155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 >> 172.21.15.153:0/4252844325 conn(0x7fb9cc0b19f0 msgr2=0x7fb9cc158650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 shutdown_connections 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.540+0000 7fb9d383e700 1 -- 172.21.15.153:0/4252844325 wait complete. 2023-11-28T13:03:38.542 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2023-11-28T13:03:38.553 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2023-11-28T13:03:38.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.675+0000 7f3a742c8700 1 Processor -- start 2023-11-28T13:03:38.675 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.675+0000 7f3a742c8700 1 -- start start 2023-11-28T13:03:38.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a742c8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a6c06ba90 0x7f3a6c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a742c8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a742c8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c0cc5f0 con 0x7f3a6c15cef0 2023-11-28T13:03:38.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a742c8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c15d470 con 0x7f3a6c06ba90 2023-11-28T13:03:38.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a742c8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c15d750 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a6c06ba90 0x7f3a6c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.678 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3a6c06ba90 0x7f3a6c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42598/0 (socket says 172.21.15.153:42598) 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 -- 172.21.15.153:0/3457012931 learned_addr learned my addr 172.21.15.153:0/3457012931 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a72865700 1 --2- 172.21.15.153:0/3457012931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 -- 172.21.15.153:0/3457012931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0cbdf0 msgr2=0x7f3a6c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 --2- 172.21.15.153:0/3457012931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 -- 172.21.15.153:0/3457012931 >> v1:172.21.15.177:6789/0 conn(0x7f3a6c15cef0 legacy=0x7f3a6c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 -- 172.21.15.153:0/3457012931 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c15da30 con 0x7f3a6c06ba90 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71863700 1 --2- 172.21.15.153:0/3457012931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c0af400 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800bb90 tx=0x7f3a6800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68afebfe48cabef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71062700 1 -- 172.21.15.153:0/3457012931 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800dd50 con 0x7f3a6c06ba90 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71062700 1 -- 172.21.15.153:0/3457012931 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a68010470 con 0x7f3a6c06ba90 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.676+0000 7f3a71062700 1 -- 172.21.15.153:0/3457012931 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68010800 con 0x7f3a6c06ba90 2023-11-28T13:03:38.679 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 -- 172.21.15.153:0/3457012931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c06ba90 msgr2=0x7f3a6c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3457012931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c0af400 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800bb90 tx=0x7f3a6800bea0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 -- 172.21.15.153:0/3457012931 shutdown_connections 2023-11-28T13:03:38.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3457012931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3457012931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c0af400 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 -- 172.21.15.153:0/3457012931 >> 172.21.15.153:0/3457012931 conn(0x7f3a6c0d0000 msgr2=0x7f3a6c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 -- 172.21.15.153:0/3457012931 shutdown_connections 2023-11-28T13:03:38.680 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.677+0000 7f3a742c8700 1 -- 172.21.15.153:0/3457012931 wait complete. 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.678+0000 7f3a742c8700 1 Processor -- start 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.678+0000 7f3a742c8700 1 -- start start 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a742c8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a742c8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a742c8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c15cef0 0x7f3a6c0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a742c8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c7bf0 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a742c8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c7ed0 con 0x7f3a6c06ba90 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a742c8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c81b0 con 0x7f3a6c15cef0 2023-11-28T13:03:38.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42604/0 (socket says 172.21.15.153:42604) 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 -- 172.21.15.153:0/3336779689 learned_addr learned my addr 172.21.15.153:0/3336779689 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a72865700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c15cef0 0x7f3a6c0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a72064700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 -- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c15cef0 msgr2=0x7f3a6c0c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c15cef0 0x7f3a6c0c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 -- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c06ba90 msgr2=0x7f3a6c154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a71863700 1 -- 172.21.15.153:0/3336779689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6800b7f0 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.679+0000 7f3a72064700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.680+0000 7f3a71863700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0c3b10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3a68000c00 tx=0x7f3a68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.680+0000 7f3a5effd700 1 -- 172.21.15.153:0/3336779689 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6801d070 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.680+0000 7f3a5effd700 1 -- 172.21.15.153:0/3336779689 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a68004040 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.680+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c15dff0 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.680+0000 7f3a5effd700 1 -- 172.21.15.153:0/3336779689 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680043f0 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.680+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c15dd10 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.681+0000 7f3a5effd700 1 -- 172.21.15.153:0/3336779689 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3a680036a0 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.682+0000 7f3a5effd700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a500682f0 0x7f3a5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.682+0000 7f3a5effd700 1 -- 172.21.15.153:0/3336779689 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a68087300 con 0x7f3a6c0cbdf0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.682+0000 7f3a72064700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a500682f0 0x7f3a5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.685 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.682+0000 7f3a72064700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a500682f0 0x7f3a5006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3a600016e0 tx=0x7f3a60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.684+0000 7f3a80451b80 1 Processor -- start 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.684+0000 7f3a80451b80 1 -- start start 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323428b0 0x561632318ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632342c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323431c0 0x56163235e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a80451b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561632202c60 con 0x5616323428b0 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616322713f0 con 0x5616323431c0 2023-11-28T13:03:38.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56163226a920 con 0x5616323401d0 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323428b0 0x561632318ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323428b0 0x561632318ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42610/0 (socket says 172.21.15.153:42610) 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 -- 172.21.15.153:0/739228901 learned_addr learned my addr 172.21.15.153:0/739228901 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a71863700 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632342c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72865700 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323431c0 0x56163235e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 -- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 msgr2=0x561632342c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632342c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 -- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323431c0 msgr2=0x56163235e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323431c0 0x56163235e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 -- 172.21.15.153:0/739228901 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616320ec680 con 0x5616323428b0 2023-11-28T13:03:38.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a72064700 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323428b0 0x561632318ac0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3a600053e0 tx=0x7f3a60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874a26ec8d5c9c2e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a4ffff700 1 -- 172.21.15.153:0/739228901 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60003ea0 con 0x5616323428b0 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a4ffff700 1 -- 172.21.15.153:0/739228901 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60021450 con 0x5616323428b0 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.685+0000 7f3a4ffff700 1 -- 172.21.15.153:0/739228901 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600217c0 con 0x5616323428b0 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 -- 172.21.15.153:0/739228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323428b0 msgr2=0x561632318ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323428b0 0x561632318ac0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3a600053e0 tx=0x7f3a60011000 comp rx=0 tx=0).stop 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 -- 172.21.15.153:0/739228901 shutdown_connections 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323428b0 0x561632318ac0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323431c0 0x56163235e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 --2- 172.21.15.153:0/739228901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632342c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 -- 172.21.15.153:0/739228901 >> 172.21.15.153:0/739228901 conn(0x561632315e00 msgr2=0x561632316790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 -- 172.21.15.153:0/739228901 shutdown_connections 2023-11-28T13:03:38.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.686+0000 7f3a80451b80 1 -- 172.21.15.153:0/739228901 wait complete. 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.687+0000 7f3a80451b80 1 -- start start 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.687+0000 7f3a80451b80 1 Processor -- start 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632341b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x5616323341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x561632331300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616320ee970 con 0x5616323431c0 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561632342040 con 0x5616323428b0 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x561632334e70 con 0x5616323401d0 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 10 client.0 _ll_register_callbacks cb 0x5616321102e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x561632331300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x561632331300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42620/0 (socket says 172.21.15.153:42620) 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 -- 172.21.15.153:0/879384629 learned_addr learned my addr 172.21.15.153:0/879384629 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72064700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632341b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a71863700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x5616323341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 -- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 msgr2=0x561632341b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632341b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 -- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 msgr2=0x5616323341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x5616323341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 -- 172.21.15.153:0/879384629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6000f040 con 0x5616323431c0 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a71863700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x5616323341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a72865700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x561632331300 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3a64004320 tx=0x7f3a64004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfd81c75ab534d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/879384629 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64007660 con 0x5616323431c0 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/879384629 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a640077c0 con 0x5616323431c0 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.688+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616322381e0 con 0x5616323431c0 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/879384629 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a64007980 con 0x5616323431c0 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 10 client.6282 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x561632098e30 con 0x5616323431c0 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/879384629 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5616322381e0 con 0x5616323431c0 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/879384629 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3a64007ae0 con 0x5616323431c0 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 10 client.6282 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 10 client.6282 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 20 client.6282 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632355d70 0x561632334730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 4 client.6282 mds_command: new command op to 5991 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616320eefb0 con 0x561632355d70 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a72064700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632355d70 0x561632334730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a72064700 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632355d70 0x561632334730 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5642b569159d84 server_cookie=903804a85a95f1af in_seq=0 out_seq=0 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.689+0000 7f3a4d7fa700 10 client.6282 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/879384629 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5616320eefb0 con 0x561632355d70 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a4d7fa700 10 client.6282 handle_command_reply: tid=0 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 client.6282 shutdown 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 msgr2=0x561632331300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x561632331300 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3a64004320 tx=0x7f3a64004630 comp rx=0 tx=0).stop 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 shutdown_connections 2023-11-28T13:03:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632355d70 0x561632334730 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x561632331300 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a4d7fa700 0 client.6282 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x561632341b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 --2- 172.21.15.153:0/879384629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x5616323341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 >> 172.21.15.153:0/879384629 conn(0x561632315e00 msgr2=0x5616323180b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 shutdown_connections 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 1 -- 172.21.15.153:0/879384629 wait complete. 2023-11-28T13:03:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.690+0000 7f3a80451b80 20 client.6282 trim_cache size 0 max 16384 2023-11-28T13:03:38.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.703+0000 7f3a80451b80 1 Processor -- start 2023-11-28T13:03:38.704 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.703+0000 7f3a80451b80 1 -- start start 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323401d0 0x561632319b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632332130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323431c0 0x561632332670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a80451b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561632356d80 con 0x5616323401d0 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163233b9a0 con 0x5616323428b0 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56163231a080 con 0x5616323431c0 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323401d0 0x561632319b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323401d0 0x561632319b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42624/0 (socket says 172.21.15.153:42624) 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 -- 172.21.15.153:0/848841946 learned_addr learned my addr 172.21.15.153:0/848841946 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 -- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323431c0 msgr2=0x561632332670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72865700 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323431c0 0x561632332670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a71863700 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632332130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323431c0 0x561632332670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 -- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 msgr2=0x561632332130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632332130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 -- 172.21.15.153:0/848841946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616322311e0 con 0x5616323401d0 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a72064700 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323401d0 0x561632319b40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3a600053b0 tx=0x7f3a60005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f44b95c14b90080 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a5cdad700 1 -- 172.21.15.153:0/848841946 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6002f430 con 0x5616323401d0 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a5cdad700 1 -- 172.21.15.153:0/848841946 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60019040 con 0x5616323401d0 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.704+0000 7f3a5cdad700 1 -- 172.21.15.153:0/848841946 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6002fa40 con 0x5616323401d0 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 -- 172.21.15.153:0/848841946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323401d0 msgr2=0x561632319b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323401d0 0x561632319b40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3a600053b0 tx=0x7f3a60005000 comp rx=0 tx=0).stop 2023-11-28T13:03:38.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 -- 172.21.15.153:0/848841946 shutdown_connections 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323401d0 0x561632319b40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323431c0 0x561632332670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 --2- 172.21.15.153:0/848841946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632332130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 -- 172.21.15.153:0/848841946 >> 172.21.15.153:0/848841946 conn(0x561632315e00 msgr2=0x561632331110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 -- 172.21.15.153:0/848841946 shutdown_connections 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 -- 172.21.15.153:0/848841946 wait complete. 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.705+0000 7f3a80451b80 1 -- start start 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 1 Processor -- start 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x5616323543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632339630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x56163235fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x561632339b70 con 0x5616323431c0 2023-11-28T13:03:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561632319a50 con 0x5616323428b0 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616323609c0 con 0x5616323401d0 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.706+0000 7f3a80451b80 10 client.0 _ll_register_callbacks cb 0x5616321102e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x56163235fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x56163235fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42634/0 (socket says 172.21.15.153:42634) 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 -- 172.21.15.153:0/3267427484 learned_addr learned my addr 172.21.15.153:0/3267427484 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72064700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x5616323543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a71863700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632339630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 -- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 msgr2=0x5616323543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x5616323543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 -- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 msgr2=0x561632339630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632339630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.714 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 -- 172.21.15.153:0/3267427484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616320ec680 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a71863700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632339630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a72865700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x56163235fca0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x561632332da0 tx=0x7f3a64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ed342cf845dd37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.153:3300/0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/3267427484 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64016ae0 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/3267427484 <== mon.0 v2:172.21.15.153:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a64016c40 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561632228200 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/3267427484 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a64016e00 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a80451b80 10 client.6288 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5616320bf2a0 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.707+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/3267427484 <== mon.0 v2:172.21.15.153:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561632228200 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/3267427484 <== mon.0 v2:172.21.15.153:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3a64007660 con 0x5616323431c0 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 10 client.6288 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:38.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 10 client.6288 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 20 client.6288 populate_metadata read hostname 'smithi153' 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632317ee0 0x5616323182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 4 client.6288 mds_command: new command op to 5991 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 --> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x561632212210 con 0x561632317ee0 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a72064700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632317ee0 0x5616323182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a4d7fa700 10 client.6288 ms_handle_connect on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a72064700 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632317ee0 0x5616323182b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be392fe0401dc46 server_cookie=e68c7c1d08227601 in_seq=0 out_seq=0 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a4d7fa700 1 -- 172.21.15.153:0/3267427484 <== mds.0 v2:172.21.15.153:6835/3815439165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x561632212210 con 0x561632317ee0 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a4d7fa700 10 client.6288 handle_command_reply: tid=0 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 client.6288 shutdown 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 msgr2=0x56163235fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x56163235fca0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x561632332da0 tx=0x7f3a64004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 shutdown_connections 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:6835/3815439165,v1:172.21.15.153:6837/3815439165] conn(0x561632317ee0 0x5616323182b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x5616323431c0 0x56163235fca0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5616323401d0 0x5616323543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a4d7fa700 0 client.6288 ms_handle_reset on v2:172.21.15.153:6835/3815439165 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 --2- 172.21.15.153:0/3267427484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616323428b0 0x561632339630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.708+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 >> 172.21.15.153:0/3267427484 conn(0x561632315e00 msgr2=0x5616323141a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 shutdown_connections 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a80451b80 1 -- 172.21.15.153:0/3267427484 wait complete. 2023-11-28T13:03:38.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a80451b80 20 client.6288 trim_cache size 0 max 16384 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout:[ 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout: { 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout: "damage_type": "backtrace", 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 2029526810, 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout: "ino": 1099511627777, 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout: "path": "/dir_test/file" 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout: } 2023-11-28T13:03:38.718 INFO:teuthology.orchestra.run.smithi153.stdout:] 2023-11-28T13:03:38.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a500682f0 msgr2=0x7f3a5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a500682f0 0x7f3a5006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3a600016e0 tx=0x7f3a60009040 comp rx=0 tx=0).stop 2023-11-28T13:03:38.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c0cbdf0 msgr2=0x7f3a6c0c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0c3b10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3a68000c00 tx=0x7f3a68004750 comp rx=0 tx=0).stop 2023-11-28T13:03:38.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 shutdown_connections 2023-11-28T13:03:38.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a6c15cef0 0x7f3a6c0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a6c06ba90 0x7f3a6c154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a500682f0 0x7f3a5006a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 --2- 172.21.15.153:0/3336779689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3a6c0cbdf0 0x7f3a6c0c3b10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 >> 172.21.15.153:0/3336779689 conn(0x7f3a6c0d0000 msgr2=0x7f3a6c06d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 shutdown_connections 2023-11-28T13:03:38.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.709+0000 7f3a742c8700 1 -- 172.21.15.153:0/3336779689 wait complete. 2023-11-28T13:03:38.722 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:03:38.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.840+0000 7f2176607700 1 Processor -- start 2023-11-28T13:03:38.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.840+0000 7f2176607700 1 -- start start 2023-11-28T13:03:38.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.841+0000 7f2176607700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f217006f850 0x7f217006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.841+0000 7f2176607700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217013ff00 con 0x7f21700cd8c0 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.841+0000 7f2176607700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21701404c0 con 0x7f2170140950 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.841+0000 7f2176607700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2170140dc0 con 0x7f217006f850 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.841+0000 7f216ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f21700cd8c0 0x7f21700cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60802/0 (socket says 172.21.15.153:60802) 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.841+0000 7f216ffff700 1 -- 172.21.15.153:0/4293036995 learned_addr learned my addr 172.21.15.153:0/4293036995 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f2174ba4700 1 --2- 172.21.15.153:0/4293036995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f217006f850 0x7f217006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238492967 0 0) 0x7f217013ff00 con 0x7f21700cd8c0 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21580035b0 con 0x7f21700cd8c0 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529781417 0 0) 0x7f21701404c0 con 0x7f2170140950 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f217013ff00 con 0x7f2170140950 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3705276514 0 0) 0x7f21580035b0 con 0x7f21700cd8c0 2023-11-28T13:03:38.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f2174ba4700 1 -- 172.21.15.153:0/4293036995 >> v1:172.21.15.177:6789/0 conn(0x7f2170140950 legacy=0x7f21700b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f2174ba4700 1 -- 172.21.15.153:0/4293036995 >> v1:172.21.15.153:6789/0 conn(0x7f21700cd8c0 legacy=0x7f21700cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f2174ba4700 1 -- 172.21.15.153:0/4293036995 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217015aa80 con 0x7f217006f850 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f2174ba4700 1 --2- 172.21.15.153:0/4293036995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f217006f850 0x7f217006fcc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2168009dc0 tx=0x7f216800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c3962a96649284 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216800e070 con 0x7f217006f850 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168003a20 con 0x7f217006f850 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.842+0000 7f216effd700 1 -- 172.21.15.153:0/4293036995 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21600017f0 con 0x7f217006f850 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.843+0000 7f2176607700 1 -- 172.21.15.153:0/4293036995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f217006f850 msgr2=0x7f217006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.843+0000 7f2176607700 1 --2- 172.21.15.153:0/4293036995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f217006f850 0x7f217006fcc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2168009dc0 tx=0x7f216800b330 comp rx=0 tx=0).stop 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.843+0000 7f2176607700 1 -- 172.21.15.153:0/4293036995 shutdown_connections 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.843+0000 7f2176607700 1 --2- 172.21.15.153:0/4293036995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f217006f850 0x7f217006fcc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.843+0000 7f2176607700 1 -- 172.21.15.153:0/4293036995 >> 172.21.15.153:0/4293036995 conn(0x7f217006aed0 msgr2=0x7f217006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:38.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.843+0000 7f2176607700 1 -- 172.21.15.153:0/4293036995 shutdown_connections 2023-11-28T13:03:38.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.843+0000 7f2176607700 1 -- 172.21.15.153:0/4293036995 wait complete. 2023-11-28T13:03:38.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.844+0000 7f2176607700 1 Processor -- start 2023-11-28T13:03:38.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.844+0000 7f2176607700 1 -- start start 2023-11-28T13:03:38.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f2176607700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f217006f850 0x7f21700c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f2176607700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21700cd8c0 0x7f21700b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f2176607700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2170140950 0x7f21701efad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f2176607700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2170141ba0 con 0x7f21700cd8c0 2023-11-28T13:03:38.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f2176607700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2170141e80 con 0x7f217006f850 2023-11-28T13:03:38.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f2176607700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2170142160 con 0x7f2170140950 2023-11-28T13:03:38.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21700cd8c0 0x7f21700b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21700cd8c0 0x7f21700b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42646/0 (socket says 172.21.15.153:42646) 2023-11-28T13:03:38.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 -- 172.21.15.153:0/1530054387 learned_addr learned my addr 172.21.15.153:0/1530054387 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:38.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216ffff700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f217006f850 0x7f21700c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f2174ba4700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2170140950 0x7f21701efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 -- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2170140950 msgr2=0x7f21701efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2170140950 0x7f21701efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 -- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f217006f850 msgr2=0x7f21700c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f217006f850 0x7f21700c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.845+0000 7f216f7fe700 1 -- 172.21.15.153:0/1530054387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168009a70 con 0x7f21700cd8c0 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.846+0000 7f216f7fe700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21700cd8c0 0x7f21700b28a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f216400c4d0 tx=0x7f216400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.846+0000 7f216cff9700 1 -- 172.21.15.153:0/1530054387 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640052d0 con 0x7f21700cd8c0 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.846+0000 7f216cff9700 1 -- 172.21.15.153:0/1530054387 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2164005430 con 0x7f21700cd8c0 2023-11-28T13:03:38.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.846+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21700b3310 con 0x7f21700cd8c0 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.846+0000 7f216cff9700 1 -- 172.21.15.153:0/1530054387 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164003880 con 0x7f21700cd8c0 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.846+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21700b3040 con 0x7f21700cd8c0 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.847+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21700b35f0 con 0x7f21700cd8c0 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.847+0000 7f216cff9700 1 -- 172.21.15.153:0/1530054387 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2164019030 con 0x7f21700cd8c0 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.847+0000 7f216cff9700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f215c0682f0 0x7f215c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.847+0000 7f216cff9700 1 -- 172.21.15.153:0/1530054387 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2164084d30 con 0x7f21700cd8c0 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.848+0000 7f216ffff700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f215c0682f0 0x7f215c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:38.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.848+0000 7f216ffff700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f215c0682f0 0x7f215c06a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2160001170 tx=0x7f2160001d80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:38.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:38.852+0000 7f216cff9700 1 -- 172.21.15.153:0/1530054387 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f216404b5e0 con 0x7f21700cd8c0 2023-11-28T13:03:39.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.078+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2170141ce0 con 0x7f21700cd8c0 2023-11-28T13:03:39.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.078+0000 7f216cff9700 1 -- 172.21.15.153:0/1530054387 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f21640170c0 con 0x7f21700cd8c0 2023-11-28T13:03:39.079 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:39.079 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:03:39.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f215c0682f0 msgr2=0x7f215c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:39.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f215c0682f0 0x7f215c06a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2160001170 tx=0x7f2160001d80 comp rx=0 tx=0).stop 2023-11-28T13:03:39.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21700cd8c0 msgr2=0x7f21700b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:39.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21700cd8c0 0x7f21700b28a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f216400c4d0 tx=0x7f216400c7e0 comp rx=0 tx=0).stop 2023-11-28T13:03:39.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 shutdown_connections 2023-11-28T13:03:39.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2170140950 0x7f21701efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:39.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f217006f850 0x7f21700c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:39.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f215c0682f0 0x7f215c06a7a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:39.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 --2- 172.21.15.153:0/1530054387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21700cd8c0 0x7f21700b28a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:39.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 >> 172.21.15.153:0/1530054387 conn(0x7f217006aed0 msgr2=0x7f217013e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:39.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 shutdown_connections 2023-11-28T13:03:39.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:39.080+0000 7f2176607700 1 -- 172.21.15.153:0/1530054387 wait complete. 2023-11-28T13:03:39.093 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:03:44.093 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:03:44.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.225+0000 7f9d37d79700 1 Processor -- start 2023-11-28T13:03:44.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.226+0000 7f9d37d79700 1 -- start start 2023-11-28T13:03:44.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.226+0000 7f9d37d79700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d30071040 0x7f9d30071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:44.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.226+0000 7f9d37d79700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d3006dee0 0x7f9d3006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:44.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.226+0000 7f9d37d79700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d30071950 con 0x7f9d3006cdc0 2023-11-28T13:03:44.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.226+0000 7f9d37d79700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d30161550 con 0x7f9d3006dee0 2023-11-28T13:03:44.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.226+0000 7f9d37d79700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3015d060 con 0x7f9d30071040 2023-11-28T13:03:44.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35b15700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d30071040 0x7f9d30071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:44.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35314700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d3006dee0 0x7f9d3006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:44.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35314700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d3006dee0 0x7f9d3006e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:52118/0 (socket says 172.21.15.153:52118) 2023-11-28T13:03:44.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35b15700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d30071040 0x7f9d30071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:60262/0 (socket says 172.21.15.153:60262) 2023-11-28T13:03:44.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d36316700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f9d3006cdc0 0x7f9d3006d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:37434/0 (socket says 172.21.15.153:37434) 2023-11-28T13:03:44.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35314700 1 -- 172.21.15.153:0/3743334035 learned_addr learned my addr 172.21.15.153:0/3743334035 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:44.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35b15700 1 -- 172.21.15.153:0/3743334035 >> v2:172.21.15.177:3300/0 conn(0x7f9d3006dee0 msgr2=0x7f9d3006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:44.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35b15700 1 --2- 172.21.15.153:0/3743334035 >> v2:172.21.15.177:3300/0 conn(0x7f9d3006dee0 0x7f9d3006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35b15700 1 -- 172.21.15.153:0/3743334035 >> v1:172.21.15.177:6789/0 conn(0x7f9d3006cdc0 legacy=0x7f9d3006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:44.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35b15700 1 -- 172.21.15.153:0/3743334035 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3015d340 con 0x7f9d30071040 2023-11-28T13:03:44.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35314700 1 --2- 172.21.15.153:0/3743334035 >> v2:172.21.15.177:3300/0 conn(0x7f9d3006dee0 0x7f9d3006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:03:44.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.227+0000 7f9d35b15700 1 --2- 172.21.15.153:0/3743334035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d30071040 0x7f9d30071410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009840 tx=0x7f9d24009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f319b600714b8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:44.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d34b13700 1 -- 172.21.15.153:0/3743334035 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24011830 con 0x7f9d30071040 2023-11-28T13:03:44.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d34b13700 1 -- 172.21.15.153:0/3743334035 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24011990 con 0x7f9d30071040 2023-11-28T13:03:44.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d34b13700 1 -- 172.21.15.153:0/3743334035 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24011d20 con 0x7f9d30071040 2023-11-28T13:03:44.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 -- 172.21.15.153:0/3743334035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d30071040 msgr2=0x7f9d30071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:44.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 --2- 172.21.15.153:0/3743334035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d30071040 0x7f9d30071410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009840 tx=0x7f9d24009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:44.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 -- 172.21.15.153:0/3743334035 shutdown_connections 2023-11-28T13:03:44.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 --2- 172.21.15.153:0/3743334035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d30071040 0x7f9d30071410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 --2- 172.21.15.153:0/3743334035 >> v2:172.21.15.177:3300/0 conn(0x7f9d3006dee0 0x7f9d3006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 -- 172.21.15.153:0/3743334035 >> 172.21.15.153:0/3743334035 conn(0x7f9d300d0000 msgr2=0x7f9d300d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:44.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 -- 172.21.15.153:0/3743334035 shutdown_connections 2023-11-28T13:03:44.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.228+0000 7f9d37d79700 1 -- 172.21.15.153:0/3743334035 wait complete. 2023-11-28T13:03:44.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.229+0000 7f9d37d79700 1 Processor -- start 2023-11-28T13:03:44.235 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.229+0000 7f9d37d79700 1 -- start start 2023-11-28T13:03:44.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d37d79700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3006cdc0 0x7f9d300aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:44.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d37d79700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d3006dee0 0x7f9d300b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:44.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d37d79700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d30071040 0x7f9d301578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:44.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d37d79700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300b50c0 con 0x7f9d3006dee0 2023-11-28T13:03:44.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d37d79700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d301e69f0 con 0x7f9d30071040 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d37d79700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300b52f0 con 0x7f9d3006cdc0 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d35314700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d3006dee0 0x7f9d300b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d35314700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d3006dee0 0x7f9d300b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42660/0 (socket says 172.21.15.153:42660) 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d35314700 1 -- 172.21.15.153:0/873659527 learned_addr learned my addr 172.21.15.153:0/873659527 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d35b15700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3006cdc0 0x7f9d300aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.230+0000 7f9d36316700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d30071040 0x7f9d301578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d35314700 1 -- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3006cdc0 msgr2=0x7f9d300aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d35314700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3006cdc0 0x7f9d300aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d35314700 1 -- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d30071040 msgr2=0x7f9d301578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:44.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d35314700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d30071040 0x7f9d301578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d35314700 1 -- 172.21.15.153:0/873659527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009550 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d35314700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d3006dee0 0x7f9d300b4b80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00e9b0 tx=0x7f9d2c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d227fc700 1 -- 172.21.15.153:0/873659527 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00ee50 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d227fc700 1 -- 172.21.15.153:0/873659527 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c00c850 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d240094a0 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d227fc700 1 -- 172.21.15.153:0/873659527 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00cbc0 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.231+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3015d900 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.232+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3015d620 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.232+0000 7f9d227fc700 1 -- 172.21.15.153:0/873659527 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9d2c019030 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.233+0000 7f9d227fc700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d140682f0 0x7f9d1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.233+0000 7f9d227fc700 1 -- 172.21.15.153:0/873659527 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d2c085b30 con 0x7f9d3006dee0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.233+0000 7f9d35b15700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d140682f0 0x7f9d1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:44.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.233+0000 7f9d35b15700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d140682f0 0x7f9d1406a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d24000c00 tx=0x7f9d24008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:44.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.237+0000 7f9d227fc700 1 -- 172.21.15.153:0/873659527 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d2c052390 con 0x7f9d3006dee0 2023-11-28T13:03:44.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.461+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9d3006e890 con 0x7f9d3006dee0 2023-11-28T13:03:44.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.462+0000 7f9d227fc700 1 -- 172.21.15.153:0/873659527 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9d2c0559b0 con 0x7f9d3006dee0 2023-11-28T13:03:44.463 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:44.463 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:03:44.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d140682f0 msgr2=0x7f9d1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:44.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d140682f0 0x7f9d1406a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d24000c00 tx=0x7f9d24008030 comp rx=0 tx=0).stop 2023-11-28T13:03:44.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d3006dee0 msgr2=0x7f9d300b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d3006dee0 0x7f9d300b4b80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00e9b0 tx=0x7f9d2c00ecc0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 shutdown_connections 2023-11-28T13:03:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d30071040 0x7f9d301578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3006cdc0 0x7f9d300aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d140682f0 0x7f9d1406a7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 --2- 172.21.15.153:0/873659527 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d3006dee0 0x7f9d300b4b80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.463+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 >> 172.21.15.153:0/873659527 conn(0x7f9d300d0000 msgr2=0x7f9d3006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.464+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 shutdown_connections 2023-11-28T13:03:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:44.464+0000 7f9d37d79700 1 -- 172.21.15.153:0/873659527 wait complete. 2023-11-28T13:03:44.478 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:03:46.587 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:46.590 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:49.479 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:03:49.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.611+0000 7fc7b40e6700 1 Processor -- start 2023-11-28T13:03:49.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.611+0000 7fc7b40e6700 1 -- start start 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.612+0000 7fc7b40e6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac071040 0x7fc7ac071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.612+0000 7fc7b40e6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc7ac160540 0x7fc7ac1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.612+0000 7fc7b40e6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ac071950 con 0x7fc7ac06dee0 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.612+0000 7fc7b40e6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ac15d060 con 0x7fc7ac160540 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.612+0000 7fc7b40e6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac15d340 con 0x7fc7ac071040 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.612+0000 7fc7b1e82700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac071040 0x7fc7ac071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.612+0000 7fc7b2683700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc7ac160540 0x7fc7ac1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1e82700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac071040 0x7fc7ac071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:60276/0 (socket says 172.21.15.153:60276) 2023-11-28T13:03:49.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2024075428 learned_addr learned my addr 172.21.15.153:0/2024075428 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b2683700 1 --2- 172.21.15.153:0/2024075428 >> v2:172.21.15.177:3300/0 conn(0x7fc7ac160540 0x7fc7ac1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:52136/0 (socket says 172.21.15.153:52136) 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1681700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc7ac06dee0 0x7fc7ac06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:37446/0 (socket says 172.21.15.153:37446) 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2024075428 >> v2:172.21.15.177:3300/0 conn(0x7fc7ac160540 msgr2=0x7fc7ac1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1e82700 1 --2- 172.21.15.153:0/2024075428 >> v2:172.21.15.177:3300/0 conn(0x7fc7ac160540 0x7fc7ac1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2024075428 >> v1:172.21.15.177:6789/0 conn(0x7fc7ac06dee0 legacy=0x7fc7ac06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2024075428 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac15d620 con 0x7fc7ac071040 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b1e82700 1 --2- 172.21.15.153:0/2024075428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac071040 0x7fc7ac071410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0098f0 tx=0x7fc79c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e575211d476f28 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b0e80700 1 -- 172.21.15.153:0/2024075428 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c011830 con 0x7fc7ac071040 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b0e80700 1 -- 172.21.15.153:0/2024075428 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c011990 con 0x7fc7ac071040 2023-11-28T13:03:49.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.613+0000 7fc7b0e80700 1 -- 172.21.15.153:0/2024075428 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c011d20 con 0x7fc7ac071040 2023-11-28T13:03:49.621 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2024075428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac071040 msgr2=0x7fc7ac071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2024075428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac071040 0x7fc7ac071410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0098f0 tx=0x7fc79c009c00 comp rx=0 tx=0).stop 2023-11-28T13:03:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2024075428 shutdown_connections 2023-11-28T13:03:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2024075428 >> v2:172.21.15.177:3300/0 conn(0x7fc7ac160540 0x7fc7ac1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2024075428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac071040 0x7fc7ac071410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2024075428 >> 172.21.15.153:0/2024075428 conn(0x7fc7ac0d0000 msgr2=0x7fc7ac0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2024075428 shutdown_connections 2023-11-28T13:03:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.614+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2024075428 wait complete. 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.615+0000 7fc7b40e6700 1 Processor -- start 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.615+0000 7fc7b40e6700 1 -- start start 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b40e6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7ac06dee0 0x7fc7ac14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b40e6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ac071040 0x7fc7ac0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b40e6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac160540 0x7fc7ac0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b40e6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac14ba00 con 0x7fc7ac06dee0 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b40e6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0c3c40 con 0x7fc7ac071040 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b40e6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac0c3ee0 con 0x7fc7ac160540 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7ac06dee0 0x7fc7ac14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7ac06dee0 0x7fc7ac14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42672/0 (socket says 172.21.15.153:42672) 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2883029594 learned_addr learned my addr 172.21.15.153:0/2883029594 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:49.623 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b2683700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac160540 0x7fc7ac0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac160540 msgr2=0x7fc7ac0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac160540 0x7fc7ac0c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1681700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ac071040 0x7fc7ac0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ac071040 msgr2=0x7fc7ac0c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ac071040 0x7fc7ac0c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 -- 172.21.15.153:0/2883029594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009550 con 0x7fc7ac06dee0 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1681700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ac071040 0x7fc7ac0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.616+0000 7fc7b1e82700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7ac06dee0 0x7fc7ac14b4c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009c40 tx=0x7fc79c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.617+0000 7fc7a27fc700 1 -- 172.21.15.153:0/2883029594 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0036a0 con 0x7fc7ac06dee0 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.617+0000 7fc7a27fc700 1 -- 172.21.15.153:0/2883029594 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c01e070 con 0x7fc7ac06dee0 2023-11-28T13:03:49.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.617+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac15ccd0 con 0x7fc7ac06dee0 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.617+0000 7fc7a27fc700 1 -- 172.21.15.153:0/2883029594 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c003a40 con 0x7fc7ac06dee0 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.617+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac15d900 con 0x7fc7ac06dee0 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.618+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac06a270 con 0x7fc7ac06dee0 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.618+0000 7fc7a27fc700 1 -- 172.21.15.153:0/2883029594 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc79c00f050 con 0x7fc7ac06dee0 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.618+0000 7fc7a27fc700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7900682f0 0x7fc79006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.619+0000 7fc7a27fc700 1 -- 172.21.15.153:0/2883029594 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc79c086c40 con 0x7fc7ac06dee0 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.619+0000 7fc7b1681700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7900682f0 0x7fc79006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:49.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.619+0000 7fc7b1681700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7900682f0 0x7fc79006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac14efc0 tx=0x7fc7a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:49.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.623+0000 7fc7a27fc700 1 -- 172.21.15.153:0/2883029594 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc79c0534a0 con 0x7fc7ac06dee0 2023-11-28T13:03:49.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.845+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc7ac0720a0 con 0x7fc7ac06dee0 2023-11-28T13:03:49.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.846+0000 7fc7a27fc700 1 -- 172.21.15.153:0/2883029594 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fc79c020080 con 0x7fc7ac06dee0 2023-11-28T13:03:49.847 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:49.847 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:03:49.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7900682f0 msgr2=0x7fc79006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:49.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7900682f0 0x7fc79006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac14efc0 tx=0x7fc7a8009040 comp rx=0 tx=0).stop 2023-11-28T13:03:49.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7ac06dee0 msgr2=0x7fc7ac14b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:49.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7ac06dee0 0x7fc7ac14b4c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009c40 tx=0x7fc79c004750 comp rx=0 tx=0).stop 2023-11-28T13:03:49.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 shutdown_connections 2023-11-28T13:03:49.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc7900682f0 0x7fc79006a7a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.850 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7ac06dee0 0x7fc7ac14b4c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7ac160540 0x7fc7ac0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 --2- 172.21.15.153:0/2883029594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ac071040 0x7fc7ac0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:49.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 >> 172.21.15.153:0/2883029594 conn(0x7fc7ac0d0000 msgr2=0x7fc7ac15f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:49.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 shutdown_connections 2023-11-28T13:03:49.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:49.848+0000 7fc7b40e6700 1 -- 172.21.15.153:0/2883029594 wait complete. 2023-11-28T13:03:49.860 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:03:54.862 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:03:54.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.993+0000 7f237c877700 1 Processor -- start 2023-11-28T13:03:54.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.993+0000 7f237c877700 1 -- start start 2023-11-28T13:03:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237c877700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f237815cef0 0x7f23780af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237c877700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f237806ba90 0x7f23780ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237c877700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23780cc5f0 con 0x7f23780cbdf0 2023-11-28T13:03:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237c877700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237815d470 con 0x7f237815cef0 2023-11-28T13:03:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237c877700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237815d750 con 0x7f237806ba90 2023-11-28T13:03:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237659c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f237815cef0 0x7f23780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f2375d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f23780cbdf0 0x7f237806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60844/0 (socket says 172.21.15.153:60844) 2023-11-28T13:03:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f2375d9b700 1 -- 172.21.15.153:0/1587387600 learned_addr learned my addr 172.21.15.153:0/1587387600 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:03:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237659c700 1 -- 172.21.15.153:0/1587387600 >> v2:172.21.15.177:3300/0 conn(0x7f237806ba90 msgr2=0x7f23780ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:54.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237659c700 1 --2- 172.21.15.153:0/1587387600 >> v2:172.21.15.177:3300/0 conn(0x7f237806ba90 0x7f23780ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:54.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237659c700 1 -- 172.21.15.153:0/1587387600 >> v1:172.21.15.153:6789/0 conn(0x7f23780cbdf0 legacy=0x7f237806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:54.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237659c700 1 -- 172.21.15.153:0/1587387600 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237815da30 con 0x7f237815cef0 2023-11-28T13:03:54.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.994+0000 7f237559a700 1 -- 172.21.15.153:0/1587387600 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489566899 0 0) 0x7f23780cc5f0 con 0x7f23780cbdf0 2023-11-28T13:03:54.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237659c700 1 --2- 172.21.15.153:0/1587387600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f23780af400 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f236c009840 tx=0x7f236c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18c9096e0a22606 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:54.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237559a700 1 -- 172.21.15.153:0/1587387600 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c011830 con 0x7f237815cef0 2023-11-28T13:03:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237559a700 1 -- 172.21.15.153:0/1587387600 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c011990 con 0x7f237815cef0 2023-11-28T13:03:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237559a700 1 -- 172.21.15.153:0/1587387600 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c011d20 con 0x7f237815cef0 2023-11-28T13:03:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 -- 172.21.15.153:0/1587387600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 msgr2=0x7f23780af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 --2- 172.21.15.153:0/1587387600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f23780af400 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f236c009840 tx=0x7f236c009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 -- 172.21.15.153:0/1587387600 shutdown_connections 2023-11-28T13:03:55.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 --2- 172.21.15.153:0/1587387600 >> v2:172.21.15.177:3300/0 conn(0x7f237806ba90 0x7f23780ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 --2- 172.21.15.153:0/1587387600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f23780af400 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 -- 172.21.15.153:0/1587387600 >> 172.21.15.153:0/1587387600 conn(0x7f23780d0000 msgr2=0x7f23780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:55.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 -- 172.21.15.153:0/1587387600 shutdown_connections 2023-11-28T13:03:55.000 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.995+0000 7f237c877700 1 -- 172.21.15.153:0/1587387600 wait complete. 2023-11-28T13:03:55.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.997+0000 7f237c877700 1 Processor -- start 2023-11-28T13:03:55.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.997+0000 7f237c877700 1 -- start start 2023-11-28T13:03:55.004 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f237806ba90 0x7f2378156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23780cbdf0 0x7f23781f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f237806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378156bd0 con 0x7f237815cef0 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23781f3180 con 0x7f237806ba90 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23781f3460 con 0x7f23780cbdf0 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f237806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f237806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42690/0 (socket says 172.21.15.153:42690) 2023-11-28T13:03:55.005 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 -- 172.21.15.153:0/1411247384 learned_addr learned my addr 172.21.15.153:0/1411247384 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2375d9b700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23780cbdf0 0x7f23781f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237659c700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f237806ba90 0x7f2378156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 -- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23780cbdf0 msgr2=0x7f23781f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23780cbdf0 0x7f23781f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 -- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f237806ba90 msgr2=0x7f2378156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f237806ba90 0x7f2378156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 -- 172.21.15.153:0/1411247384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0094a0 con 0x7f237815cef0 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2376d9d700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f237806aba0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f237000b9e0 tx=0x7f237000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2366ffd700 1 -- 172.21.15.153:0/1411247384 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237000d8e0 con 0x7f237815cef0 2023-11-28T13:03:55.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2366ffd700 1 -- 172.21.15.153:0/1411247384 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237000da40 con 0x7f237815cef0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237815dff0 con 0x7f237815cef0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f2366ffd700 1 -- 172.21.15.153:0/1411247384 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237000ddf0 con 0x7f237815cef0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.998+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237815dd10 con 0x7f237815cef0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:54.999+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23780cab90 con 0x7f237815cef0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.000+0000 7f2366ffd700 1 -- 172.21.15.153:0/1411247384 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2370015420 con 0x7f237815cef0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.000+0000 7f2366ffd700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23580682f0 0x7f235806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.000+0000 7f2366ffd700 1 -- 172.21.15.153:0/1411247384 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2370011070 con 0x7f237815cef0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.000+0000 7f237659c700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23580682f0 0x7f235806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:55.007 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.001+0000 7f237659c700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23580682f0 0x7f235806a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f236c0090b0 tx=0x7f236c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:55.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.004+0000 7f2366ffd700 1 -- 172.21.15.153:0/1411247384 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f237004d5e0 con 0x7f237815cef0 2023-11-28T13:03:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.233+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2378066a20 con 0x7f237815cef0 2023-11-28T13:03:55.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.233+0000 7f2366ffd700 1 -- 172.21.15.153:0/1411247384 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f2370052520 con 0x7f237815cef0 2023-11-28T13:03:55.234 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:03:55.234 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:03:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23580682f0 msgr2=0x7f235806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23580682f0 0x7f235806a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f236c0090b0 tx=0x7f236c01c040 comp rx=0 tx=0).stop 2023-11-28T13:03:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 msgr2=0x7f237806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f237806aba0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f237000b9e0 tx=0x7f237000bcf0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 shutdown_connections 2023-11-28T13:03:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f23580682f0 0x7f235806a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f237815cef0 0x7f237806aba0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f237806ba90 0x7f2378156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 --2- 172.21.15.153:0/1411247384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23780cbdf0 0x7f23781f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 >> 172.21.15.153:0/1411247384 conn(0x7f23780d0000 msgr2=0x7f237806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 shutdown_connections 2023-11-28T13:03:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:03:55.235+0000 7f237c877700 1 -- 172.21.15.153:0/1411247384 wait complete. 2023-11-28T13:03:55.248 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:00.249 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:00.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.382+0000 7fc70e335700 1 Processor -- start 2023-11-28T13:04:00.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.382+0000 7fc70e335700 1 -- start start 2023-11-28T13:04:00.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70e335700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc7080d0a00 0x7fc70815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:00.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70e335700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc7080b2930 0x7fc7080b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:00.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70e335700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70806b420 con 0x7fc7080ce040 2023-11-28T13:04:00.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70e335700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7080b32f0 con 0x7fc7080b2930 2023-11-28T13:04:00.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70e335700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70806ff60 con 0x7fc7080d0a00 2023-11-28T13:04:00.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70c8d2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc7080b2930 0x7fc7080b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc707fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc7080ce040 0x7fc70806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60858/0 (socket says 172.21.15.153:60858) 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc707fff700 1 -- 172.21.15.153:0/1139589160 learned_addr learned my addr 172.21.15.153:0/1139589160 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc7077fe700 1 --2- 172.21.15.153:0/1139589160 >> v2:172.21.15.177:3300/0 conn(0x7fc7080d0a00 0x7fc70815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70c8d2700 1 -- 172.21.15.153:0/1139589160 >> v2:172.21.15.177:3300/0 conn(0x7fc7080d0a00 msgr2=0x7fc70815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70c8d2700 1 --2- 172.21.15.153:0/1139589160 >> v2:172.21.15.177:3300/0 conn(0x7fc7080d0a00 0x7fc70815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70c8d2700 1 -- 172.21.15.153:0/1139589160 >> v1:172.21.15.153:6789/0 conn(0x7fc7080ce040 legacy=0x7fc70806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc706ffd700 1 -- 172.21.15.153:0/1139589160 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372377607 0 0) 0x7fc70806b420 con 0x7fc7080ce040 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70c8d2700 1 -- 172.21.15.153:0/1139589160 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70806b700 con 0x7fc7080b2930 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.383+0000 7fc70c8d2700 1 --2- 172.21.15.153:0/1139589160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080b2930 0x7fc7080b2db0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc70806ed50 tx=0x7fc70000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780980918f8b61ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc706ffd700 1 -- 172.21.15.153:0/1139589160 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70000e070 con 0x7fc7080b2930 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc706ffd700 1 -- 172.21.15.153:0/1139589160 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70000bbd0 con 0x7fc7080b2930 2023-11-28T13:04:00.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc706ffd700 1 -- 172.21.15.153:0/1139589160 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700012430 con 0x7fc7080b2930 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 -- 172.21.15.153:0/1139589160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080b2930 msgr2=0x7fc7080b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 --2- 172.21.15.153:0/1139589160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080b2930 0x7fc7080b2db0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc70806ed50 tx=0x7fc70000b330 comp rx=0 tx=0).stop 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 -- 172.21.15.153:0/1139589160 shutdown_connections 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 --2- 172.21.15.153:0/1139589160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080b2930 0x7fc7080b2db0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 --2- 172.21.15.153:0/1139589160 >> v2:172.21.15.177:3300/0 conn(0x7fc7080d0a00 0x7fc70815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 -- 172.21.15.153:0/1139589160 >> 172.21.15.153:0/1139589160 conn(0x7fc70806aed0 msgr2=0x7fc70806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 -- 172.21.15.153:0/1139589160 shutdown_connections 2023-11-28T13:04:00.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.384+0000 7fc70e335700 1 -- 172.21.15.153:0/1139589160 wait complete. 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.385+0000 7fc70e335700 1 Processor -- start 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.386+0000 7fc70e335700 1 -- start start 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.386+0000 7fc70e335700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7080b2930 0x7fc708151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.386+0000 7fc70e335700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080ce040 0x7fc7080b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.386+0000 7fc70e335700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7080d0a00 0x7fc7080b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.386+0000 7fc70e335700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081f1e90 con 0x7fc7080ce040 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.386+0000 7fc70e335700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7080c5960 con 0x7fc7080d0a00 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.386+0000 7fc70e335700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7080c5f00 con 0x7fc7080b2930 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080ce040 0x7fc7080b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080ce040 0x7fc7080b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42702/0 (socket says 172.21.15.153:42702) 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 -- 172.21.15.153:0/1789482583 learned_addr learned my addr 172.21.15.153:0/1789482583 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:00.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc707fff700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7080b2930 0x7fc708151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc70c8d2700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7080d0a00 0x7fc7080b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 -- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7080b2930 msgr2=0x7fc708151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7080b2930 0x7fc708151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 -- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7080d0a00 msgr2=0x7fc7080b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7080d0a00 0x7fc7080b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 -- 172.21.15.153:0/1789482583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc700009a70 con 0x7fc7080ce040 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc70c8d2700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7080d0a00 0x7fc7080b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc7077fe700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080ce040 0x7fc7080b62a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800d800 tx=0x7fc6f800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc704ff9700 1 -- 172.21.15.153:0/1789482583 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80125f0 con 0x7fc7080ce040 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc704ff9700 1 -- 172.21.15.153:0/1789482583 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8012750 con 0x7fc7080ce040 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70806bcc0 con 0x7fc7080ce040 2023-11-28T13:04:00.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc704ff9700 1 -- 172.21.15.153:0/1789482583 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8012b00 con 0x7fc7080ce040 2023-11-28T13:04:00.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.387+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70806b9e0 con 0x7fc7080ce040 2023-11-28T13:04:00.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.388+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70806bfa0 con 0x7fc7080ce040 2023-11-28T13:04:00.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.389+0000 7fc704ff9700 1 -- 172.21.15.153:0/1789482583 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6f8012c60 con 0x7fc7080ce040 2023-11-28T13:04:00.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.389+0000 7fc704ff9700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6f40682f0 0x7fc6f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:00.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.389+0000 7fc704ff9700 1 -- 172.21.15.153:0/1789482583 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc6f80861e0 con 0x7fc7080ce040 2023-11-28T13:04:00.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.389+0000 7fc707fff700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6f40682f0 0x7fc6f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:00.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.389+0000 7fc707fff700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6f40682f0 0x7fc6f406a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc001750 tx=0x7fc6fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:00.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.393+0000 7fc704ff9700 1 -- 172.21.15.153:0/1789482583 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6f80529b0 con 0x7fc7080ce040 2023-11-28T13:04:00.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.622+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc708068690 con 0x7fc7080ce040 2023-11-28T13:04:00.624 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.623+0000 7fc704ff9700 1 -- 172.21.15.153:0/1789482583 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+312 (secure 0 0 0) 0x7fc6f8055fd0 con 0x7fc7080ce040 2023-11-28T13:04:00.624 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:00.624 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false},"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg inactive, 1 pg peering","count":2},"muted":false}},"mutes":[]} 2023-11-28T13:04:00.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6f40682f0 msgr2=0x7fc6f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:00.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6f40682f0 0x7fc6f406a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc001750 tx=0x7fc6fc009040 comp rx=0 tx=0).stop 2023-11-28T13:04:00.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080ce040 msgr2=0x7fc7080b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:00.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080ce040 0x7fc7080b62a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800d800 tx=0x7fc6f800dbc0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 shutdown_connections 2023-11-28T13:04:00.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7080d0a00 0x7fc7080b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7080b2930 0x7fc708151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6f40682f0 0x7fc6f406a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 --2- 172.21.15.153:0/1789482583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7080ce040 0x7fc7080b62a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:00.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 >> 172.21.15.153:0/1789482583 conn(0x7fc70806aed0 msgr2=0x7fc70815ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:00.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 shutdown_connections 2023-11-28T13:04:00.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:00.625+0000 7fc70e335700 1 -- 172.21.15.153:0/1789482583 wait complete. 2023-11-28T13:04:00.637 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:05.639 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:05.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.770+0000 7f3f105ff700 1 Processor -- start 2023-11-28T13:04:05.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.770+0000 7f3f105ff700 1 -- start start 2023-11-28T13:04:05.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f105ff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f0806dee0 0x7f3f0806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f105ff700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0815d060 con 0x7f3f08160540 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f105ff700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0815d340 con 0x7f3f08071040 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f105ff700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0815d620 con 0x7f3f0806dee0 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0db9a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3f0806dee0 0x7f3f0806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0eb9c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f3f08160540 0x7f3f08160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60876/0 (socket says 172.21.15.153:60876) 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0eb9c700 1 -- 172.21.15.153:0/1266852495 learned_addr learned my addr 172.21.15.153:0/1266852495 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0db9a700 1 -- 172.21.15.153:0/1266852495 >> v1:172.21.15.177:6789/0 conn(0x7f3f08071040 legacy=0x7f3f08071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0db9a700 1 -- 172.21.15.153:0/1266852495 >> v1:172.21.15.153:6789/0 conn(0x7f3f08160540 legacy=0x7f3f08160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0db9a700 1 -- 172.21.15.153:0/1266852495 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0815d900 con 0x7f3f0806dee0 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0d399700 1 -- 172.21.15.153:0/1266852495 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224686519 0 0) 0x7f3f0815d060 con 0x7f3f08160540 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0db9a700 1 --2- 172.21.15.153:0/1266852495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f0806dee0 0x7f3f0806e330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009840 tx=0x7f3f04009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0a5f787ddaa2cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0d399700 1 -- 172.21.15.153:0/1266852495 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04011830 con 0x7f3f0806dee0 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0d399700 1 -- 172.21.15.153:0/1266852495 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04011990 con 0x7f3f0806dee0 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.771+0000 7f3f0d399700 1 -- 172.21.15.153:0/1266852495 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04011d20 con 0x7f3f0806dee0 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.772+0000 7f3f105ff700 1 -- 172.21.15.153:0/1266852495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f0806dee0 msgr2=0x7f3f0806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.772+0000 7f3f105ff700 1 --2- 172.21.15.153:0/1266852495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f0806dee0 0x7f3f0806e330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009840 tx=0x7f3f04009b50 comp rx=0 tx=0).stop 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.772+0000 7f3f105ff700 1 -- 172.21.15.153:0/1266852495 shutdown_connections 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.772+0000 7f3f105ff700 1 --2- 172.21.15.153:0/1266852495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f0806dee0 0x7f3f0806e330 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.772+0000 7f3f105ff700 1 -- 172.21.15.153:0/1266852495 >> 172.21.15.153:0/1266852495 conn(0x7f3f080d0000 msgr2=0x7f3f080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.772+0000 7f3f105ff700 1 -- 172.21.15.153:0/1266852495 shutdown_connections 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.772+0000 7f3f105ff700 1 -- 172.21.15.153:0/1266852495 wait complete. 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.773+0000 7f3f105ff700 1 Processor -- start 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.773+0000 7f3f105ff700 1 -- start start 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f105ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f0806dee0 0x7f3f08153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f105ff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f08071040 0x7f3f080b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f105ff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f08160540 0x7f3f0815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f105ff700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08159120 con 0x7f3f08071040 2023-11-28T13:04:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f105ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080b0920 con 0x7f3f0806dee0 2023-11-28T13:04:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f105ff700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080b0bd0 con 0x7f3f08160540 2023-11-28T13:04:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f08071040 0x7f3f080b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f08071040 0x7f3f080b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42716/0 (socket says 172.21.15.153:42716) 2023-11-28T13:04:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 -- 172.21.15.153:0/315349557 learned_addr learned my addr 172.21.15.153:0/315349557 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0e39b700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f0806dee0 0x7f3f08153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0eb9c700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f08160540 0x7f3f0815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 -- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f08160540 msgr2=0x7f3f0815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f08160540 0x7f3f0815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 -- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f0806dee0 msgr2=0x7f3f08153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f0806dee0 0x7f3f08153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0db9a700 1 -- 172.21.15.153:0/315349557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040094a0 con 0x7f3f08071040 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.774+0000 7f3f0eb9c700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f08160540 0x7f3f0815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.775+0000 7f3f0db9a700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f08071040 0x7f3f080b03e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3f04014040 tx=0x7f3f04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.775+0000 7f3efeffd700 1 -- 172.21.15.153:0/315349557 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0401a7e0 con 0x7f3f08071040 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.775+0000 7f3efeffd700 1 -- 172.21.15.153:0/315349557 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0401a940 con 0x7f3f08071040 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.775+0000 7f3efeffd700 1 -- 172.21.15.153:0/315349557 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0401acb0 con 0x7f3f08071040 2023-11-28T13:04:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.775+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0815ccd0 con 0x7f3f08071040 2023-11-28T13:04:05.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.775+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0806e870 con 0x7f3f08071040 2023-11-28T13:04:05.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.776+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0806a270 con 0x7f3f08071040 2023-11-28T13:04:05.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.776+0000 7f3efeffd700 1 -- 172.21.15.153:0/315349557 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f0402b030 con 0x7f3f08071040 2023-11-28T13:04:05.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.776+0000 7f3efeffd700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eec0682f0 0x7f3eec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.777+0000 7f3efeffd700 1 -- 172.21.15.153:0/315349557 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3f04086ec0 con 0x7f3f08071040 2023-11-28T13:04:05.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.777+0000 7f3f0e39b700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eec0682f0 0x7f3eec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.777+0000 7f3f0e39b700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eec0682f0 0x7f3eec06a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8000f60 tx=0x7f3ef8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:05.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:05.781+0000 7f3efeffd700 1 -- 172.21.15.153:0/315349557 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f04053740 con 0x7f3f08071040 2023-11-28T13:04:06.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.004+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3f080720a0 con 0x7f3f08071040 2023-11-28T13:04:06.006 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.004+0000 7f3efeffd700 1 -- 172.21.15.153:0/315349557 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+312 (secure 0 0 0) 0x7f3f04029150 con 0x7f3f08071040 2023-11-28T13:04:06.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:06.006 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false},"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg inactive, 1 pg peering","count":2},"muted":false}},"mutes":[]} 2023-11-28T13:04:06.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eec0682f0 msgr2=0x7f3eec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:06.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eec0682f0 0x7f3eec06a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8000f60 tx=0x7f3ef8009040 comp rx=0 tx=0).stop 2023-11-28T13:04:06.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f08071040 msgr2=0x7f3f080b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f08071040 0x7f3f080b03e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3f04014040 tx=0x7f3f04004750 comp rx=0 tx=0).stop 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 shutdown_connections 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f08160540 0x7f3f0815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f0806dee0 0x7f3f08153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eec0682f0 0x7f3eec06a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 --2- 172.21.15.153:0/315349557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f08071040 0x7f3f080b03e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 >> 172.21.15.153:0/315349557 conn(0x7f3f080d0000 msgr2=0x7f3f0815f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 shutdown_connections 2023-11-28T13:04:06.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:06.006+0000 7f3f105ff700 1 -- 172.21.15.153:0/315349557 wait complete. 2023-11-28T13:04:06.020 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:11.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:11.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.151+0000 7f3eca12d700 1 Processor -- start 2023-11-28T13:04:11.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.151+0000 7f3eca12d700 1 -- start start 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3eca12d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3ec4071040 0x7f3ec4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3eca12d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3ec4160540 0x7f3ec4160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3eca12d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec4071950 con 0x7f3ec406dee0 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3eca12d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec415d060 con 0x7f3ec4160540 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3eca12d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec415d340 con 0x7f3ec4071040 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3ec4160540 0x7f3ec4160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3ec4160540 0x7f3ec4160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42724/0 (socket says 172.21.15.153:42724) 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 -- 172.21.15.153:0/1716402233 learned_addr learned my addr 172.21.15.153:0/1716402233 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec37fe700 1 --2- 172.21.15.153:0/1716402233 >> v2:172.21.15.177:3300/0 conn(0x7f3ec4071040 0x7f3ec4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:11.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 -- 172.21.15.153:0/1716402233 >> v2:172.21.15.177:3300/0 conn(0x7f3ec4071040 msgr2=0x7f3ec4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 --2- 172.21.15.153:0/1716402233 >> v2:172.21.15.177:3300/0 conn(0x7f3ec4071040 0x7f3ec4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 -- 172.21.15.153:0/1716402233 >> v1:172.21.15.177:6789/0 conn(0x7f3ec406dee0 legacy=0x7f3ec406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 -- 172.21.15.153:0/1716402233 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec415d620 con 0x7f3ec4160540 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec37fe700 1 --2- 172.21.15.153:0/1716402233 >> v2:172.21.15.177:3300/0 conn(0x7f3ec4071040 0x7f3ec4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec3fff700 1 --2- 172.21.15.153:0/1716402233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec4160980 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40d0f50 tx=0x7f3ebc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaef7fb7f798cfb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec27fc700 1 -- 172.21.15.153:0/1716402233 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00e070 con 0x7f3ec4160540 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec27fc700 1 -- 172.21.15.153:0/1716402233 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc003a20 con 0x7f3ec4160540 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.152+0000 7f3ec27fc700 1 -- 172.21.15.153:0/1716402233 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc003dd0 con 0x7f3ec4160540 2023-11-28T13:04:11.157 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 -- 172.21.15.153:0/1716402233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 msgr2=0x7f3ec4160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 --2- 172.21.15.153:0/1716402233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec4160980 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40d0f50 tx=0x7f3ebc00b330 comp rx=0 tx=0).stop 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 -- 172.21.15.153:0/1716402233 shutdown_connections 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 --2- 172.21.15.153:0/1716402233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec4160980 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 --2- 172.21.15.153:0/1716402233 >> v2:172.21.15.177:3300/0 conn(0x7f3ec4071040 0x7f3ec4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 -- 172.21.15.153:0/1716402233 >> 172.21.15.153:0/1716402233 conn(0x7f3ec40d0000 msgr2=0x7f3ec40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 -- 172.21.15.153:0/1716402233 shutdown_connections 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.153+0000 7f3eca12d700 1 -- 172.21.15.153:0/1716402233 wait complete. 2023-11-28T13:04:11.158 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.154+0000 7f3eca12d700 1 Processor -- start 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3eca12d700 1 -- start start 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3eca12d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ec406dee0 0x7f3ec40b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3eca12d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ec4071040 0x7f3ec40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3eca12d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3eca12d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41f6610 con 0x7f3ec4160540 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3eca12d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec40b3370 con 0x7f3ec406dee0 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3eca12d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec41f5990 con 0x7f3ec4071040 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3ec3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3ec3fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec415b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42730/0 (socket says 172.21.15.153:42730) 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.155+0000 7f3ec3fff700 1 -- 172.21.15.153:0/3658807441 learned_addr learned my addr 172.21.15.153:0/3658807441 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:11.161 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec37fe700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ec406dee0 0x7f3ec40b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec2ffd700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ec4071040 0x7f3ec40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec3fff700 1 -- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ec4071040 msgr2=0x7f3ec40b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec3fff700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ec4071040 0x7f3ec40b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec3fff700 1 -- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ec406dee0 msgr2=0x7f3ec40b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec3fff700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ec406dee0 0x7f3ec40b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec3fff700 1 -- 172.21.15.153:0/3658807441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc009b30 con 0x7f3ec4160540 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3ec3fff700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec415b300 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3ec406c7b0 tx=0x7f3ebc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3eabfff700 1 -- 172.21.15.153:0/3658807441 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004440 con 0x7f3ec4160540 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3eabfff700 1 -- 172.21.15.153:0/3658807441 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc0045a0 con 0x7f3ec4160540 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec415ccd0 con 0x7f3ec4160540 2023-11-28T13:04:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3eabfff700 1 -- 172.21.15.153:0/3658807441 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004930 con 0x7f3ec4160540 2023-11-28T13:04:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.156+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec415d900 con 0x7f3ec4160540 2023-11-28T13:04:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.157+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec406a270 con 0x7f3ec4160540 2023-11-28T13:04:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.157+0000 7f3eabfff700 1 -- 172.21.15.153:0/3658807441 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ebc022030 con 0x7f3ec4160540 2023-11-28T13:04:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.158+0000 7f3eabfff700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eb00682f0 0x7f3eb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.158+0000 7f3ec37fe700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eb00682f0 0x7f3eb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.158+0000 7f3eabfff700 1 -- 172.21.15.153:0/3658807441 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3ebc087110 con 0x7f3ec4160540 2023-11-28T13:04:11.163 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.158+0000 7f3ec37fe700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eb00682f0 0x7f3eb006a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4006fd0 tx=0x7f3eb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:11.164 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.162+0000 7f3eabfff700 1 -- 172.21.15.153:0/3658807441 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ebc0538e0 con 0x7f3ec4160540 2023-11-28T13:04:11.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.386+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3ec41f6750 con 0x7f3ec4160540 2023-11-28T13:04:11.388 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.386+0000 7f3eabfff700 1 -- 172.21.15.153:0/3658807441 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3ec41f6750 con 0x7f3ec4160540 2023-11-28T13:04:11.388 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:11.388 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:11.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eb00682f0 msgr2=0x7f3eb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:11.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eb00682f0 0x7f3eb006a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4006fd0 tx=0x7f3eb4008040 comp rx=0 tx=0).stop 2023-11-28T13:04:11.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 msgr2=0x7f3ec415b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:11.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec415b300 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3ec406c7b0 tx=0x7f3ebc004ad0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 shutdown_connections 2023-11-28T13:04:11.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3eb00682f0 0x7f3eb006a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ec4160540 0x7f3ec415b300 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ec406dee0 0x7f3ec40b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 --2- 172.21.15.153:0/3658807441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ec4071040 0x7f3ec40b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:11.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 >> 172.21.15.153:0/3658807441 conn(0x7f3ec40d0000 msgr2=0x7f3ec415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:11.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 shutdown_connections 2023-11-28T13:04:11.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:11.388+0000 7f3eca12d700 1 -- 172.21.15.153:0/3658807441 wait complete. 2023-11-28T13:04:11.402 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:16.403 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:16.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.547+0000 7f21002a8700 1 Processor -- start 2023-11-28T13:04:16.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.547+0000 7f21002a8700 1 -- start start 2023-11-28T13:04:16.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f21002a8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f20f815cef0 0x7f20f80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:16.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f21002a8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f20f806ba90 0x7f20f80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:16.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f21002a8700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f80cc5f0 con 0x7f20f80cbdf0 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f21002a8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f815d470 con 0x7f20f815cef0 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f21002a8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f815d750 con 0x7f20f806ba90 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe044700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f20f815cef0 0x7f20f80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fd843700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f20f80cbdf0 0x7f20f806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60900/0 (socket says 172.21.15.153:60900) 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fd843700 1 -- 172.21.15.153:0/882911112 learned_addr learned my addr 172.21.15.153:0/882911112 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe044700 1 -- 172.21.15.153:0/882911112 >> v2:172.21.15.177:3300/0 conn(0x7f20f806ba90 msgr2=0x7f20f80ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe845700 1 --2- 172.21.15.153:0/882911112 >> v2:172.21.15.177:3300/0 conn(0x7f20f806ba90 0x7f20f80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe044700 1 --2- 172.21.15.153:0/882911112 >> v2:172.21.15.177:3300/0 conn(0x7f20f806ba90 0x7f20f80ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fd042700 1 -- 172.21.15.153:0/882911112 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505002987 0 0) 0x7f20f80cc5f0 con 0x7f20f80cbdf0 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe044700 1 -- 172.21.15.153:0/882911112 >> v1:172.21.15.153:6789/0 conn(0x7f20f80cbdf0 legacy=0x7f20f806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe044700 1 -- 172.21.15.153:0/882911112 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f815da30 con 0x7f20f815cef0 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe845700 1 --2- 172.21.15.153:0/882911112 >> v2:172.21.15.177:3300/0 conn(0x7f20f806ba90 0x7f20f80ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fe044700 1 --2- 172.21.15.153:0/882911112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f815cef0 0x7f20f80af400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009840 tx=0x7f20f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a5b48c0d4b32eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.548+0000 7f20fd042700 1 -- 172.21.15.153:0/882911112 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4011830 con 0x7f20f815cef0 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f20fd042700 1 -- 172.21.15.153:0/882911112 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f4011990 con 0x7f20f815cef0 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f20fd042700 1 -- 172.21.15.153:0/882911112 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4011d20 con 0x7f20f815cef0 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 -- 172.21.15.153:0/882911112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f815cef0 msgr2=0x7f20f80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 --2- 172.21.15.153:0/882911112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f815cef0 0x7f20f80af400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009840 tx=0x7f20f4009b50 comp rx=0 tx=0).stop 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 -- 172.21.15.153:0/882911112 shutdown_connections 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 --2- 172.21.15.153:0/882911112 >> v2:172.21.15.177:3300/0 conn(0x7f20f806ba90 0x7f20f80ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 --2- 172.21.15.153:0/882911112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f815cef0 0x7f20f80af400 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 -- 172.21.15.153:0/882911112 >> 172.21.15.153:0/882911112 conn(0x7f20f80d0000 msgr2=0x7f20f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:16.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 -- 172.21.15.153:0/882911112 shutdown_connections 2023-11-28T13:04:16.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.549+0000 7f21002a8700 1 -- 172.21.15.153:0/882911112 wait complete. 2023-11-28T13:04:16.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.550+0000 7f21002a8700 1 Processor -- start 2023-11-28T13:04:16.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.550+0000 7f21002a8700 1 -- start start 2023-11-28T13:04:16.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f21002a8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f806ba90 0x7f20f80b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:16.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f21002a8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20f80cbdf0 0x7f20f8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:16.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f21002a8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20f815cef0 0x7f20f80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:16.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f21002a8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f814ca60 con 0x7f20f806ba90 2023-11-28T13:04:16.556 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f21002a8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b2e30 con 0x7f20f815cef0 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f21002a8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b3110 con 0x7f20f80cbdf0 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f806ba90 0x7f20f80b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f806ba90 0x7f20f80b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42740/0 (socket says 172.21.15.153:42740) 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 -- 172.21.15.153:0/1543403234 learned_addr learned my addr 172.21.15.153:0/1543403234 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 -- 172.21.15.153:0/1543403234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20f80cbdf0 msgr2=0x7f20f8159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20f80cbdf0 0x7f20f8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 -- 172.21.15.153:0/1543403234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20f815cef0 msgr2=0x7f20f80b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20f815cef0 0x7f20f80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.551+0000 7f20fe044700 1 -- 172.21.15.153:0/1543403234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40094a0 con 0x7f20f806ba90 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.552+0000 7f20fe044700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f806ba90 0x7f20f80b5700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009b90 tx=0x7f20f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:16.557 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.552+0000 7f20eeffd700 1 -- 172.21.15.153:0/1543403234 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f401abc0 con 0x7f20f806ba90 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.552+0000 7f20eeffd700 1 -- 172.21.15.153:0/1543403234 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f401ad20 con 0x7f20f806ba90 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.552+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f815dff0 con 0x7f20f806ba90 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.552+0000 7f20eeffd700 1 -- 172.21.15.153:0/1543403234 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f40245f0 con 0x7f20f806ba90 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.552+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f815dd10 con 0x7f20f806ba90 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.553+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f80cab90 con 0x7f20f806ba90 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.553+0000 7f20eeffd700 1 -- 172.21.15.153:0/1543403234 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20f400f050 con 0x7f20f806ba90 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.553+0000 7f20eeffd700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f20dc0682f0 0x7f20dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:16.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.554+0000 7f20eeffd700 1 -- 172.21.15.153:0/1543403234 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f20f4087270 con 0x7f20f806ba90 2023-11-28T13:04:16.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.554+0000 7f20fd843700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f20dc0682f0 0x7f20dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:16.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.554+0000 7f20fd843700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f20dc0682f0 0x7f20dc06a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20e80011e0 tx=0x7f20e80079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:16.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.558+0000 7f20eeffd700 1 -- 172.21.15.153:0/1543403234 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20f4053a40 con 0x7f20f806ba90 2023-11-28T13:04:16.686 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:16.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:16.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.786+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f20f80ce4f0 con 0x7f20f806ba90 2023-11-28T13:04:16.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.787+0000 7f20eeffd700 1 -- 172.21.15.153:0/1543403234 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f20f4057060 con 0x7f20f806ba90 2023-11-28T13:04:16.789 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:16.789 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f20dc0682f0 msgr2=0x7f20dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f20dc0682f0 0x7f20dc06a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20e80011e0 tx=0x7f20e80079d0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f806ba90 msgr2=0x7f20f80b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f806ba90 0x7f20f80b5700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009b90 tx=0x7f20f4004750 comp rx=0 tx=0).stop 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 shutdown_connections 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f20dc0682f0 0x7f20dc06a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20f806ba90 0x7f20f80b5700 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20f815cef0 0x7f20f80b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 --2- 172.21.15.153:0/1543403234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20f80cbdf0 0x7f20f8159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 >> 172.21.15.153:0/1543403234 conn(0x7f20f80d0000 msgr2=0x7f20f8073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:16.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 shutdown_connections 2023-11-28T13:04:16.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:16.789+0000 7f21002a8700 1 -- 172.21.15.153:0/1543403234 wait complete. 2023-11-28T13:04:16.802 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:21.803 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:21.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.933+0000 7f3c4fbb7700 1 Processor -- start 2023-11-28T13:04:21.935 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.933+0000 7f3c4fbb7700 1 -- start start 2023-11-28T13:04:21.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4fbb7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3c48154e70 0x7f3c480cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4fbb7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3c480cf360 0x7f3c48071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4fbb7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c48071980 con 0x7f3c48071580 2023-11-28T13:04:21.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4fbb7700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c48155240 con 0x7f3c480cf360 2023-11-28T13:04:21.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4fbb7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4815b100 con 0x7f3c48154e70 2023-11-28T13:04:21.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d152700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3c480cf360 0x7f3c48071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d152700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3c480cf360 0x7f3c48071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42746/0 (socket says 172.21.15.153:42746) 2023-11-28T13:04:21.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d152700 1 -- 172.21.15.153:0/2980627326 learned_addr learned my addr 172.21.15.153:0/2980627326 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:21.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d953700 1 --2- 172.21.15.153:0/2980627326 >> v2:172.21.15.177:3300/0 conn(0x7f3c48154e70 0x7f3c480cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d152700 1 -- 172.21.15.153:0/2980627326 >> v2:172.21.15.177:3300/0 conn(0x7f3c48154e70 msgr2=0x7f3c480cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:21.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d152700 1 --2- 172.21.15.153:0/2980627326 >> v2:172.21.15.177:3300/0 conn(0x7f3c48154e70 0x7f3c480cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d152700 1 -- 172.21.15.153:0/2980627326 >> v1:172.21.15.177:6789/0 conn(0x7f3c48071580 legacy=0x7f3c4815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d152700 1 -- 172.21.15.153:0/2980627326 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4815b3e0 con 0x7f3c480cf360 2023-11-28T13:04:21.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.934+0000 7f3c4d953700 1 --2- 172.21.15.153:0/2980627326 >> v2:172.21.15.177:3300/0 conn(0x7f3c48154e70 0x7f3c480cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:21.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4d152700 1 --2- 172.21.15.153:0/2980627326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c480cf360 0x7f3c48071040 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3c48073b80 tx=0x7f3c38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876d098bf169bced server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:21.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4c951700 1 -- 172.21.15.153:0/2980627326 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800b870 con 0x7f3c480cf360 2023-11-28T13:04:21.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4c951700 1 -- 172.21.15.153:0/2980627326 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800b9d0 con 0x7f3c480cf360 2023-11-28T13:04:21.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4c951700 1 -- 172.21.15.153:0/2980627326 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800bd60 con 0x7f3c480cf360 2023-11-28T13:04:21.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/2980627326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c480cf360 msgr2=0x7f3c48071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/2980627326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c480cf360 0x7f3c48071040 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3c48073b80 tx=0x7f3c38009bd0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/2980627326 shutdown_connections 2023-11-28T13:04:21.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/2980627326 >> v2:172.21.15.177:3300/0 conn(0x7f3c48154e70 0x7f3c480cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/2980627326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c480cf360 0x7f3c48071040 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/2980627326 >> 172.21.15.153:0/2980627326 conn(0x7f3c480d0000 msgr2=0x7f3c480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:21.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/2980627326 shutdown_connections 2023-11-28T13:04:21.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.935+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/2980627326 wait complete. 2023-11-28T13:04:21.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.936+0000 7f3c4fbb7700 1 Processor -- start 2023-11-28T13:04:21.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4fbb7700 1 -- start start 2023-11-28T13:04:21.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4fbb7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c48071580 0x7f3c480c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4fbb7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c480cf360 0x7f3c4815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4fbb7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c48154e70 0x7f3c4815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4fbb7700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480ca310 con 0x7f3c48154e70 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4fbb7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480c9480 con 0x7f3c480cf360 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4fbb7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480c9700 con 0x7f3c48071580 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4e154700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c48154e70 0x7f3c4815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4e154700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c48154e70 0x7f3c4815a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42754/0 (socket says 172.21.15.153:42754) 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.937+0000 7f3c4e154700 1 -- 172.21.15.153:0/1490121295 learned_addr learned my addr 172.21.15.153:0/1490121295 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4d152700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c480cf360 0x7f3c4815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4e154700 1 -- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c48071580 msgr2=0x7f3c480c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4d953700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c48071580 0x7f3c480c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4e154700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c48071580 0x7f3c480c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4e154700 1 -- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c480cf360 msgr2=0x7f3c4815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4e154700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c480cf360 0x7f3c4815a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4e154700 1 -- 172.21.15.153:0/1490121295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38009550 con 0x7f3c48154e70 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4d152700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c480cf360 0x7f3c4815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4e154700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c48154e70 0x7f3c4815a610 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000cc50 tx=0x7f3c4000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c3e7fc700 1 -- 172.21.15.153:0/1490121295 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000ad80 con 0x7f3c48154e70 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c3e7fc700 1 -- 172.21.15.153:0/1490121295 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4000ce50 con 0x7f3c48154e70 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4815b9a0 con 0x7f3c48154e70 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c3e7fc700 1 -- 172.21.15.153:0/1490121295 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40003c40 con 0x7f3c48154e70 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.938+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4815b6c0 con 0x7f3c48154e70 2023-11-28T13:04:21.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.939+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c480af520 con 0x7f3c48154e70 2023-11-28T13:04:21.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.939+0000 7f3c3e7fc700 1 -- 172.21.15.153:0/1490121295 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3c40010050 con 0x7f3c48154e70 2023-11-28T13:04:21.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.940+0000 7f3c3e7fc700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3c2c0682f0 0x7f3c2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.940+0000 7f3c3e7fc700 1 -- 172.21.15.153:0/1490121295 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3c40085cd0 con 0x7f3c48154e70 2023-11-28T13:04:21.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.940+0000 7f3c4d953700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3c2c0682f0 0x7f3c2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.940+0000 7f3c4d953700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3c2c0682f0 0x7f3c2c06a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c48153cc0 tx=0x7f3c44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:21.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:21.944+0000 7f3c3e7fc700 1 -- 172.21.15.153:0/1490121295 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3c40052440 con 0x7f3c48154e70 2023-11-28T13:04:22.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.171+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3c48072260 con 0x7f3c48154e70 2023-11-28T13:04:22.174 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.172+0000 7f3c3e7fc700 1 -- 172.21.15.153:0/1490121295 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3c40055a60 con 0x7f3c48154e70 2023-11-28T13:04:22.174 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:22.174 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:22.175 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.173+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3c2c0682f0 msgr2=0x7f3c2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.173+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3c2c0682f0 0x7f3c2c06a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c48153cc0 tx=0x7f3c44008040 comp rx=0 tx=0).stop 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.173+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c48154e70 msgr2=0x7f3c4815a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.173+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c48154e70 0x7f3c4815a610 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000cc50 tx=0x7f3c4000a340 comp rx=0 tx=0).stop 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.173+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 shutdown_connections 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.173+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3c2c0682f0 0x7f3c2c06a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.173+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c48154e70 0x7f3c4815a610 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.174+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c48071580 0x7f3c480c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.174+0000 7f3c4fbb7700 1 --2- 172.21.15.153:0/1490121295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c480cf360 0x7f3c4815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.174+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 >> 172.21.15.153:0/1490121295 conn(0x7f3c480d0000 msgr2=0x7f3c4806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.174+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 shutdown_connections 2023-11-28T13:04:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:22.174+0000 7f3c4fbb7700 1 -- 172.21.15.153:0/1490121295 wait complete. 2023-11-28T13:04:22.187 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:27.188 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:27.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.318+0000 7f22000a3700 1 Processor -- start 2023-11-28T13:04:27.320 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.318+0000 7f22000a3700 1 -- start start 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f22000a3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80ce040 0x7f21f815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f22000a3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f21f80b2930 0x7f21f80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f22000a3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f806b420 con 0x7f21f80d0a00 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f22000a3700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80b32f0 con 0x7f21f80b2930 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f22000a3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f806ff60 con 0x7f21f80ce040 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f21fe640700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f21f80b2930 0x7f21f80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f21fd63e700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f21f80d0a00 0x7f21f806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60922/0 (socket says 172.21.15.153:60922) 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.319+0000 7f21fd63e700 1 -- 172.21.15.153:0/329839086 learned_addr learned my addr 172.21.15.153:0/329839086 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fde3f700 1 --2- 172.21.15.153:0/329839086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80ce040 0x7f21f815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:27.323 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fe640700 1 -- 172.21.15.153:0/329839086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80ce040 msgr2=0x7f21f815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fe640700 1 --2- 172.21.15.153:0/329839086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80ce040 0x7f21f815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fe640700 1 -- 172.21.15.153:0/329839086 >> v1:172.21.15.153:6789/0 conn(0x7f21f80d0a00 legacy=0x7f21f806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fe640700 1 -- 172.21.15.153:0/329839086 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f806b700 con 0x7f21f80b2930 2023-11-28T13:04:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fde3f700 1 --2- 172.21.15.153:0/329839086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80ce040 0x7f21f815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fe640700 1 --2- 172.21.15.153:0/329839086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80b2930 0x7f21f80b2db0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009d90 tx=0x7f21f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629e161d6704e8c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fce3d700 1 -- 172.21.15.153:0/329839086 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f000e070 con 0x7f21f80b2930 2023-11-28T13:04:27.324 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fce3d700 1 -- 172.21.15.153:0/329839086 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f000bbd0 con 0x7f21f80b2930 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f21fce3d700 1 -- 172.21.15.153:0/329839086 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0012430 con 0x7f21f80b2930 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f22000a3700 1 -- 172.21.15.153:0/329839086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80b2930 msgr2=0x7f21f80b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.320+0000 7f22000a3700 1 --2- 172.21.15.153:0/329839086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80b2930 0x7f21f80b2db0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009d90 tx=0x7f21f000b330 comp rx=0 tx=0).stop 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.321+0000 7f22000a3700 1 -- 172.21.15.153:0/329839086 shutdown_connections 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.321+0000 7f22000a3700 1 --2- 172.21.15.153:0/329839086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80ce040 0x7f21f815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.321+0000 7f22000a3700 1 --2- 172.21.15.153:0/329839086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80b2930 0x7f21f80b2db0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.321+0000 7f22000a3700 1 -- 172.21.15.153:0/329839086 >> 172.21.15.153:0/329839086 conn(0x7f21f806aed0 msgr2=0x7f21f806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.321+0000 7f22000a3700 1 -- 172.21.15.153:0/329839086 shutdown_connections 2023-11-28T13:04:27.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.321+0000 7f22000a3700 1 -- 172.21.15.153:0/329839086 wait complete. 2023-11-28T13:04:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.322+0000 7f22000a3700 1 Processor -- start 2023-11-28T13:04:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.322+0000 7f22000a3700 1 -- start start 2023-11-28T13:04:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80b2930 0x7f21f80c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80ce040 0x7f21f81f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80d0a00 0x7f21f80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:27.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f814dbc0 con 0x7f21f80ce040 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80c6240 con 0x7f21f80d0a00 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f81f2cd0 con 0x7f21f80b2930 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80ce040 0x7f21f81f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80ce040 0x7f21f81f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42766/0 (socket says 172.21.15.153:42766) 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 -- 172.21.15.153:0/1191385835 learned_addr learned my addr 172.21.15.153:0/1191385835 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fde3f700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80b2930 0x7f21f80c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fe640700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80d0a00 0x7f21f80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 -- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80b2930 msgr2=0x7f21f80c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80b2930 0x7f21f80c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 -- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80d0a00 msgr2=0x7f21f80cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80d0a00 0x7f21f80cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 -- 172.21.15.153:0/1191385835 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f0009a70 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fe640700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80d0a00 0x7f21f80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21fd63e700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80ce040 0x7f21f81f2790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f21f400c520 tx=0x7f21f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21ee7fc700 1 -- 172.21.15.153:0/1191385835 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4005d70 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21ee7fc700 1 -- 172.21.15.153:0/1191385835 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f40070d0 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f806bcc0 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f21ee7fc700 1 -- 172.21.15.153:0/1191385835 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4007440 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.323+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f806b9e0 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.324+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f806bfa0 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.325+0000 7f21ee7fc700 1 -- 172.21.15.153:0/1191385835 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f21f4019030 con 0x7f21f80ce040 2023-11-28T13:04:27.330 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.325+0000 7f21ee7fc700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f21dc0682f0 0x7f21dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.325+0000 7f21ee7fc700 1 -- 172.21.15.153:0/1191385835 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f21f4084fc0 con 0x7f21f80ce040 2023-11-28T13:04:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.325+0000 7f21fde3f700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f21dc0682f0 0x7f21dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.326+0000 7f21fde3f700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f21dc0682f0 0x7f21dc06a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21f80c7a00 tx=0x7f21e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:27.331 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.329+0000 7f21ee7fc700 1 -- 172.21.15.153:0/1191385835 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f21f404b5e0 con 0x7f21f80ce040 2023-11-28T13:04:27.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.554+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f21f8068690 con 0x7f21f80ce040 2023-11-28T13:04:27.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.556+0000 7f21ee7fc700 1 -- 172.21.15.153:0/1191385835 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f21f40170c0 con 0x7f21f80ce040 2023-11-28T13:04:27.559 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:27.559 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:27.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f21dc0682f0 msgr2=0x7f21dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f21dc0682f0 0x7f21dc06a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21f80c7a00 tx=0x7f21e8008040 comp rx=0 tx=0).stop 2023-11-28T13:04:27.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80ce040 msgr2=0x7f21f81f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80ce040 0x7f21f81f2790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f21f400c520 tx=0x7f21f400c8e0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 shutdown_connections 2023-11-28T13:04:27.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80d0a00 0x7f21f80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f80b2930 0x7f21f80c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f21dc0682f0 0x7f21dc06a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 --2- 172.21.15.153:0/1191385835 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f21f80ce040 0x7f21f81f2790 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 >> 172.21.15.153:0/1191385835 conn(0x7f21f806aed0 msgr2=0x7f21f80d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:27.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 shutdown_connections 2023-11-28T13:04:27.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:27.559+0000 7f22000a3700 1 -- 172.21.15.153:0/1191385835 wait complete. 2023-11-28T13:04:27.573 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:32.573 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:32.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.705+0000 7f7b707c0700 1 Processor -- start 2023-11-28T13:04:32.706 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.705+0000 7f7b707c0700 1 -- start start 2023-11-28T13:04:32.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b707c0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7b680d0a00 0x7f7b6815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b707c0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 0x7f7b680b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b707c0700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6806b420 con 0x7f7b680ce040 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b707c0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b680b32f0 con 0x7f7b680d0a00 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b707c0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6806ff60 con 0x7f7b680b2930 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b6e55c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f7b680ce040 0x7f7b6806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60936/0 (socket says 172.21.15.153:60936) 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b6e55c700 1 -- 172.21.15.153:0/1345132247 learned_addr learned my addr 172.21.15.153:0/1345132247 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b6ed5d700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 0x7f7b680b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b6dd5b700 1 --2- 172.21.15.153:0/1345132247 >> v2:172.21.15.177:3300/0 conn(0x7f7b680d0a00 0x7f7b6815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195929851 0 0) 0x7f7b6806b420 con 0x7f7b680ce040 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.706+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b540035b0 con 0x7f7b680ce040 2023-11-28T13:04:32.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3732810266 0 0) 0x7f7b540035b0 con 0x7f7b680ce040 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 msgr2=0x7f7b680b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 0x7f7b680b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 >> v2:172.21.15.177:3300/0 conn(0x7f7b680d0a00 msgr2=0x7f7b6815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 --2- 172.21.15.153:0/1345132247 >> v2:172.21.15.177:3300/0 conn(0x7f7b680d0a00 0x7f7b6815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6806b700 con 0x7f7b680ce040 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f7b5c0017f0 con 0x7f7b680ce040 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 >> v1:172.21.15.153:6789/0 conn(0x7f7b680ce040 legacy=0x7f7b6806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 0x7f7b54003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54003f80 0x7f7b54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b54004d90 0x7f7b54005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680b32f0 con 0x7f7b54004d90 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6806ff60 con 0x7f7b54003f80 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5400ba10 con 0x7f7b680b2930 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6ed5d700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b54004d90 0x7f7b54005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6e55c700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 0x7f7b54003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6dd5b700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54003f80 0x7f7b54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6ed5d700 1 -- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 msgr2=0x7f7b54003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6ed5d700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680b2930 0x7f7b54003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6ed5d700 1 -- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54003f80 msgr2=0x7f7b54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6ed5d700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54003f80 0x7f7b54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6ed5d700 1 -- 172.21.15.153:0/1345132247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540035b0 con 0x7f7b54004d90 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6ed5d700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b54004d90 0x7f7b54005200 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009fc0 tx=0x7f7b54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b6ceb1b49e51b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.707+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000ec30 con 0x7f7b54004d90 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000f040 con 0x7f7b54004d90 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b6d55a700 1 -- 172.21.15.153:0/1345132247 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b60014480 con 0x7f7b54004d90 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b707c0700 1 -- 172.21.15.153:0/1345132247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b54004d90 msgr2=0x7f7b54005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b707c0700 1 --2- 172.21.15.153:0/1345132247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b54004d90 0x7f7b54005200 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009fc0 tx=0x7f7b54005880 comp rx=0 tx=0).stop 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b6ed5d700 1 -- 172.21.15.153:0/1345132247 reap_dead start 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b707c0700 1 -- 172.21.15.153:0/1345132247 shutdown_connections 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b707c0700 1 -- 172.21.15.153:0/1345132247 >> 172.21.15.153:0/1345132247 conn(0x7f7b6806aed0 msgr2=0x7f7b6806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b707c0700 1 -- 172.21.15.153:0/1345132247 shutdown_connections 2023-11-28T13:04:32.713 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.708+0000 7f7b707c0700 1 -- 172.21.15.153:0/1345132247 wait complete. 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.710+0000 7f7b707c0700 1 Processor -- start 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.710+0000 7f7b707c0700 1 -- start start 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b707c0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54004d90 0x7f7b680c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b707c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b68152250 0x7f7b680c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b707c0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680c89a0 0x7f7b6813f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b707c0700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680c9580 con 0x7f7b68152250 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b707c0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680c9ec0 con 0x7f7b54004d90 2023-11-28T13:04:32.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b707c0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b680c8f30 con 0x7f7b680c89a0 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b68152250 0x7f7b680c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b68152250 0x7f7b680c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42786/0 (socket says 172.21.15.153:42786) 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 -- 172.21.15.153:0/2569904612 learned_addr learned my addr 172.21.15.153:0/2569904612 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6ed5d700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680c89a0 0x7f7b6813f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6e55c700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54004d90 0x7f7b680c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 -- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680c89a0 msgr2=0x7f7b6813f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680c89a0 0x7f7b6813f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 -- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54004d90 msgr2=0x7f7b680c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54004d90 0x7f7b680c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 -- 172.21.15.153:0/2569904612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6806bcc0 con 0x7f7b68152250 2023-11-28T13:04:32.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.711+0000 7f7b6dd5b700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b68152250 0x7f7b680c8460 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7b64011ad0 tx=0x7f7b64011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.712+0000 7f7b5affd700 1 -- 172.21.15.153:0/2569904612 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400f8e0 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.712+0000 7f7b5affd700 1 -- 172.21.15.153:0/2569904612 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6400fa60 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.712+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6806b9e0 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.712+0000 7f7b5affd700 1 -- 172.21.15.153:0/2569904612 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400fe10 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.712+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6806bfa0 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.712+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b68068690 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.713+0000 7f7b5affd700 1 -- 172.21.15.153:0/2569904612 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b64014440 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.713+0000 7f7b5affd700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b4c0682f0 0x7f7b4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.713+0000 7f7b5affd700 1 -- 172.21.15.153:0/2569904612 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7b640863e0 con 0x7f7b68152250 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.713+0000 7f7b6e55c700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b4c0682f0 0x7f7b4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.714+0000 7f7b6e55c700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b4c0682f0 0x7f7b4c06a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000ef0 tx=0x7f7b5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:32.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.718+0000 7f7b5affd700 1 -- 172.21.15.153:0/2569904612 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b64052c10 con 0x7f7b68152250 2023-11-28T13:04:32.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.941+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7b6815f490 con 0x7f7b68152250 2023-11-28T13:04:32.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.942+0000 7f7b5affd700 1 -- 172.21.15.153:0/2569904612 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f7b64056230 con 0x7f7b68152250 2023-11-28T13:04:32.943 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:32.943 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.943+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b4c0682f0 msgr2=0x7f7b4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.943+0000 7f7b707c0700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b4c0682f0 0x7f7b4c06a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000ef0 tx=0x7f7b5c009040 comp rx=0 tx=0).stop 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b68152250 msgr2=0x7f7b680c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b68152250 0x7f7b680c8460 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7b64011ad0 tx=0x7f7b64011de0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 shutdown_connections 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b680c89a0 0x7f7b6813f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b54004d90 0x7f7b680c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7b4c0682f0 0x7f7b4c06a7a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.945 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 --2- 172.21.15.153:0/2569904612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b68152250 0x7f7b680c8460 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 >> 172.21.15.153:0/2569904612 conn(0x7f7b6806aed0 msgr2=0x7f7b680700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:32.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 shutdown_connections 2023-11-28T13:04:32.946 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:32.944+0000 7f7b707c0700 1 -- 172.21.15.153:0/2569904612 wait complete. 2023-11-28T13:04:32.956 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:37.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:38.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.088+0000 7f8265834700 1 Processor -- start 2023-11-28T13:04:38.089 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.088+0000 7f8265834700 1 -- start start 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f8265834700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8260161340 0x7f82600d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f8265834700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82600d2320 0x7f82600cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f8265834700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600d1950 con 0x7f826015c9b0 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f8265834700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82600b5800 con 0x7f8260161340 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f8265834700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600b5300 con 0x7f82600d2320 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f825f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82600d2320 0x7f82600cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f825e7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8260161340 0x7f82600d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f825f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82600d2320 0x7f82600cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:60406/0 (socket says 172.21.15.153:60406) 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.089+0000 7f825f7fe700 1 -- 172.21.15.153:0/130446586 learned_addr learned my addr 172.21.15.153:0/130446586 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:38.092 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825e7fc700 1 -- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82600d2320 msgr2=0x7f82600cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825e7fc700 1 --2- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82600d2320 0x7f82600cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825e7fc700 1 -- 172.21.15.153:0/130446586 >> v1:172.21.15.177:6789/0 conn(0x7f826015c9b0 legacy=0x7f82600b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825e7fc700 1 -- 172.21.15.153:0/130446586 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600b5520 con 0x7f8260161340 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825f7fe700 1 --2- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82600d2320 0x7f82600cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825dffb700 1 -- 172.21.15.153:0/130446586 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307123582 0 0) 0x7f82600d1950 con 0x7f826015c9b0 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825e7fc700 1 --2- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 0x7f82600d1de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8250009840 tx=0x7f8250009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=192cf8774a255485 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825dffb700 1 -- 172.21.15.153:0/130446586 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250011830 con 0x7f8260161340 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825dffb700 1 -- 172.21.15.153:0/130446586 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250011990 con 0x7f8260161340 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.090+0000 7f825dffb700 1 -- 172.21.15.153:0/130446586 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250011d20 con 0x7f8260161340 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 -- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 msgr2=0x7f82600d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 --2- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 0x7f82600d1de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8250009840 tx=0x7f8250009b50 comp rx=0 tx=0).stop 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 -- 172.21.15.153:0/130446586 shutdown_connections 2023-11-28T13:04:38.093 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 --2- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82600d2320 0x7f82600cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 --2- 172.21.15.153:0/130446586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 0x7f82600d1de0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 -- 172.21.15.153:0/130446586 >> 172.21.15.153:0/130446586 conn(0x7f8260072600 msgr2=0x7f82600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:38.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 -- 172.21.15.153:0/130446586 shutdown_connections 2023-11-28T13:04:38.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.091+0000 7f8265834700 1 -- 172.21.15.153:0/130446586 wait complete. 2023-11-28T13:04:38.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.092+0000 7f8265834700 1 Processor -- start 2023-11-28T13:04:38.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.093+0000 7f8265834700 1 -- start start 2023-11-28T13:04:38.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.093+0000 7f8265834700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82600d2320 0x7f826014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:38.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.093+0000 7f8265834700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826015c9b0 0x7f826015fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:38.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.093+0000 7f8265834700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 0x7f82601e7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.093+0000 7f8265834700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c9860 con 0x7f82600d2320 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.093+0000 7f8265834700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c7140 con 0x7f8260161340 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.093+0000 7f8265834700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826014e6b0 con 0x7f826015c9b0 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82600d2320 0x7f826014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82600d2320 0x7f826014e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42798/0 (socket says 172.21.15.153:42798) 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 -- 172.21.15.153:0/140225542 learned_addr learned my addr 172.21.15.153:0/140225542 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825e7fc700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826015c9b0 0x7f826015fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825f7fe700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 0x7f82601e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 -- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826015c9b0 msgr2=0x7f826015fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826015c9b0 0x7f826015fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 -- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 msgr2=0x7f82601e7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:38.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 0x7f82601e7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 -- 172.21.15.153:0/140225542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009b30 con 0x7f82600d2320 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f825effd700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82600d2320 0x7f826014e170 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f825400c550 tx=0x7f825400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f8247fff700 1 -- 172.21.15.153:0/140225542 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82540054e0 con 0x7f82600d2320 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f8247fff700 1 -- 172.21.15.153:0/140225542 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8254003680 con 0x7f82600d2320 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82500094a0 con 0x7f82600d2320 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f8247fff700 1 -- 172.21.15.153:0/140225542 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254003a30 con 0x7f82600d2320 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.094+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82600d15e0 con 0x7f82600d2320 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.095+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82600d1300 con 0x7f82600d2320 2023-11-28T13:04:38.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.096+0000 7f8247fff700 1 -- 172.21.15.153:0/140225542 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8254019030 con 0x7f82600d2320 2023-11-28T13:04:38.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.096+0000 7f8247fff700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f824c0682f0 0x7f824c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:38.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.096+0000 7f8247fff700 1 -- 172.21.15.153:0/140225542 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8254085040 con 0x7f82600d2320 2023-11-28T13:04:38.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.096+0000 7f825e7fc700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f824c0682f0 0x7f824c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:38.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.097+0000 7f825e7fc700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f824c0682f0 0x7f824c06a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82500090b0 tx=0x7f8250008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:38.101 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.100+0000 7f8247fff700 1 -- 172.21.15.153:0/140225542 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8254051720 con 0x7f82600d2320 2023-11-28T13:04:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.324+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f826015cd80 con 0x7f82600d2320 2023-11-28T13:04:38.325 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.324+0000 7f8247fff700 1 -- 172.21.15.153:0/140225542 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8254017020 con 0x7f82600d2320 2023-11-28T13:04:38.325 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:38.326 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f824c0682f0 msgr2=0x7f824c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f824c0682f0 0x7f824c06a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82500090b0 tx=0x7f8250008030 comp rx=0 tx=0).stop 2023-11-28T13:04:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82600d2320 msgr2=0x7f826014e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82600d2320 0x7f826014e170 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f825400c550 tx=0x7f825400c860 comp rx=0 tx=0).stop 2023-11-28T13:04:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 shutdown_connections 2023-11-28T13:04:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f824c0682f0 0x7f824c06a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f82600d2320 0x7f826014e170 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260161340 0x7f82601e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 --2- 172.21.15.153:0/140225542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826015c9b0 0x7f826015fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 >> 172.21.15.153:0/140225542 conn(0x7f8260072600 msgr2=0x7f82600b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 shutdown_connections 2023-11-28T13:04:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:38.326+0000 7f8265834700 1 -- 172.21.15.153:0/140225542 wait complete. 2023-11-28T13:04:38.339 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:43.340 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:43.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.471+0000 7f8736fa9700 1 Processor -- start 2023-11-28T13:04:43.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.472+0000 7f8736fa9700 1 -- start start 2023-11-28T13:04:43.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.472+0000 7f8736fa9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 0x7f87300ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.472+0000 7f8736fa9700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873015ad60 con 0x7f87300cf070 2023-11-28T13:04:43.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.472+0000 7f8736fa9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300adb90 con 0x7f87300b3090 2023-11-28T13:04:43.476 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.472+0000 7f8736fa9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87300cf4f0 con 0x7f87300cb620 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.472+0000 7f8735546700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f87300cf070 0x7f87300ae300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:60974/0 (socket says 172.21.15.153:60974) 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.472+0000 7f8735546700 1 -- 172.21.15.153:0/348895392 learned_addr learned my addr 172.21.15.153:0/348895392 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872ffff700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 0x7f87300ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321353130 0 0) 0x7f873015ad60 con 0x7f87300cf070 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87200035b0 con 0x7f87300cf070 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1776990934 0 0) 0x7f87200035b0 con 0x7f87300cf070 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 msgr2=0x7f87300ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 0x7f87300ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 >> v1:172.21.15.177:6789/0 conn(0x7f87300b3090 legacy=0x7f87300cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300ccdd0 con 0x7f87300cf070 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f87280019f0 con 0x7f87300cf070 2023-11-28T13:04:43.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 >> v1:172.21.15.153:6789/0 conn(0x7f87300cf070 legacy=0x7f87300ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8720003a30 0x7f8720003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 0x7f8720004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8720004d90 0x7f8720005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300cf4f0 con 0x7f8720004d90 2023-11-28T13:04:43.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872000ba10 con 0x7f8720003a30 2023-11-28T13:04:43.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872000bba0 con 0x7f87300cb620 2023-11-28T13:04:43.478 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f8735546700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8720004d90 0x7f8720005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f8734d45700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8720003a30 0x7f8720003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.473+0000 7f872ffff700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 0x7f8720004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8735546700 1 -- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 msgr2=0x7f8720004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:43.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8735546700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cb620 0x7f8720004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8735546700 1 -- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8720003a30 msgr2=0x7f8720003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:43.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8735546700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8720003a30 0x7f8720003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8735546700 1 -- 172.21.15.153:0/348895392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200035b0 con 0x7f8720004d90 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8735546700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8720004d90 0x7f8720005200 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f872800a830 tx=0x7f8720005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849de611a261c922 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728012600 con 0x7f8720004d90 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8728012780 con 0x7f8720004d90 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f872f7fe700 1 -- 172.21.15.153:0/348895392 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728012b30 con 0x7f8720004d90 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8736fa9700 1 -- 172.21.15.153:0/348895392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8720004d90 msgr2=0x7f8720005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8736fa9700 1 --2- 172.21.15.153:0/348895392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8720004d90 0x7f8720005200 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f872800a830 tx=0x7f8720005880 comp rx=0 tx=0).stop 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8735546700 1 -- 172.21.15.153:0/348895392 reap_dead start 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8736fa9700 1 -- 172.21.15.153:0/348895392 shutdown_connections 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8736fa9700 1 -- 172.21.15.153:0/348895392 >> 172.21.15.153:0/348895392 conn(0x7f873009a230 msgr2=0x7f873008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:43.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8736fa9700 1 -- 172.21.15.153:0/348895392 shutdown_connections 2023-11-28T13:04:43.481 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.474+0000 7f8736fa9700 1 -- 172.21.15.153:0/348895392 wait complete. 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.476+0000 7f8736fa9700 1 Processor -- start 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.476+0000 7f8736fa9700 1 -- start start 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f8736fa9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8730159030 0x7f87301efa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f8736fa9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87301eff80 0x7f87300cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f8736fa9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cdbe0 0x7f87300ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f8736fa9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300ce050 con 0x7f87301eff80 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f8736fa9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730159400 con 0x7f8730159030 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f8736fa9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87301f3030 con 0x7f87300cdbe0 2023-11-28T13:04:43.484 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f872ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87301eff80 0x7f87300cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f872ffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87301eff80 0x7f87300cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42820/0 (socket says 172.21.15.153:42820) 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.477+0000 7f872ffff700 1 -- 172.21.15.153:0/2757529045 learned_addr learned my addr 172.21.15.153:0/2757529045 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f8735546700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cdbe0 0x7f87300ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f8734d45700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8730159030 0x7f87301efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872ffff700 1 -- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cdbe0 msgr2=0x7f87300ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872ffff700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cdbe0 0x7f87300ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872ffff700 1 -- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8730159030 msgr2=0x7f87301efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.485 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872ffff700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8730159030 0x7f87301efa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872ffff700 1 -- 172.21.15.153:0/2757529045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300cd390 con 0x7f87301eff80 2023-11-28T13:04:43.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872ffff700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87301eff80 0x7f87300cd6a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f871c011aa0 tx=0x7f871c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:43.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872d7fa700 1 -- 172.21.15.153:0/2757529045 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00f980 con 0x7f87301eff80 2023-11-28T13:04:43.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872d7fa700 1 -- 172.21.15.153:0/2757529045 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c00fae0 con 0x7f87301eff80 2023-11-28T13:04:43.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87300cd0b0 con 0x7f87301eff80 2023-11-28T13:04:43.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f872d7fa700 1 -- 172.21.15.153:0/2757529045 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00fe90 con 0x7f87301eff80 2023-11-28T13:04:43.486 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.478+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87300adde0 con 0x7f87301eff80 2023-11-28T13:04:43.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.479+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87300745c0 con 0x7f87301eff80 2023-11-28T13:04:43.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.479+0000 7f872d7fa700 1 -- 172.21.15.153:0/2757529045 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f871c014440 con 0x7f87301eff80 2023-11-28T13:04:43.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.480+0000 7f872d7fa700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f87140682f0 0x7f871406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.480+0000 7f872d7fa700 1 -- 172.21.15.153:0/2757529045 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f871c0865c0 con 0x7f87301eff80 2023-11-28T13:04:43.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.480+0000 7f8734d45700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f87140682f0 0x7f871406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.487 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.480+0000 7f8734d45700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f87140682f0 0x7f871406a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8724000c00 tx=0x7f8724009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:43.488 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.484+0000 7f872d7fa700 1 -- 172.21.15.153:0/2757529045 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f871c052df0 con 0x7f87301eff80 2023-11-28T13:04:43.708 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.707+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8730073b00 con 0x7f87301eff80 2023-11-28T13:04:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.707+0000 7f872d7fa700 1 -- 172.21.15.153:0/2757529045 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f871c056410 con 0x7f87301eff80 2023-11-28T13:04:43.709 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:43.709 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:43.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f87140682f0 msgr2=0x7f871406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f87140682f0 0x7f871406a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8724000c00 tx=0x7f8724009040 comp rx=0 tx=0).stop 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87301eff80 msgr2=0x7f87300cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87301eff80 0x7f87300cd6a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f871c011aa0 tx=0x7f871c011e60 comp rx=0 tx=0).stop 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 shutdown_connections 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87300cdbe0 0x7f87300ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8730159030 0x7f87301efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f87140682f0 0x7f871406a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.709+0000 7f8736fa9700 1 --2- 172.21.15.153:0/2757529045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87301eff80 0x7f87300cd6a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.710+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 >> 172.21.15.153:0/2757529045 conn(0x7f873009a230 msgr2=0x7f873006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.710+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 shutdown_connections 2023-11-28T13:04:43.711 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:43.710+0000 7f8736fa9700 1 -- 172.21.15.153:0/2757529045 wait complete. 2023-11-28T13:04:43.722 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:46.779 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:46.782 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:48.724 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:48.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.855+0000 7fe97804d700 1 Processor -- start 2023-11-28T13:04:48.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.855+0000 7fe97804d700 1 -- start start 2023-11-28T13:04:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe97804d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe970160740 0x7fe9700cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe97804d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe97006b060 0x7fe97006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe97804d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9700cc030 con 0x7fe970160b10 2023-11-28T13:04:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe97804d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9700ca720 con 0x7fe97006b060 2023-11-28T13:04:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe97804d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9700c9e10 con 0x7fe970160740 2023-11-28T13:04:48.859 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe9765ea700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe97006b060 0x7fe97006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe9765ea700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe97006b060 0x7fe97006b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42826/0 (socket says 172.21.15.153:42826) 2023-11-28T13:04:48.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe9765ea700 1 -- 172.21.15.153:0/937322286 learned_addr learned my addr 172.21.15.153:0/937322286 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:48.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe975de9700 1 --2- 172.21.15.153:0/937322286 >> v2:172.21.15.177:3300/0 conn(0x7fe970160740 0x7fe9700cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe9765ea700 1 -- 172.21.15.153:0/937322286 >> v2:172.21.15.177:3300/0 conn(0x7fe970160740 msgr2=0x7fe9700cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe9765ea700 1 --2- 172.21.15.153:0/937322286 >> v2:172.21.15.177:3300/0 conn(0x7fe970160740 0x7fe9700cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe9765ea700 1 -- 172.21.15.153:0/937322286 >> v1:172.21.15.177:6789/0 conn(0x7fe970160b10 legacy=0x7fe9700cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.856+0000 7fe9765ea700 1 -- 172.21.15.153:0/937322286 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97006efc0 con 0x7fe97006b060 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.857+0000 7fe9765ea700 1 --2- 172.21.15.153:0/937322286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe97006b4a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe9700b23b0 tx=0x7fe96800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c31a4ad8db0316 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.857+0000 7fe974de7700 1 -- 172.21.15.153:0/937322286 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800e070 con 0x7fe97006b060 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.857+0000 7fe974de7700 1 -- 172.21.15.153:0/937322286 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96800b9c0 con 0x7fe97006b060 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.857+0000 7fe974de7700 1 -- 172.21.15.153:0/937322286 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800bd70 con 0x7fe97006b060 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 -- 172.21.15.153:0/937322286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 msgr2=0x7fe97006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 --2- 172.21.15.153:0/937322286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe97006b4a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe9700b23b0 tx=0x7fe96800b330 comp rx=0 tx=0).stop 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 -- 172.21.15.153:0/937322286 shutdown_connections 2023-11-28T13:04:48.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 --2- 172.21.15.153:0/937322286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe97006b4a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 --2- 172.21.15.153:0/937322286 >> v2:172.21.15.177:3300/0 conn(0x7fe970160740 0x7fe9700cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 -- 172.21.15.153:0/937322286 >> 172.21.15.153:0/937322286 conn(0x7fe9700b12b0 msgr2=0x7fe9700b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:48.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 -- 172.21.15.153:0/937322286 shutdown_connections 2023-11-28T13:04:48.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.858+0000 7fe97804d700 1 -- 172.21.15.153:0/937322286 wait complete. 2023-11-28T13:04:48.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.859+0000 7fe97804d700 1 Processor -- start 2023-11-28T13:04:48.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.859+0000 7fe97804d700 1 -- start start 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe970159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe970160740 0x7fe9700c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe970160b10 0x7fe9700c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970159730 con 0x7fe97006b060 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe970158ab0 con 0x7fe970160b10 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe970158d90 con 0x7fe970160740 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe970159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe970159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42830/0 (socket says 172.21.15.153:42830) 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 -- 172.21.15.153:0/2963491249 learned_addr learned my addr 172.21.15.153:0/2963491249 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 -- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe970160740 msgr2=0x7fe9700c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe9765ea700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe970160b10 0x7fe9700c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe9755e8700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe970160740 0x7fe9700c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe970160740 0x7fe9700c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 -- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe970160b10 msgr2=0x7fe9700c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe970160b10 0x7fe9700c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 -- 172.21.15.153:0/2963491249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe968009b30 con 0x7fe97006b060 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe975de9700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe970159b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e810 tx=0x7fe96000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe9667fc700 1 -- 172.21.15.153:0/2963491249 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960013610 con 0x7fe97006b060 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe9667fc700 1 -- 172.21.15.153:0/2963491249 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960013770 con 0x7fe97006b060 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9700ca3b0 con 0x7fe97006b060 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe9667fc700 1 -- 172.21.15.153:0/2963491249 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960013b20 con 0x7fe97006b060 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.860+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9700ca0d0 con 0x7fe97006b060 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.861+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe970160f60 con 0x7fe97006b060 2023-11-28T13:04:48.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.862+0000 7fe9667fc700 1 -- 172.21.15.153:0/2963491249 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe960023070 con 0x7fe97006b060 2023-11-28T13:04:48.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.862+0000 7fe9667fc700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9540682f0 0x7fe95406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.862+0000 7fe9667fc700 1 -- 172.21.15.153:0/2963491249 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe960085c50 con 0x7fe97006b060 2023-11-28T13:04:48.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.862+0000 7fe9755e8700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9540682f0 0x7fe95406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.863+0000 7fe9755e8700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9540682f0 0x7fe95406a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe9701ea820 tx=0x7fe96c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:48.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:48.866+0000 7fe9667fc700 1 -- 172.21.15.153:0/2963491249 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe9600523c0 con 0x7fe97006b060 2023-11-28T13:04:49.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.094+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe9700cb180 con 0x7fe97006b060 2023-11-28T13:04:49.095 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.094+0000 7fe9667fc700 1 -- 172.21.15.153:0/2963491249 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe9600559e0 con 0x7fe97006b060 2023-11-28T13:04:49.095 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:49.095 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:49.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9540682f0 msgr2=0x7fe95406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9540682f0 0x7fe95406a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe9701ea820 tx=0x7fe96c009040 comp rx=0 tx=0).stop 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 msgr2=0x7fe970159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe970159b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e810 tx=0x7fe96000ebd0 comp rx=0 tx=0).stop 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 shutdown_connections 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe9540682f0 0x7fe95406a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe97006b060 0x7fe970159b50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.096+0000 7fe97804d700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe970160b10 0x7fe9700c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.097+0000 7fe97804d700 1 --2- 172.21.15.153:0/2963491249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe970160740 0x7fe9700c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.097+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 >> 172.21.15.153:0/2963491249 conn(0x7fe9700b12b0 msgr2=0x7fe9700b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.097+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 shutdown_connections 2023-11-28T13:04:49.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:49.097+0000 7fe97804d700 1 -- 172.21.15.153:0/2963491249 wait complete. 2023-11-28T13:04:49.109 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:54.110 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:54.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.241+0000 7f94bc92b700 1 Processor -- start 2023-11-28T13:04:54.242 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.241+0000 7f94bc92b700 1 -- start start 2023-11-28T13:04:54.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94bc92b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f94b806dee0 0x7f94b806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94bc92b700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b815d060 con 0x7f94b8160540 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94bc92b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b815d340 con 0x7f94b8071040 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94bc92b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b815d620 con 0x7f94b806dee0 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94b5d9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f94b806dee0 0x7f94b806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94b6d9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f94b8160540 0x7f94b8160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32774/0 (socket says 172.21.15.153:32774) 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94b6d9d700 1 -- 172.21.15.153:0/77434256 learned_addr learned my addr 172.21.15.153:0/77434256 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94b5d9b700 1 -- 172.21.15.153:0/77434256 >> v1:172.21.15.177:6789/0 conn(0x7f94b8071040 legacy=0x7f94b8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.242+0000 7f94b5d9b700 1 -- 172.21.15.153:0/77434256 >> v1:172.21.15.153:6789/0 conn(0x7f94b8160540 legacy=0x7f94b8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94b559a700 1 -- 172.21.15.153:0/77434256 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117859524 0 0) 0x7f94b815d060 con 0x7f94b8160540 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94b5d9b700 1 -- 172.21.15.153:0/77434256 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b815d900 con 0x7f94b806dee0 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94b5d9b700 1 --2- 172.21.15.153:0/77434256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b806dee0 0x7f94b806e330 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009840 tx=0x7f94a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803f24d16219e681 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:54.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94b559a700 1 -- 172.21.15.153:0/77434256 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8004230 con 0x7f94b806dee0 2023-11-28T13:04:54.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94b559a700 1 -- 172.21.15.153:0/77434256 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a8004390 con 0x7f94b806dee0 2023-11-28T13:04:54.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94b559a700 1 -- 172.21.15.153:0/77434256 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80036a0 con 0x7f94b806dee0 2023-11-28T13:04:54.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94bc92b700 1 -- 172.21.15.153:0/77434256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b806dee0 msgr2=0x7f94b806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94bc92b700 1 --2- 172.21.15.153:0/77434256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b806dee0 0x7f94b806e330 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009840 tx=0x7f94a8009b50 comp rx=0 tx=0).stop 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94bc92b700 1 -- 172.21.15.153:0/77434256 shutdown_connections 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94bc92b700 1 --2- 172.21.15.153:0/77434256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b806dee0 0x7f94b806e330 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94bc92b700 1 -- 172.21.15.153:0/77434256 >> 172.21.15.153:0/77434256 conn(0x7f94b80d0000 msgr2=0x7f94b80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94bc92b700 1 -- 172.21.15.153:0/77434256 shutdown_connections 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.243+0000 7f94bc92b700 1 -- 172.21.15.153:0/77434256 wait complete. 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.245+0000 7f94bc92b700 1 Processor -- start 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.245+0000 7f94bc92b700 1 -- start start 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94bc92b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94b806dee0 0x7f94b81572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:54.248 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94bc92b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b8071040 0x7f94b8156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94bc92b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b8160540 0x7f94b80c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94bc92b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80b3ae0 con 0x7f94b8160540 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94bc92b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80b2ed0 con 0x7f94b8071040 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94bc92b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80b31b0 con 0x7f94b806dee0 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b8160540 0x7f94b80c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b8160540 0x7f94b80c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42848/0 (socket says 172.21.15.153:42848) 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 -- 172.21.15.153:0/377497730 learned_addr learned my addr 172.21.15.153:0/377497730 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b5d9b700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b8071040 0x7f94b8156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b659c700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94b806dee0 0x7f94b81572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 -- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94b806dee0 msgr2=0x7f94b81572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94b806dee0 0x7f94b81572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.249 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 -- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b8071040 msgr2=0x7f94b8156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b8071040 0x7f94b8156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 -- 172.21.15.153:0/377497730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80094a0 con 0x7f94b8160540 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94b6d9d700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b8160540 0x7f94b80c6320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f94b000cba0 tx=0x7f94b000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94a6ffd700 1 -- 172.21.15.153:0/377497730 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b000aaa0 con 0x7f94b8160540 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94a6ffd700 1 -- 172.21.15.153:0/377497730 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b000ac20 con 0x7f94b8160540 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b815ccd0 con 0x7f94b8160540 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.246+0000 7f94a6ffd700 1 -- 172.21.15.153:0/377497730 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00070d0 con 0x7f94b8160540 2023-11-28T13:04:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.247+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b806e870 con 0x7f94b8160540 2023-11-28T13:04:54.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.247+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b806a270 con 0x7f94b8160540 2023-11-28T13:04:54.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.248+0000 7f94a6ffd700 1 -- 172.21.15.153:0/377497730 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f94b001a030 con 0x7f94b8160540 2023-11-28T13:04:54.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.248+0000 7f94a6ffd700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f94980682f0 0x7f949806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:54.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.248+0000 7f94a6ffd700 1 -- 172.21.15.153:0/377497730 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f94b00858a0 con 0x7f94b8160540 2023-11-28T13:04:54.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.248+0000 7f94b659c700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f94980682f0 0x7f949806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:54.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.249+0000 7f94b659c700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f94980682f0 0x7f949806a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f94b80b5ec0 tx=0x7f94ac0090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:54.253 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.253+0000 7f94a6ffd700 1 -- 172.21.15.153:0/377497730 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f94b0052070 con 0x7f94b8160540 2023-11-28T13:04:54.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.476+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f94b80720a0 con 0x7f94b8160540 2023-11-28T13:04:54.477 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.476+0000 7f94a6ffd700 1 -- 172.21.15.153:0/377497730 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f94b0055690 con 0x7f94b8160540 2023-11-28T13:04:54.478 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:54.478 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:54.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f94980682f0 msgr2=0x7f949806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f94980682f0 0x7f949806a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f94b80b5ec0 tx=0x7f94ac0090e0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b8160540 msgr2=0x7f94b80c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.479 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b8160540 0x7f94b80c6320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f94b000cba0 tx=0x7f94b000ceb0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 shutdown_connections 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f94980682f0 0x7f949806a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94b8160540 0x7f94b80c6320 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94b806dee0 0x7f94b81572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 --2- 172.21.15.153:0/377497730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b8071040 0x7f94b8156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 >> 172.21.15.153:0/377497730 conn(0x7f94b80d0000 msgr2=0x7f94b80af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 shutdown_connections 2023-11-28T13:04:54.480 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:54.478+0000 7f94bc92b700 1 -- 172.21.15.153:0/377497730 wait complete. 2023-11-28T13:04:54.491 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:59.492 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:59.626 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.625+0000 7f9d24848700 1 Processor -- start 2023-11-28T13:04:59.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.626+0000 7f9d24848700 1 -- start start 2023-11-28T13:04:59.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.626+0000 7f9d24848700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d20071040 0x7f9d20071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d24848700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9d2006dee0 0x7f9d2006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d24848700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d20071950 con 0x7f9d2006cdc0 2023-11-28T13:04:59.629 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d24848700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d20161550 con 0x7f9d2006dee0 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d24848700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2015d060 con 0x7f9d20071040 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9d2006dee0 0x7f9d2006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1ed9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9d2006cdc0 0x7f9d2006d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32788/0 (socket says 172.21.15.153:32788) 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1ed9d700 1 -- 172.21.15.153:0/543138506 learned_addr learned my addr 172.21.15.153:0/543138506 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9d2006dee0 0x7f9d2006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42854/0 (socket says 172.21.15.153:42854) 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1e59c700 1 --2- 172.21.15.153:0/543138506 >> v2:172.21.15.177:3300/0 conn(0x7f9d20071040 0x7f9d20071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1dd9b700 1 -- 172.21.15.153:0/543138506 >> v2:172.21.15.177:3300/0 conn(0x7f9d20071040 msgr2=0x7f9d20071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1dd9b700 1 --2- 172.21.15.153:0/543138506 >> v2:172.21.15.177:3300/0 conn(0x7f9d20071040 0x7f9d20071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1dd9b700 1 -- 172.21.15.153:0/543138506 >> v1:172.21.15.153:6789/0 conn(0x7f9d2006cdc0 legacy=0x7f9d2006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.630 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1dd9b700 1 -- 172.21.15.153:0/543138506 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2015d340 con 0x7f9d2006dee0 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1d59a700 1 -- 172.21.15.153:0/543138506 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316025446 0 0) 0x7f9d20071950 con 0x7f9d2006cdc0 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1dd9b700 1 --2- 172.21.15.153:0/543138506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006dee0 0x7f9d2006e330 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9d200d0f50 tx=0x7f9d10009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ca9608ed58c03d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1d59a700 1 -- 172.21.15.153:0/543138506 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011830 con 0x7f9d2006dee0 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1d59a700 1 -- 172.21.15.153:0/543138506 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d10011990 con 0x7f9d2006dee0 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.627+0000 7f9d1d59a700 1 -- 172.21.15.153:0/543138506 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011d20 con 0x7f9d2006dee0 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 -- 172.21.15.153:0/543138506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006dee0 msgr2=0x7f9d2006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 --2- 172.21.15.153:0/543138506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006dee0 0x7f9d2006e330 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9d200d0f50 tx=0x7f9d10009b20 comp rx=0 tx=0).stop 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 -- 172.21.15.153:0/543138506 shutdown_connections 2023-11-28T13:04:59.631 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 --2- 172.21.15.153:0/543138506 >> v2:172.21.15.177:3300/0 conn(0x7f9d20071040 0x7f9d20071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 --2- 172.21.15.153:0/543138506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006dee0 0x7f9d2006e330 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 -- 172.21.15.153:0/543138506 >> 172.21.15.153:0/543138506 conn(0x7f9d200d0000 msgr2=0x7f9d200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:59.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 -- 172.21.15.153:0/543138506 shutdown_connections 2023-11-28T13:04:59.632 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.628+0000 7f9d24848700 1 -- 172.21.15.153:0/543138506 wait complete. 2023-11-28T13:04:59.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.629+0000 7f9d24848700 1 Processor -- start 2023-11-28T13:04:59.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.629+0000 7f9d24848700 1 -- start start 2023-11-28T13:04:59.634 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d24848700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006cdc0 0x7f9d201524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d24848700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2006dee0 0x7f9d201f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d24848700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d20071040 0x7f9d20153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d24848700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201f6580 con 0x7f9d2006cdc0 2023-11-28T13:04:59.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d24848700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201f6860 con 0x7f9d20071040 2023-11-28T13:04:59.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d24848700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200b2fb0 con 0x7f9d2006dee0 2023-11-28T13:04:59.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d1e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006cdc0 0x7f9d201524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.635 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d1e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006cdc0 0x7f9d201524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42858/0 (socket says 172.21.15.153:42858) 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.630+0000 7f9d1e59c700 1 -- 172.21.15.153:0/1965058296 learned_addr learned my addr 172.21.15.153:0/1965058296 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d1e59c700 1 -- 172.21.15.153:0/1965058296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2006dee0 msgr2=0x7f9d201f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d1ed9d700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d20071040 0x7f9d20153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d1e59c700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2006dee0 0x7f9d201f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d1e59c700 1 -- 172.21.15.153:0/1965058296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d20071040 msgr2=0x7f9d20153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d1e59c700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d20071040 0x7f9d20153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d1e59c700 1 -- 172.21.15.153:0/1965058296 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100094a0 con 0x7f9d2006cdc0 2023-11-28T13:04:59.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d1e59c700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006cdc0 0x7f9d201524a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e870 tx=0x7f9d1400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d0effd700 1 -- 172.21.15.153:0/1965058296 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400ee70 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d0effd700 1 -- 172.21.15.153:0/1965058296 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400c850 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2015d900 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d0effd700 1 -- 172.21.15.153:0/1965058296 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400cbc0 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.631+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2015d620 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.632+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2006e870 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.632+0000 7f9d0effd700 1 -- 172.21.15.153:0/1965058296 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9d14023070 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.633+0000 7f9d0effd700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d000682f0 0x7f9d0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.633+0000 7f9d0effd700 1 -- 172.21.15.153:0/1965058296 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9d14085ef0 con 0x7f9d2006cdc0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.633+0000 7f9d1dd9b700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d000682f0 0x7f9d0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.637 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.633+0000 7f9d1dd9b700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d000682f0 0x7f9d0006a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9d2006c3b0 tx=0x7f9d10010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:59.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.637+0000 7f9d0effd700 1 -- 172.21.15.153:0/1965058296 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d140526c0 con 0x7f9d2006cdc0 2023-11-28T13:04:59.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.863+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9d200720a0 con 0x7f9d2006cdc0 2023-11-28T13:04:59.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.863+0000 7f9d0effd700 1 -- 172.21.15.153:0/1965058296 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9d14055ce0 con 0x7f9d2006cdc0 2023-11-28T13:04:59.864 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:04:59.864 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:59.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d000682f0 msgr2=0x7f9d0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d000682f0 0x7f9d0006a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9d2006c3b0 tx=0x7f9d10010000 comp rx=0 tx=0).stop 2023-11-28T13:04:59.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006cdc0 msgr2=0x7f9d201524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006cdc0 0x7f9d201524a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e870 tx=0x7f9d1400ec30 comp rx=0 tx=0).stop 2023-11-28T13:04:59.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 shutdown_connections 2023-11-28T13:04:59.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9d000682f0 0x7f9d0006a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d2006cdc0 0x7f9d201524a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d20071040 0x7f9d20153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 --2- 172.21.15.153:0/1965058296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d2006dee0 0x7f9d201f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 >> 172.21.15.153:0/1965058296 conn(0x7f9d200d0000 msgr2=0x7f9d2006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:59.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 shutdown_connections 2023-11-28T13:04:59.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:04:59.865+0000 7f9d24848700 1 -- 172.21.15.153:0/1965058296 wait complete. 2023-11-28T13:04:59.877 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:04.879 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:05.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c7a2b4700 1 Processor -- start 2023-11-28T13:05:05.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c7a2b4700 1 -- start start 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c7a2b4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1c74140950 0x7f1c740b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c7a2b4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1c740b2e70 0x7f1c7406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c7a2b4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7413ff00 con 0x7f1c740cd8c0 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c7a2b4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c741404c0 con 0x7f1c74140950 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c7a2b4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c74140dc0 con 0x7f1c740b2e70 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.059+0000 7f1c737fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1c74140950 0x7f1c740b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c737fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f1c74140950 0x7f1c740b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42868/0 (socket says 172.21.15.153:42868) 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c737fe700 1 -- 172.21.15.153:0/3996850921 learned_addr learned my addr 172.21.15.153:0/3996850921 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c78851700 1 --2- 172.21.15.153:0/3996850921 >> v2:172.21.15.177:3300/0 conn(0x7f1c740b2e70 0x7f1c7406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c737fe700 1 -- 172.21.15.153:0/3996850921 >> v2:172.21.15.177:3300/0 conn(0x7f1c740b2e70 msgr2=0x7f1c7406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c737fe700 1 --2- 172.21.15.153:0/3996850921 >> v2:172.21.15.177:3300/0 conn(0x7f1c740b2e70 0x7f1c7406f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c737fe700 1 -- 172.21.15.153:0/3996850921 >> v1:172.21.15.177:6789/0 conn(0x7f1c740cd8c0 legacy=0x7f1c740cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c737fe700 1 -- 172.21.15.153:0/3996850921 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7415aa80 con 0x7f1c74140950 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c737fe700 1 --2- 172.21.15.153:0/3996850921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c74140950 0x7f1c740b2930 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1c74144350 tx=0x7f1c68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591e57c8c9b29d1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c72ffd700 1 -- 172.21.15.153:0/3996850921 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800b870 con 0x7f1c74140950 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c72ffd700 1 -- 172.21.15.153:0/3996850921 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6800b9d0 con 0x7f1c74140950 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.060+0000 7f1c72ffd700 1 -- 172.21.15.153:0/3996850921 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800bd60 con 0x7f1c74140950 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/3996850921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c74140950 msgr2=0x7f1c740b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/3996850921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c74140950 0x7f1c740b2930 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1c74144350 tx=0x7f1c68009bd0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/3996850921 shutdown_connections 2023-11-28T13:05:05.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/3996850921 >> v2:172.21.15.177:3300/0 conn(0x7f1c740b2e70 0x7f1c7406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/3996850921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c74140950 0x7f1c740b2930 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/3996850921 >> 172.21.15.153:0/3996850921 conn(0x7f1c7406aed0 msgr2=0x7f1c7406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:05.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/3996850921 shutdown_connections 2023-11-28T13:05:05.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.061+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/3996850921 wait complete. 2023-11-28T13:05:05.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.062+0000 7f1c7a2b4700 1 Processor -- start 2023-11-28T13:05:05.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.062+0000 7f1c7a2b4700 1 -- start start 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c7a2b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c740b2e70 0x7f1c741f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c7a2b4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c740cd8c0 0x7f1c7413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c7a2b4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c74140950 0x7f1c7413d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c7a2b4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7414b4f0 con 0x7f1c740cd8c0 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c7a2b4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7406b380 con 0x7f1c740b2e70 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c7a2b4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c9cd0 con 0x7f1c74140950 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c737fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c740cd8c0 0x7f1c7413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c737fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c740cd8c0 0x7f1c7413ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42874/0 (socket says 172.21.15.153:42874) 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c737fe700 1 -- 172.21.15.153:0/4043449124 learned_addr learned my addr 172.21.15.153:0/4043449124 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c78851700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c74140950 0x7f1c7413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:05.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c73fff700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c740b2e70 0x7f1c741f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c737fe700 1 -- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c74140950 msgr2=0x7f1c7413d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c737fe700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c74140950 0x7f1c7413d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c737fe700 1 -- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c740b2e70 msgr2=0x7f1c741f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.063+0000 7f1c737fe700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c740b2e70 0x7f1c741f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c737fe700 1 -- 172.21.15.153:0/4043449124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009550 con 0x7f1c740cd8c0 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c73fff700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c740b2e70 0x7f1c741f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c737fe700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c740cd8c0 0x7f1c7413ccf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1c68000c00 tx=0x7f1c68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c70ff9700 1 -- 172.21.15.153:0/4043449124 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68004520 con 0x7f1c740cd8c0 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c70ff9700 1 -- 172.21.15.153:0/4043449124 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6801e070 con 0x7f1c740cd8c0 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740b35f0 con 0x7f1c740cd8c0 2023-11-28T13:05:05.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c70ff9700 1 -- 172.21.15.153:0/4043449124 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680038a0 con 0x7f1c740cd8c0 2023-11-28T13:05:05.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.064+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740b3310 con 0x7f1c740cd8c0 2023-11-28T13:05:05.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.065+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c74140150 con 0x7f1c740cd8c0 2023-11-28T13:05:05.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.065+0000 7f1c70ff9700 1 -- 172.21.15.153:0/4043449124 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c6800f050 con 0x7f1c740cd8c0 2023-11-28T13:05:05.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.066+0000 7f1c70ff9700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c600682f0 0x7f1c6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:05.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.066+0000 7f1c73fff700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c600682f0 0x7f1c6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:05.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.066+0000 7f1c70ff9700 1 -- 172.21.15.153:0/4043449124 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1c68020070 con 0x7f1c740cd8c0 2023-11-28T13:05:05.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.066+0000 7f1c73fff700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c600682f0 0x7f1c6006a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c741f5e60 tx=0x7f1c64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:05.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.070+0000 7f1c70ff9700 1 -- 172.21.15.153:0/4043449124 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c68053b60 con 0x7f1c740cd8c0 2023-11-28T13:05:05.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.294+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1c74068690 con 0x7f1c740cd8c0 2023-11-28T13:05:05.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.295+0000 7f1c70ff9700 1 -- 172.21.15.153:0/4043449124 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1c68057180 con 0x7f1c740cd8c0 2023-11-28T13:05:05.297 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:05.297 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c600682f0 msgr2=0x7f1c6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c600682f0 0x7f1c6006a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c741f5e60 tx=0x7f1c64009040 comp rx=0 tx=0).stop 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c740cd8c0 msgr2=0x7f1c7413ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c740cd8c0 0x7f1c7413ccf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1c68000c00 tx=0x7f1c68004750 comp rx=0 tx=0).stop 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 shutdown_connections 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c74140950 0x7f1c7413d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c740b2e70 0x7f1c741f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f1c600682f0 0x7f1c6006a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 --2- 172.21.15.153:0/4043449124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c740cd8c0 0x7f1c7413ccf0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:05.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 >> 172.21.15.153:0/4043449124 conn(0x7f1c7406aed0 msgr2=0x7f1c740d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 shutdown_connections 2023-11-28T13:05:05.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:05.297+0000 7f1c7a2b4700 1 -- 172.21.15.153:0/4043449124 wait complete. 2023-11-28T13:05:05.311 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:10.312 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:10.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.443+0000 7fb0f398c700 1 Processor -- start 2023-11-28T13:05:10.445 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.444+0000 7fb0f398c700 1 -- start start 2023-11-28T13:05:10.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.444+0000 7fb0f398c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb0ec15cef0 0x7fb0ec0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.444+0000 7fb0f398c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb0ec06ba90 0x7fb0ec0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.444+0000 7fb0f398c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ec0cc5f0 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.444+0000 7fb0f398c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec15d470 con 0x7fb0ec06ba90 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.444+0000 7fb0f398c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec15d750 con 0x7fb0ec15cef0 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb0ec06ba90 0x7fb0ec0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fb0ec06ba90 0x7fb0ec0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42884/0 (socket says 172.21.15.153:42884) 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 -- 172.21.15.153:0/2487113217 learned_addr learned my addr 172.21.15.153:0/2487113217 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:10.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1728700 1 --2- 172.21.15.153:0/2487113217 >> v2:172.21.15.177:3300/0 conn(0x7fb0ec15cef0 0x7fb0ec0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 -- 172.21.15.153:0/2487113217 >> v2:172.21.15.177:3300/0 conn(0x7fb0ec15cef0 msgr2=0x7fb0ec0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 --2- 172.21.15.153:0/2487113217 >> v2:172.21.15.177:3300/0 conn(0x7fb0ec15cef0 0x7fb0ec0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 -- 172.21.15.153:0/2487113217 >> v1:172.21.15.177:6789/0 conn(0x7fb0ec0cbdf0 legacy=0x7fb0ec06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 -- 172.21.15.153:0/2487113217 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec15da30 con 0x7fb0ec06ba90 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0f1f29700 1 --2- 172.21.15.153:0/2487113217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec0ca650 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009e50 tx=0x7fb0e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1f4e47c41d4374 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0e3fff700 1 -- 172.21.15.153:0/2487113217 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400e070 con 0x7fb0ec06ba90 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0e3fff700 1 -- 172.21.15.153:0/2487113217 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400b8d0 con 0x7fb0ec06ba90 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.445+0000 7fb0e3fff700 1 -- 172.21.15.153:0/2487113217 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400bc80 con 0x7fb0ec06ba90 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 -- 172.21.15.153:0/2487113217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec06ba90 msgr2=0x7fb0ec0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 --2- 172.21.15.153:0/2487113217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec0ca650 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009e50 tx=0x7fb0e400b330 comp rx=0 tx=0).stop 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 -- 172.21.15.153:0/2487113217 shutdown_connections 2023-11-28T13:05:10.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 --2- 172.21.15.153:0/2487113217 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec0ca650 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 --2- 172.21.15.153:0/2487113217 >> v2:172.21.15.177:3300/0 conn(0x7fb0ec15cef0 0x7fb0ec0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 -- 172.21.15.153:0/2487113217 >> 172.21.15.153:0/2487113217 conn(0x7fb0ec0d0000 msgr2=0x7fb0ec0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:10.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 -- 172.21.15.153:0/2487113217 shutdown_connections 2023-11-28T13:05:10.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.446+0000 7fb0f398c700 1 -- 172.21.15.153:0/2487113217 wait complete. 2023-11-28T13:05:10.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.447+0000 7fb0f398c700 1 Processor -- start 2023-11-28T13:05:10.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.447+0000 7fb0f398c700 1 -- start start 2023-11-28T13:05:10.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f398c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f398c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec0cbdf0 0x7fb0ec0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f398c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0ec15cef0 0x7fb0ec06a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f398c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec158950 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.453 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f398c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec1596f0 con 0x7fb0ec06ba90 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f398c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec1f36f0 con 0x7fb0ec15cef0 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec0cbdf0 0x7fb0ec0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec0cbdf0 0x7fb0ec0c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42886/0 (socket says 172.21.15.153:42886) 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 -- 172.21.15.153:0/285999003 learned_addr learned my addr 172.21.15.153:0/285999003 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f1728700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 -- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0ec15cef0 msgr2=0x7fb0ec06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f1f29700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0ec15cef0 0x7fb0ec06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0ec15cef0 0x7fb0ec06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 -- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0ec06ba90 msgr2=0x7fb0ec158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 -- 172.21.15.153:0/285999003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4009b30 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.454 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f1728700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f1f29700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0ec15cef0 0x7fb0ec06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.448+0000 7fb0f0f27700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec0cbdf0 0x7fb0ec0c9b50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c490 tx=0x7fb0e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.449+0000 7fb0e1ffb700 1 -- 172.21.15.153:0/285999003 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80052d0 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.449+0000 7fb0e1ffb700 1 -- 172.21.15.153:0/285999003 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e8005450 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.449+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec15dff0 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.449+0000 7fb0e1ffb700 1 -- 172.21.15.153:0/285999003 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8006aa0 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.449+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec15dd10 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.450+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec0cab90 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.450+0000 7fb0e1ffb700 1 -- 172.21.15.153:0/285999003 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0e8010050 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.450+0000 7fb0e1ffb700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb0d00682f0 0x7fb0d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.451+0000 7fb0e1ffb700 1 -- 172.21.15.153:0/285999003 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb0e8084e90 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.451+0000 7fb0f1728700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb0d00682f0 0x7fb0d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.451+0000 7fb0f1728700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb0d00682f0 0x7fb0d006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009ac0 tx=0x7fb0dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:10.456 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.455+0000 7fb0e1ffb700 1 -- 172.21.15.153:0/285999003 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0e804a5e0 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.683 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.681+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb0ec0af7d0 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.684 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.682+0000 7fb0e1ffb700 1 -- 172.21.15.153:0/285999003 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb0e8051660 con 0x7fb0ec0cbdf0 2023-11-28T13:05:10.684 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:10.684 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:10.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb0d00682f0 msgr2=0x7fb0d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb0d00682f0 0x7fb0d006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009ac0 tx=0x7fb0dc008040 comp rx=0 tx=0).stop 2023-11-28T13:05:10.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec0cbdf0 msgr2=0x7fb0ec0c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec0cbdf0 0x7fb0ec0c9b50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800c490 tx=0x7fb0e800c850 comp rx=0 tx=0).stop 2023-11-28T13:05:10.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 shutdown_connections 2023-11-28T13:05:10.686 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0ec15cef0 0x7fb0ec06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0ec06ba90 0x7fb0ec158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fb0d00682f0 0x7fb0d006a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 --2- 172.21.15.153:0/285999003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0ec0cbdf0 0x7fb0ec0c9b50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 >> 172.21.15.153:0/285999003 conn(0x7fb0ec0d0000 msgr2=0x7fb0ec06e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:10.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 shutdown_connections 2023-11-28T13:05:10.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:10.684+0000 7fb0f398c700 1 -- 172.21.15.153:0/285999003 wait complete. 2023-11-28T13:05:10.698 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:15.698 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:15.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.829+0000 7f619fc36700 1 Processor -- start 2023-11-28T13:05:15.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.829+0000 7f619fc36700 1 -- start start 2023-11-28T13:05:15.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619fc36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198071040 0x7f6198071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.840 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619fc36700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6198160540 0x7f61981609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619fc36700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6198071950 con 0x7f619806dee0 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619fc36700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619815d060 con 0x7f6198160540 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619fc36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619815d340 con 0x7f6198071040 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6198160540 0x7f61981609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f6198160540 0x7f61981609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42898/0 (socket says 172.21.15.153:42898) 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 -- 172.21.15.153:0/3190594558 learned_addr learned my addr 172.21.15.153:0/3190594558 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619d9d2700 1 --2- 172.21.15.153:0/3190594558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198071040 0x7f6198071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 -- 172.21.15.153:0/3190594558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198071040 msgr2=0x7f6198071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 --2- 172.21.15.153:0/3190594558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198071040 0x7f6198071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 -- 172.21.15.153:0/3190594558 >> v1:172.21.15.177:6789/0 conn(0x7f619806dee0 legacy=0x7f619806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:15.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 -- 172.21.15.153:0/3190594558 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619815d620 con 0x7f6198160540 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.830+0000 7f619e1d3700 1 --2- 172.21.15.153:0/3190594558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6198160540 0x7f61981609c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6190009e50 tx=0x7f619000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec047c0a850271 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619c9d0700 1 -- 172.21.15.153:0/3190594558 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000e070 con 0x7f6198160540 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619c9d0700 1 -- 172.21.15.153:0/3190594558 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6190003a20 con 0x7f6198160540 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619c9d0700 1 -- 172.21.15.153:0/3190594558 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190003dd0 con 0x7f6198160540 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 -- 172.21.15.153:0/3190594558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6198160540 msgr2=0x7f61981609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 --2- 172.21.15.153:0/3190594558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6198160540 0x7f61981609c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6190009e50 tx=0x7f619000b330 comp rx=0 tx=0).stop 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 -- 172.21.15.153:0/3190594558 shutdown_connections 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 --2- 172.21.15.153:0/3190594558 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6198160540 0x7f61981609c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 --2- 172.21.15.153:0/3190594558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198071040 0x7f6198071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 -- 172.21.15.153:0/3190594558 >> 172.21.15.153:0/3190594558 conn(0x7f61980d0000 msgr2=0x7f61980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 -- 172.21.15.153:0/3190594558 shutdown_connections 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.831+0000 7f619fc36700 1 -- 172.21.15.153:0/3190594558 wait complete. 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 Processor -- start 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 -- start start 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f619806dee0 0x7f61980b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6198071040 0x7f61980c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198160540 0x7f61980ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61981f2350 con 0x7f619806dee0 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61981f3600 con 0x7f6198071040 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619fc36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61981f3320 con 0x7f6198160540 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619d9d2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f619806dee0 0x7f61980b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619d9d2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f619806dee0 0x7f61980b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42900/0 (socket says 172.21.15.153:42900) 2023-11-28T13:05:15.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.833+0000 7f619d9d2700 1 -- 172.21.15.153:0/3607023249 learned_addr learned my addr 172.21.15.153:0/3607023249 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d1d1700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6198071040 0x7f61980c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d9d2700 1 -- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198160540 msgr2=0x7f61980ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d9d2700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198160540 0x7f61980ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d9d2700 1 -- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6198071040 msgr2=0x7f61980c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d9d2700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6198071040 0x7f61980c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d9d2700 1 -- 172.21.15.153:0/3607023249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6190009b30 con 0x7f619806dee0 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d1d1700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6198071040 0x7f61980c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619d9d2700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f619806dee0 0x7f61980b4110 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f619400d800 tx=0x7f619400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f618e7fc700 1 -- 172.21.15.153:0/3607023249 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940125f0 con 0x7f619806dee0 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f618e7fc700 1 -- 172.21.15.153:0/3607023249 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194012750 con 0x7f619806dee0 2023-11-28T13:05:15.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619815ccd0 con 0x7f619806dee0 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f618e7fc700 1 -- 172.21.15.153:0/3607023249 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194012b00 con 0x7f619806dee0 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.834+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619815d900 con 0x7f619806dee0 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.835+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619806a270 con 0x7f619806dee0 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.836+0000 7f618e7fc700 1 -- 172.21.15.153:0/3607023249 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6194012c60 con 0x7f619806dee0 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.836+0000 7f618e7fc700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f617c0682f0 0x7f617c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.836+0000 7f618e7fc700 1 -- 172.21.15.153:0/3607023249 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6194086ac0 con 0x7f619806dee0 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.836+0000 7f619d1d1700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f617c0682f0 0x7f617c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.837+0000 7f619d1d1700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f617c0682f0 0x7f617c06a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6188006fd0 tx=0x7f6188008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:15.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:15.840+0000 7f618e7fc700 1 -- 172.21.15.153:0/3607023249 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6194053330 con 0x7f619806dee0 2023-11-28T13:05:16.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.065+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f61980b3490 con 0x7f619806dee0 2023-11-28T13:05:16.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.065+0000 7f618e7fc700 1 -- 172.21.15.153:0/3607023249 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f61980b3490 con 0x7f619806dee0 2023-11-28T13:05:16.067 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:16.068 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:16.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f617c0682f0 msgr2=0x7f617c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:16.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f617c0682f0 0x7f617c06a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6188006fd0 tx=0x7f6188008040 comp rx=0 tx=0).stop 2023-11-28T13:05:16.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f619806dee0 msgr2=0x7f61980b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:16.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f619806dee0 0x7f61980b4110 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f619400d800 tx=0x7f619400dbc0 comp rx=0 tx=0).stop 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 shutdown_connections 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f617c0682f0 0x7f617c06a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f619806dee0 0x7f61980b4110 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6198160540 0x7f61980ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 --2- 172.21.15.153:0/3607023249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6198071040 0x7f61980c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 >> 172.21.15.153:0/3607023249 conn(0x7f61980d0000 msgr2=0x7f619815e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 shutdown_connections 2023-11-28T13:05:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:16.067+0000 7f619fc36700 1 -- 172.21.15.153:0/3607023249 wait complete. 2023-11-28T13:05:16.082 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:16.850 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:16.853 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:21.082 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:21.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.212+0000 7f96816b8700 1 Processor -- start 2023-11-28T13:05:21.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f96816b8700 1 -- start start 2023-11-28T13:05:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f96816b8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f967c158ce0 0x7f967c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f96816b8700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f967c0ce4b0 0x7f967c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f96816b8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c06bf10 con 0x7f967c06fb30 2023-11-28T13:05:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f96816b8700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967c15d730 con 0x7f967c0ce4b0 2023-11-28T13:05:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f96816b8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967c1590b0 con 0x7f967c158ce0 2023-11-28T13:05:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f967b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f967c0ce4b0 0x7f967c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f967b7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f967c0ce4b0 0x7f967c06c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42910/0 (socket says 172.21.15.153:42910) 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.213+0000 7f967b7fe700 1 -- 172.21.15.153:0/47215701 learned_addr learned my addr 172.21.15.153:0/47215701 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f967affd700 1 --2- 172.21.15.153:0/47215701 >> v2:172.21.15.177:3300/0 conn(0x7f967c158ce0 0x7f967c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f967b7fe700 1 -- 172.21.15.153:0/47215701 >> v2:172.21.15.177:3300/0 conn(0x7f967c158ce0 msgr2=0x7f967c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f967b7fe700 1 --2- 172.21.15.153:0/47215701 >> v2:172.21.15.177:3300/0 conn(0x7f967c158ce0 0x7f967c15d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f967b7fe700 1 -- 172.21.15.153:0/47215701 >> v1:172.21.15.177:6789/0 conn(0x7f967c06fb30 legacy=0x7f967c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f967b7fe700 1 -- 172.21.15.153:0/47215701 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0cb0d0 con 0x7f967c0ce4b0 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f967b7fe700 1 --2- 172.21.15.153:0/47215701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c0ce4b0 0x7f967c06c3a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f967c06fa30 tx=0x7f967400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f73320d49eb2b61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f9679ffb700 1 -- 172.21.15.153:0/47215701 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967400e070 con 0x7f967c0ce4b0 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f9679ffb700 1 -- 172.21.15.153:0/47215701 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674003a20 con 0x7f967c0ce4b0 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.214+0000 7f9679ffb700 1 -- 172.21.15.153:0/47215701 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674003dd0 con 0x7f967c0ce4b0 2023-11-28T13:05:21.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 -- 172.21.15.153:0/47215701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c0ce4b0 msgr2=0x7f967c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:21.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 --2- 172.21.15.153:0/47215701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c0ce4b0 0x7f967c06c3a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f967c06fa30 tx=0x7f967400b330 comp rx=0 tx=0).stop 2023-11-28T13:05:21.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 -- 172.21.15.153:0/47215701 shutdown_connections 2023-11-28T13:05:21.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 --2- 172.21.15.153:0/47215701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c0ce4b0 0x7f967c06c3a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 --2- 172.21.15.153:0/47215701 >> v2:172.21.15.177:3300/0 conn(0x7f967c158ce0 0x7f967c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 -- 172.21.15.153:0/47215701 >> 172.21.15.153:0/47215701 conn(0x7f967c0cb670 msgr2=0x7f967c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:21.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 -- 172.21.15.153:0/47215701 shutdown_connections 2023-11-28T13:05:21.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.215+0000 7f96816b8700 1 -- 172.21.15.153:0/47215701 wait complete. 2023-11-28T13:05:21.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.216+0000 7f96816b8700 1 Processor -- start 2023-11-28T13:05:21.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.216+0000 7f96816b8700 1 -- start start 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f96816b8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967c06fb30 0x7f967c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f96816b8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f967c0ce4b0 0x7f967c0adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f96816b8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c158ce0 0x7f967c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f96816b8700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0aca60 con 0x7f967c158ce0 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f96816b8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0acd00 con 0x7f967c06fb30 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f96816b8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0acfe0 con 0x7f967c0ce4b0 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c158ce0 0x7f967c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c158ce0 0x7f967c1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42916/0 (socket says 172.21.15.153:42916) 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 -- 172.21.15.153:0/357172603 learned_addr learned my addr 172.21.15.153:0/357172603 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967a7fc700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f967c0ce4b0 0x7f967c0adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967affd700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967c06fb30 0x7f967c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:21.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 -- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f967c0ce4b0 msgr2=0x7f967c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f967c0ce4b0 0x7f967c0adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 -- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967c06fb30 msgr2=0x7f967c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967c06fb30 0x7f967c15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 -- 172.21.15.153:0/357172603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674009b30 con 0x7f967c158ce0 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967a7fc700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f967c0ce4b0 0x7f967c0adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.217+0000 7f967b7fe700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c158ce0 0x7f967c1571a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9674015040 tx=0x7f9674004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.218+0000 7f96637fe700 1 -- 172.21.15.153:0/357172603 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674004440 con 0x7f967c158ce0 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.218+0000 7f96637fe700 1 -- 172.21.15.153:0/357172603 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96740045a0 con 0x7f967c158ce0 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.218+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c06b8c0 con 0x7f967c158ce0 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.218+0000 7f96637fe700 1 -- 172.21.15.153:0/357172603 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674004950 con 0x7f967c158ce0 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.218+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c0cb3b0 con 0x7f967c158ce0 2023-11-28T13:05:21.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.219+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967c06bba0 con 0x7f967c158ce0 2023-11-28T13:05:21.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.219+0000 7f96637fe700 1 -- 172.21.15.153:0/357172603 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9674022030 con 0x7f967c158ce0 2023-11-28T13:05:21.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.219+0000 7f96637fe700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f96680682f0 0x7f966806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:21.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.219+0000 7f96637fe700 1 -- 172.21.15.153:0/357172603 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f96740871e0 con 0x7f967c158ce0 2023-11-28T13:05:21.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.219+0000 7f967affd700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f96680682f0 0x7f966806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:21.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.220+0000 7f967affd700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f96680682f0 0x7f966806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f967000a950 tx=0x7f9670008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:21.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.224+0000 7f96637fe700 1 -- 172.21.15.153:0/357172603 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9674053930 con 0x7f967c158ce0 2023-11-28T13:05:21.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.446+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f967c0745c0 con 0x7f967c158ce0 2023-11-28T13:05:21.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.447+0000 7f96637fe700 1 -- 172.21.15.153:0/357172603 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9674007310 con 0x7f967c158ce0 2023-11-28T13:05:21.448 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:21.449 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f96680682f0 msgr2=0x7f966806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f96680682f0 0x7f966806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f967000a950 tx=0x7f9670008040 comp rx=0 tx=0).stop 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c158ce0 msgr2=0x7f967c1571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c158ce0 0x7f967c1571a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9674015040 tx=0x7f9674004ad0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 shutdown_connections 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f96680682f0 0x7f966806a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f967c158ce0 0x7f967c1571a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967c06fb30 0x7f967c15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 --2- 172.21.15.153:0/357172603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f967c0ce4b0 0x7f967c0adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 >> 172.21.15.153:0/357172603 conn(0x7f967c0cb670 msgr2=0x7f967c06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:21.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 shutdown_connections 2023-11-28T13:05:21.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:21.449+0000 7f96816b8700 1 -- 172.21.15.153:0/357172603 wait complete. 2023-11-28T13:05:21.462 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:26.463 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:26.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.596+0000 7f9c0fbee700 1 Processor -- start 2023-11-28T13:05:26.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.596+0000 7f9c0fbee700 1 -- start start 2023-11-28T13:05:26.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0fbee700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c0806dee0 0x7f9c0806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0fbee700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9c08160540 0x7f9c08160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0fbee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0815d060 con 0x7f9c08071040 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0fbee700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0815d340 con 0x7f9c0806dee0 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0fbee700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0815d620 con 0x7f9c08160540 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0d189700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c0806dee0 0x7f9c0806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0d189700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f9c0806dee0 0x7f9c0806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42920/0 (socket says 172.21.15.153:42920) 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.597+0000 7f9c0d189700 1 -- 172.21.15.153:0/2346494960 learned_addr learned my addr 172.21.15.153:0/2346494960 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0e18b700 1 --2- 172.21.15.153:0/2346494960 >> v2:172.21.15.177:3300/0 conn(0x7f9c08160540 0x7f9c08160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0d189700 1 -- 172.21.15.153:0/2346494960 >> v2:172.21.15.177:3300/0 conn(0x7f9c08160540 msgr2=0x7f9c08160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0d189700 1 --2- 172.21.15.153:0/2346494960 >> v2:172.21.15.177:3300/0 conn(0x7f9c08160540 0x7f9c08160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0d189700 1 -- 172.21.15.153:0/2346494960 >> v1:172.21.15.177:6789/0 conn(0x7f9c08071040 legacy=0x7f9c08071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0d189700 1 -- 172.21.15.153:0/2346494960 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0815d900 con 0x7f9c0806dee0 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0d189700 1 --2- 172.21.15.153:0/2346494960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0806e330 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9c080c3a10 tx=0x7f9bf8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5724c64202167510 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0c988700 1 -- 172.21.15.153:0/2346494960 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8004230 con 0x7f9c0806dee0 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0c988700 1 -- 172.21.15.153:0/2346494960 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf8004390 con 0x7f9c0806dee0 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.598+0000 7f9c0c988700 1 -- 172.21.15.153:0/2346494960 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80036a0 con 0x7f9c0806dee0 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 -- 172.21.15.153:0/2346494960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 msgr2=0x7f9c0806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/2346494960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0806e330 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9c080c3a10 tx=0x7f9bf8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 -- 172.21.15.153:0/2346494960 shutdown_connections 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/2346494960 >> v2:172.21.15.177:3300/0 conn(0x7f9c08160540 0x7f9c08160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/2346494960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0806e330 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 -- 172.21.15.153:0/2346494960 >> 172.21.15.153:0/2346494960 conn(0x7f9c080d0000 msgr2=0x7f9c080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 -- 172.21.15.153:0/2346494960 shutdown_connections 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.599+0000 7f9c0fbee700 1 -- 172.21.15.153:0/2346494960 wait complete. 2023-11-28T13:05:26.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.600+0000 7f9c0fbee700 1 Processor -- start 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.600+0000 7f9c0fbee700 1 -- start start 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c08071040 0x7f9c08144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c08160540 0x7f9c080c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08145220 con 0x7f9c0806dee0 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080b4f20 con 0x7f9c08160540 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0815bda0 con 0x7f9c08071040 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0815b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42926/0 (socket says 172.21.15.153:42926) 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 -- 172.21.15.153:0/1246268319 learned_addr learned my addr 172.21.15.153:0/1246268319 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:26.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 -- 172.21.15.153:0/1246268319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c08071040 msgr2=0x7f9c08144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c08071040 0x7f9c08144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 -- 172.21.15.153:0/1246268319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c08160540 msgr2=0x7f9c080c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c08160540 0x7f9c080c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 -- 172.21.15.153:0/1246268319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8009550 con 0x7f9c0806dee0 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0d98a700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0815b7b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c480 tx=0x7f9c0400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9bfe7fc700 1 -- 172.21.15.153:0/1246268319 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011790 con 0x7f9c0806dee0 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9bfe7fc700 1 -- 172.21.15.153:0/1246268319 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c04011910 con 0x7f9c0806dee0 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9bfe7fc700 1 -- 172.21.15.153:0/1246268319 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c04011cc0 con 0x7f9c0806dee0 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0815ccd0 con 0x7f9c0806dee0 2023-11-28T13:05:26.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.601+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0806e870 con 0x7f9c0806dee0 2023-11-28T13:05:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.602+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0806a270 con 0x7f9c0806dee0 2023-11-28T13:05:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.603+0000 7f9bfe7fc700 1 -- 172.21.15.153:0/1246268319 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c04010050 con 0x7f9c0806dee0 2023-11-28T13:05:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.603+0000 7f9bfe7fc700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9bec068320 0x7f9bec06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.603+0000 7f9bfe7fc700 1 -- 172.21.15.153:0/1246268319 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9c04085a50 con 0x7f9c0806dee0 2023-11-28T13:05:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.603+0000 7f9c0d189700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9bec068320 0x7f9bec06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.604+0000 7f9c0d189700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9bec068320 0x7f9bec06a7d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9c081f63e0 tx=0x7f9bf8010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.608+0000 7f9bfe7fc700 1 -- 172.21.15.153:0/1246268319 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c04052190 con 0x7f9c0806dee0 2023-11-28T13:05:26.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.834+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9c080720a0 con 0x7f9c0806dee0 2023-11-28T13:05:26.835 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.834+0000 7f9bfe7fc700 1 -- 172.21.15.153:0/1246268319 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9c040557b0 con 0x7f9c0806dee0 2023-11-28T13:05:26.835 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:26.835 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:26.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9bec068320 msgr2=0x7f9bec06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9bec068320 0x7f9bec06a7d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9c081f63e0 tx=0x7f9bf8010000 comp rx=0 tx=0).stop 2023-11-28T13:05:26.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 msgr2=0x7f9c0815b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0815b7b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400c480 tx=0x7f9c0400c840 comp rx=0 tx=0).stop 2023-11-28T13:05:26.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 shutdown_connections 2023-11-28T13:05:26.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9bec068320 0x7f9bec06a7d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c0806dee0 0x7f9c0815b7b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c08160540 0x7f9c080c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 --2- 172.21.15.153:0/1246268319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c08071040 0x7f9c08144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 >> 172.21.15.153:0/1246268319 conn(0x7f9c080d0000 msgr2=0x7f9c080aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:26.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 shutdown_connections 2023-11-28T13:05:26.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:26.836+0000 7f9c0fbee700 1 -- 172.21.15.153:0/1246268319 wait complete. 2023-11-28T13:05:26.849 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:31.850 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:31.981 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.981+0000 7f3d96de9700 1 Processor -- start 2023-11-28T13:05:31.982 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.981+0000 7f3d96de9700 1 -- start start 2023-11-28T13:05:31.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.981+0000 7f3d96de9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d90161340 0x7f3d900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.981+0000 7f3d96de9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3d900d2320 0x7f3d900cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.981+0000 7f3d96de9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900d1950 con 0x7f3d9015c9b0 2023-11-28T13:05:31.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.981+0000 7f3d96de9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900b5800 con 0x7f3d900d2320 2023-11-28T13:05:31.983 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.981+0000 7f3d96de9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900b5300 con 0x7f3d90161340 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d8ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d90161340 0x7f3d900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d94b85700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f3d9015c9b0 0x7f3d900b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:37712/0 (socket says 172.21.15.153:37712) 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d94b85700 1 -- 172.21.15.153:0/2364463902 learned_addr learned my addr 172.21.15.153:0/2364463902 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d95386700 1 --2- 172.21.15.153:0/2364463902 >> v2:172.21.15.177:3300/0 conn(0x7f3d900d2320 0x7f3d900cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d8ffff700 1 -- 172.21.15.153:0/2364463902 >> v2:172.21.15.177:3300/0 conn(0x7f3d900d2320 msgr2=0x7f3d900cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d8f7fe700 1 -- 172.21.15.153:0/2364463902 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957400453 0 0) 0x7f3d900d1950 con 0x7f3d9015c9b0 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d8ffff700 1 --2- 172.21.15.153:0/2364463902 >> v2:172.21.15.177:3300/0 conn(0x7f3d900d2320 0x7f3d900cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d8ffff700 1 -- 172.21.15.153:0/2364463902 >> v1:172.21.15.177:6789/0 conn(0x7f3d9015c9b0 legacy=0x7f3d900b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d8ffff700 1 -- 172.21.15.153:0/2364463902 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900b5520 con 0x7f3d90161340 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.982+0000 7f3d8ffff700 1 --2- 172.21.15.153:0/2364463902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d90161340 0x7f3d900d1de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009840 tx=0x7f3d7c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3aa6c1ae4987a69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:31.984 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d8f7fe700 1 -- 172.21.15.153:0/2364463902 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c011830 con 0x7f3d90161340 2023-11-28T13:05:31.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d8f7fe700 1 -- 172.21.15.153:0/2364463902 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c011990 con 0x7f3d90161340 2023-11-28T13:05:31.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d8f7fe700 1 -- 172.21.15.153:0/2364463902 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c011d20 con 0x7f3d90161340 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 -- 172.21.15.153:0/2364463902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d90161340 msgr2=0x7f3d900d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 --2- 172.21.15.153:0/2364463902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d90161340 0x7f3d900d1de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009840 tx=0x7f3d7c009b50 comp rx=0 tx=0).stop 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 -- 172.21.15.153:0/2364463902 shutdown_connections 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 --2- 172.21.15.153:0/2364463902 >> v2:172.21.15.177:3300/0 conn(0x7f3d900d2320 0x7f3d900cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 --2- 172.21.15.153:0/2364463902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d90161340 0x7f3d900d1de0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 -- 172.21.15.153:0/2364463902 >> 172.21.15.153:0/2364463902 conn(0x7f3d90072600 msgr2=0x7f3d900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 -- 172.21.15.153:0/2364463902 shutdown_connections 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.983+0000 7f3d96de9700 1 -- 172.21.15.153:0/2364463902 wait complete. 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.984+0000 7f3d96de9700 1 Processor -- start 2023-11-28T13:05:31.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.984+0000 7f3d96de9700 1 -- start start 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.985+0000 7f3d96de9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d900d2320 0x7f3d901502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.985+0000 7f3d96de9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d9015c9b0 0x7f3d9015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.985+0000 7f3d96de9700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d90161340 0x7f3d9015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.985+0000 7f3d96de9700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d901e7300 con 0x7f3d90161340 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.985+0000 7f3d96de9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d901507f0 con 0x7f3d9015c9b0 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.985+0000 7f3d96de9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d901f7240 con 0x7f3d900d2320 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.985+0000 7f3d95386700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d90161340 0x7f3d9015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d90161340 0x7f3d9015afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42944/0 (socket says 172.21.15.153:42944) 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 -- 172.21.15.153:0/3935767583 learned_addr learned my addr 172.21.15.153:0/3935767583 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d94b85700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d900d2320 0x7f3d901502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d8ffff700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d9015c9b0 0x7f3d9015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 -- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d900d2320 msgr2=0x7f3d901502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d900d2320 0x7f3d901502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 -- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d9015c9b0 msgr2=0x7f3d9015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d9015c9b0 0x7f3d9015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 -- 172.21.15.153:0/3935767583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0094a0 con 0x7f3d90161340 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d8ffff700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d9015c9b0 0x7f3d9015aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d95386700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d90161340 0x7f3d9015afa0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800efc0 tx=0x7f3d8800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d8d7fa700 1 -- 172.21.15.153:0/3935767583 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8800cb80 con 0x7f3d90161340 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d8d7fa700 1 -- 172.21.15.153:0/3935767583 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8800f040 con 0x7f3d90161340 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d900d15e0 con 0x7f3d90161340 2023-11-28T13:05:31.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d8d7fa700 1 -- 172.21.15.153:0/3935767583 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88014480 con 0x7f3d90161340 2023-11-28T13:05:31.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.986+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d900d1300 con 0x7f3d90161340 2023-11-28T13:05:31.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.987+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9015cd80 con 0x7f3d90161340 2023-11-28T13:05:31.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.988+0000 7f3d8d7fa700 1 -- 172.21.15.153:0/3935767583 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d880145e0 con 0x7f3d90161340 2023-11-28T13:05:31.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.988+0000 7f3d8d7fa700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d740682f0 0x7f3d7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.988+0000 7f3d8d7fa700 1 -- 172.21.15.153:0/3935767583 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3d88086670 con 0x7f3d90161340 2023-11-28T13:05:31.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.988+0000 7f3d94b85700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d740682f0 0x7f3d7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.992 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.989+0000 7f3d94b85700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d740682f0 0x7f3d7406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3d84007fd0 tx=0x7f3d84007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:31.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:31.992+0000 7f3d8d7fa700 1 -- 172.21.15.153:0/3935767583 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d88052e40 con 0x7f3d90161340 2023-11-28T13:05:32.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.217+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3d900745c0 con 0x7f3d90161340 2023-11-28T13:05:32.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.217+0000 7f3d8d7fa700 1 -- 172.21.15.153:0/3935767583 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3d88056460 con 0x7f3d90161340 2023-11-28T13:05:32.218 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:32.218 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d740682f0 msgr2=0x7f3d7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d740682f0 0x7f3d7406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3d84007fd0 tx=0x7f3d84007de0 comp rx=0 tx=0).stop 2023-11-28T13:05:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d90161340 msgr2=0x7f3d9015afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d90161340 0x7f3d9015afa0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800efc0 tx=0x7f3d8800c0c0 comp rx=0 tx=0).stop 2023-11-28T13:05:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 shutdown_connections 2023-11-28T13:05:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d740682f0 0x7f3d7406a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d90161340 0x7f3d9015afa0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d900d2320 0x7f3d901502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 --2- 172.21.15.153:0/3935767583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d9015c9b0 0x7f3d9015aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 >> 172.21.15.153:0/3935767583 conn(0x7f3d90072600 msgr2=0x7f3d9015eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 shutdown_connections 2023-11-28T13:05:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:32.219+0000 7f3d96de9700 1 -- 172.21.15.153:0/3935767583 wait complete. 2023-11-28T13:05:32.232 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:37.233 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.364+0000 7f80e76a2700 1 Processor -- start 2023-11-28T13:05:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.365+0000 7f80e76a2700 1 -- start start 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e76a2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f80e006f850 0x7f80e006fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e76a2700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e013ff00 con 0x7f80e0140950 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e76a2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e01404c0 con 0x7f80e00cd8c0 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e76a2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e0140dc0 con 0x7f80e006f850 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e4c3d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f80e0140950 0x7f80e00b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32880/0 (socket says 172.21.15.153:32880) 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e4c3d700 1 -- 172.21.15.153:0/1813961481 learned_addr learned my addr 172.21.15.153:0/1813961481 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e5c3f700 1 --2- 172.21.15.153:0/1813961481 >> v2:172.21.15.177:3300/0 conn(0x7f80e006f850 0x7f80e006fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215854179 0 0) 0x7f80e013ff00 con 0x7f80e0140950 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80cc0035b0 con 0x7f80e0140950 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908245344 0 0) 0x7f80cc0035b0 con 0x7f80e0140950 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 >> v2:172.21.15.177:3300/0 conn(0x7f80e006f850 msgr2=0x7f80e006fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 --2- 172.21.15.153:0/1813961481 >> v2:172.21.15.177:3300/0 conn(0x7f80e006f850 0x7f80e006fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 >> v1:172.21.15.177:6789/0 conn(0x7f80e00cd8c0 legacy=0x7f80e00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e015aa80 con 0x7f80e0140950 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f80d0001670 con 0x7f80e0140950 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 >> v1:172.21.15.153:6789/0 conn(0x7f80e0140950 legacy=0x7f80e00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80cc003a00 0x7f80cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80cc0043f0 0x7f80cc0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc005150 0x7f80cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e0140dc0 con 0x7f80cc0043f0 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc00c080 con 0x7f80cc003a00 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc00c240 con 0x7f80cc005150 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.366+0000 7f80e4c3d700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80cc0043f0 0x7f80cc0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e543e700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80cc003a00 0x7f80cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e5c3f700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc005150 0x7f80cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e4c3d700 1 -- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc005150 msgr2=0x7f80cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e4c3d700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc005150 0x7f80cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e4c3d700 1 -- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80cc003a00 msgr2=0x7f80cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e4c3d700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80cc003a00 0x7f80cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e5c3f700 1 -- 172.21.15.153:0/1813961481 reap_dead start 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e4c3d700 1 -- 172.21.15.153:0/1813961481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0035b0 con 0x7f80cc0043f0 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e543e700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80cc003a00 0x7f80cc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:05:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80e4c3d700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80cc0043f0 0x7f80cc0048b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f80e00cea30 tx=0x7f80cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309170a4f6afb3d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d000cd80 con 0x7f80cc0043f0 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0005960 con 0x7f80cc0043f0 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.367+0000 7f80d7fff700 1 -- 172.21.15.153:0/1813961481 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0005d10 con 0x7f80cc0043f0 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.368+0000 7f80e76a2700 1 -- 172.21.15.153:0/1813961481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80cc0043f0 msgr2=0x7f80cc0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.368+0000 7f80e76a2700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80cc0043f0 0x7f80cc0048b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f80e00cea30 tx=0x7f80cc005880 comp rx=0 tx=0).stop 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.368+0000 7f80e76a2700 1 -- 172.21.15.153:0/1813961481 shutdown_connections 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.368+0000 7f80e76a2700 1 --2- 172.21.15.153:0/1813961481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80cc0043f0 0x7f80cc0048b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.368+0000 7f80e76a2700 1 -- 172.21.15.153:0/1813961481 >> 172.21.15.153:0/1813961481 conn(0x7f80e006aed0 msgr2=0x7f80e006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.368+0000 7f80e76a2700 1 -- 172.21.15.153:0/1813961481 shutdown_connections 2023-11-28T13:05:37.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.368+0000 7f80e76a2700 1 -- 172.21.15.153:0/1813961481 wait complete. 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.369+0000 7f80e76a2700 1 Processor -- start 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.369+0000 7f80e76a2700 1 -- start start 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e76a2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc0043f0 0x7f80e00c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e76a2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80e00cf000 0x7f80e00c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e76a2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e0157930 0x7f80e00c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e76a2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00c99d0 con 0x7f80e0157930 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e76a2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00cf490 con 0x7f80e00cf000 2023-11-28T13:05:37.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e76a2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00c6450 con 0x7f80cc0043f0 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e0157930 0x7f80e00c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e0157930 0x7f80e00c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42962/0 (socket says 172.21.15.153:42962) 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 -- 172.21.15.153:0/4222607429 learned_addr learned my addr 172.21.15.153:0/4222607429 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e543e700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc0043f0 0x7f80e00c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e4c3d700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80e00cf000 0x7f80e00c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 -- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc0043f0 msgr2=0x7f80e00c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc0043f0 0x7f80e00c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 -- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80e00cf000 msgr2=0x7f80e00c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80e00cf000 0x7f80e00c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 -- 172.21.15.153:0/4222607429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00b35f0 con 0x7f80e0157930 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e5c3f700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e0157930 0x7f80e00c7f80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f80e015e150 tx=0x7f80d80111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80d5ffb700 1 -- 172.21.15.153:0/4222607429 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d800e070 con 0x7f80e0157930 2023-11-28T13:05:37.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80d5ffb700 1 -- 172.21.15.153:0/4222607429 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8003cf0 con 0x7f80e0157930 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80d5ffb700 1 -- 172.21.15.153:0/4222607429 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8003ea0 con 0x7f80e0157930 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.370+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e00b3310 con 0x7f80e0157930 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.371+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e0140150 con 0x7f80e0157930 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.372+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e006ffd0 con 0x7f80e0157930 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.372+0000 7f80d5ffb700 1 -- 172.21.15.153:0/4222607429 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f80d80117a0 con 0x7f80e0157930 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.372+0000 7f80d5ffb700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f80c40682f0 0x7f80c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.372+0000 7f80d5ffb700 1 -- 172.21.15.153:0/4222607429 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f80d8086340 con 0x7f80e0157930 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.372+0000 7f80e543e700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f80c40682f0 0x7f80c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.373+0000 7f80e543e700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f80c40682f0 0x7f80c406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007fd0 tx=0x7f80dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:37.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.377+0000 7f80d5ffb700 1 -- 172.21.15.153:0/4222607429 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80d8052ab0 con 0x7f80e0157930 2023-11-28T13:05:37.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.602+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f80e0068690 con 0x7f80e0157930 2023-11-28T13:05:37.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.602+0000 7f80d5ffb700 1 -- 172.21.15.153:0/4222607429 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f80d80560d0 con 0x7f80e0157930 2023-11-28T13:05:37.603 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:37.604 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:37.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f80c40682f0 msgr2=0x7f80c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f80c40682f0 0x7f80c406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007fd0 tx=0x7f80dc009040 comp rx=0 tx=0).stop 2023-11-28T13:05:37.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e0157930 msgr2=0x7f80e00c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e0157930 0x7f80e00c7f80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f80e015e150 tx=0x7f80d80111d0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 shutdown_connections 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f80c40682f0 0x7f80c406a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80e0157930 0x7f80e00c7f80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80cc0043f0 0x7f80e00c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 --2- 172.21.15.153:0/4222607429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80e00cf000 0x7f80e00c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 >> 172.21.15.153:0/4222607429 conn(0x7f80e006aed0 msgr2=0x7f80e00b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 shutdown_connections 2023-11-28T13:05:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:37.605+0000 7f80e76a2700 1 -- 172.21.15.153:0/4222607429 wait complete. 2023-11-28T13:05:37.618 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:42.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:42.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.751+0000 7f5913123700 1 Processor -- start 2023-11-28T13:05:42.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.751+0000 7f5913123700 1 -- start start 2023-11-28T13:05:42.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f5913123700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f590c0b3040 0x7f590c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f5913123700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c06b420 con 0x7f590c0d0a00 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f5913123700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c06b700 con 0x7f590c0ce040 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f5913123700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c06ff60 con 0x7f590c0b3040 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f59116c0700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f590c0b3040 0x7f590c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f590bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f590c0d0a00 0x7f590c0b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32900/0 (socket says 172.21.15.153:32900) 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f590bfff700 1 -- 172.21.15.153:0/1366235183 learned_addr learned my addr 172.21.15.153:0/1366235183 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f59116c0700 1 -- 172.21.15.153:0/1366235183 >> v1:172.21.15.177:6789/0 conn(0x7f590c0ce040 legacy=0x7f590c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f59116c0700 1 -- 172.21.15.153:0/1366235183 >> v1:172.21.15.153:6789/0 conn(0x7f590c0d0a00 legacy=0x7f590c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f59116c0700 1 -- 172.21.15.153:0/1366235183 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c06b9e0 con 0x7f590c0b3040 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.752+0000 7f590b7fe700 1 -- 172.21.15.153:0/1366235183 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485450899 0 0) 0x7f590c06b420 con 0x7f590c0d0a00 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f59116c0700 1 --2- 172.21.15.153:0/1366235183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0b3040 0x7f590c15df10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5904009d90 tx=0x7f590400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b388ee067764e6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f590b7fe700 1 -- 172.21.15.153:0/1366235183 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590400e070 con 0x7f590c0b3040 2023-11-28T13:05:42.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f590b7fe700 1 -- 172.21.15.153:0/1366235183 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590400bb90 con 0x7f590c0b3040 2023-11-28T13:05:42.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f590b7fe700 1 -- 172.21.15.153:0/1366235183 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904003a20 con 0x7f590c0b3040 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f5913123700 1 -- 172.21.15.153:0/1366235183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0b3040 msgr2=0x7f590c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f5913123700 1 --2- 172.21.15.153:0/1366235183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0b3040 0x7f590c15df10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5904009d90 tx=0x7f590400b330 comp rx=0 tx=0).stop 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f5913123700 1 -- 172.21.15.153:0/1366235183 shutdown_connections 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f5913123700 1 --2- 172.21.15.153:0/1366235183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0b3040 0x7f590c15df10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f5913123700 1 -- 172.21.15.153:0/1366235183 >> 172.21.15.153:0/1366235183 conn(0x7f590c06aed0 msgr2=0x7f590c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f5913123700 1 -- 172.21.15.153:0/1366235183 shutdown_connections 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.753+0000 7f5913123700 1 -- 172.21.15.153:0/1366235183 wait complete. 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.755+0000 7f5913123700 1 Processor -- start 2023-11-28T13:05:42.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.755+0000 7f5913123700 1 -- start start 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f590c0b3040 0x7f590c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590c0ce040 0x7f590c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0d0a00 0x7f590c0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c1543b0 con 0x7f590c0d0a00 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c154620 con 0x7f590c0ce040 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c154900 con 0x7f590c0b3040 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0d0a00 0x7f590c0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0d0a00 0x7f590c0ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42976/0 (socket says 172.21.15.153:42976) 2023-11-28T13:05:42.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 -- 172.21.15.153:0/3370373001 learned_addr learned my addr 172.21.15.153:0/3370373001 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5910ebf700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f590c0b3040 0x7f590c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f590bfff700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590c0ce040 0x7f590c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 -- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f590c0b3040 msgr2=0x7f590c151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f590c0b3040 0x7f590c151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 -- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590c0ce040 msgr2=0x7f590c155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590c0ce040 0x7f590c155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 -- 172.21.15.153:0/3370373001 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5904009a70 con 0x7f590c0d0a00 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59116c0700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0d0a00 0x7f590c0ca440 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f590c0b3aa0 tx=0x7f5904004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59097fa700 1 -- 172.21.15.153:0/3370373001 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904003a20 con 0x7f590c0d0a00 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59097fa700 1 -- 172.21.15.153:0/3370373001 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904012650 con 0x7f590c0d0a00 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f59097fa700 1 -- 172.21.15.153:0/3370373001 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904012a00 con 0x7f590c0d0a00 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c06bfa0 con 0x7f590c0d0a00 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.756+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c06bcc0 con 0x7f590c0d0a00 2023-11-28T13:05:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.757+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c06c4c0 con 0x7f590c0d0a00 2023-11-28T13:05:42.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.758+0000 7f59097fa700 1 -- 172.21.15.153:0/3370373001 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f590402c070 con 0x7f590c0d0a00 2023-11-28T13:05:42.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.758+0000 7f59097fa700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f58f00682f0 0x7f58f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.758+0000 7f5910ebf700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f58f00682f0 0x7f58f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.758+0000 7f59097fa700 1 -- 172.21.15.153:0/3370373001 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5904087db0 con 0x7f590c0d0a00 2023-11-28T13:05:42.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.759+0000 7f5910ebf700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f58f00682f0 0x7f58f006a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f590c155ca0 tx=0x7f5900009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:42.763 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.763+0000 7f59097fa700 1 -- 172.21.15.153:0/3370373001 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5904054580 con 0x7f590c0d0a00 2023-11-28T13:05:42.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.985+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f590c154e20 con 0x7f590c0d0a00 2023-11-28T13:05:42.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.985+0000 7f59097fa700 1 -- 172.21.15.153:0/3370373001 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f590402a0c0 con 0x7f590c0d0a00 2023-11-28T13:05:42.986 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:42.986 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:42.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.987+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f58f00682f0 msgr2=0x7f58f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.987+0000 7f5913123700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f58f00682f0 0x7f58f006a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f590c155ca0 tx=0x7f5900009040 comp rx=0 tx=0).stop 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.987+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0d0a00 msgr2=0x7f590c0ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.987+0000 7f5913123700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0d0a00 0x7f590c0ca440 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f590c0b3aa0 tx=0x7f5904004ad0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 shutdown_connections 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f58f00682f0 0x7f58f006a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f590c0d0a00 0x7f590c0ca440 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f590c0b3040 0x7f590c151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 --2- 172.21.15.153:0/3370373001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590c0ce040 0x7f590c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 >> 172.21.15.153:0/3370373001 conn(0x7f590c06aed0 msgr2=0x7f590c0d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 shutdown_connections 2023-11-28T13:05:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:42.988+0000 7f5913123700 1 -- 172.21.15.153:0/3370373001 wait complete. 2023-11-28T13:05:43.000 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:46.914 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:46.917 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:48.002 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:48.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.132+0000 7f610619b700 1 Processor -- start 2023-11-28T13:05:48.134 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f610619b700 1 -- start start 2023-11-28T13:05:48.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f610619b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6100071040 0x7f6100071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:48.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f610619b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f610006dee0 0x7f610006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:48.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f610619b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6100071950 con 0x7f610006cdc0 2023-11-28T13:05:48.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f610619b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6100161550 con 0x7f610006dee0 2023-11-28T13:05:48.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f610619b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610015d060 con 0x7f6100071040 2023-11-28T13:05:48.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f60feffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f610006dee0 0x7f610006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:48.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f60feffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f610006dee0 0x7f610006e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42982/0 (socket says 172.21.15.153:42982) 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.133+0000 7f60feffd700 1 -- 172.21.15.153:0/1136552180 learned_addr learned my addr 172.21.15.153:0/1136552180 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60ff7fe700 1 --2- 172.21.15.153:0/1136552180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6100071040 0x7f6100071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60feffd700 1 -- 172.21.15.153:0/1136552180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6100071040 msgr2=0x7f6100071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60feffd700 1 --2- 172.21.15.153:0/1136552180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6100071040 0x7f6100071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60feffd700 1 -- 172.21.15.153:0/1136552180 >> v1:172.21.15.177:6789/0 conn(0x7f610006cdc0 legacy=0x7f610006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60feffd700 1 -- 172.21.15.153:0/1136552180 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610015d340 con 0x7f610006dee0 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60feffd700 1 --2- 172.21.15.153:0/1136552180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f610006dee0 0x7f610006e350 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f60f00098f0 tx=0x7f60f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa4c320557c67c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60fe7fc700 1 -- 172.21.15.153:0/1136552180 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0004230 con 0x7f610006dee0 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60fe7fc700 1 -- 172.21.15.153:0/1136552180 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f0004390 con 0x7f610006dee0 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.134+0000 7f60fe7fc700 1 -- 172.21.15.153:0/1136552180 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f00036a0 con 0x7f610006dee0 2023-11-28T13:05:48.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 -- 172.21.15.153:0/1136552180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f610006dee0 msgr2=0x7f610006e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:48.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 --2- 172.21.15.153:0/1136552180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f610006dee0 0x7f610006e350 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f60f00098f0 tx=0x7f60f0009c00 comp rx=0 tx=0).stop 2023-11-28T13:05:48.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 -- 172.21.15.153:0/1136552180 shutdown_connections 2023-11-28T13:05:48.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 --2- 172.21.15.153:0/1136552180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6100071040 0x7f6100071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 --2- 172.21.15.153:0/1136552180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f610006dee0 0x7f610006e350 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 -- 172.21.15.153:0/1136552180 >> 172.21.15.153:0/1136552180 conn(0x7f61000d0000 msgr2=0x7f61000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:48.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 -- 172.21.15.153:0/1136552180 shutdown_connections 2023-11-28T13:05:48.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.135+0000 7f610619b700 1 -- 172.21.15.153:0/1136552180 wait complete. 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.136+0000 7f610619b700 1 Processor -- start 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.136+0000 7f610619b700 1 -- start start 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f610619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f610006cdc0 0x7f610014a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f610619b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f610006dee0 0x7f61000c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f610619b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6100071040 0x7f610015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f610619b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61001598a0 con 0x7f6100071040 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f610619b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6100155b00 con 0x7f610006cdc0 2023-11-28T13:05:48.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f610619b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6100155de0 con 0x7f610006dee0 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6100071040 0x7f610015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6100071040 0x7f610015b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42990/0 (socket says 172.21.15.153:42990) 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 -- 172.21.15.153:0/3541537179 learned_addr learned my addr 172.21.15.153:0/3541537179 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60ff7fe700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f610006cdc0 0x7f610014a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60feffd700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f610006dee0 0x7f61000c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 -- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f610006dee0 msgr2=0x7f61000c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f610006dee0 0x7f61000c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 -- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f610006cdc0 msgr2=0x7f610014a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f610006cdc0 0x7f610014a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 -- 172.21.15.153:0/3541537179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0009550 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60feffd700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f610006dee0 0x7f61000c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.137+0000 7f60fffff700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6100071040 0x7f610015b160 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b970 tx=0x7f60f800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.138+0000 7f60f7fff700 1 -- 172.21.15.153:0/3541537179 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f800c9b0 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.138+0000 7f60f7fff700 1 -- 172.21.15.153:0/3541537179 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f800cb10 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.138+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610015d900 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.138+0000 7f60f7fff700 1 -- 172.21.15.153:0/3541537179 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8014440 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.138+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610015d620 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.139+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610006e890 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.139+0000 7f60f7fff700 1 -- 172.21.15.153:0/3541537179 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f60f801b030 con 0x7f6100071040 2023-11-28T13:05:48.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.139+0000 7f60f7fff700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60e00682f0 0x7f60e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:48.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.139+0000 7f60f7fff700 1 -- 172.21.15.153:0/3541537179 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f60f80858c0 con 0x7f6100071040 2023-11-28T13:05:48.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.139+0000 7f60ff7fe700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60e00682f0 0x7f60e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:48.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.140+0000 7f60ff7fe700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60e00682f0 0x7f60e006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f60e800a950 tx=0x7f60e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:48.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.144+0000 7f60f7fff700 1 -- 172.21.15.153:0/3541537179 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f60f8052000 con 0x7f6100071040 2023-11-28T13:05:48.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.370+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6100156300 con 0x7f6100071040 2023-11-28T13:05:48.371 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.370+0000 7f60f7fff700 1 -- 172.21.15.153:0/3541537179 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f60f80190c0 con 0x7f6100071040 2023-11-28T13:05:48.371 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:48.371 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:48.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60e00682f0 msgr2=0x7f60e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:48.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60e00682f0 0x7f60e006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f60e800a950 tx=0x7f60e8008040 comp rx=0 tx=0).stop 2023-11-28T13:05:48.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6100071040 msgr2=0x7f610015b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:48.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6100071040 0x7f610015b160 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b970 tx=0x7f60f800bd30 comp rx=0 tx=0).stop 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 shutdown_connections 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f60e00682f0 0x7f60e006a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6100071040 0x7f610015b160 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f610006cdc0 0x7f610014a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 --2- 172.21.15.153:0/3541537179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f610006dee0 0x7f61000c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 >> 172.21.15.153:0/3541537179 conn(0x7f61000d0000 msgr2=0x7f61000cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.372+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 shutdown_connections 2023-11-28T13:05:48.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:48.373+0000 7f610619b700 1 -- 172.21.15.153:0/3541537179 wait complete. 2023-11-28T13:05:48.385 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:53.386 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:53.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.518+0000 7fe8c766d700 1 Processor -- start 2023-11-28T13:05:53.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.518+0000 7fe8c766d700 1 -- start start 2023-11-28T13:05:53.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c766d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe8c015cef0 0x7fe8c00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c766d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 0x7fe8c006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c766d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c00cc5f0 con 0x7fe8c006b8c0 2023-11-28T13:05:53.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c766d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c006bd40 con 0x7fe8c015cef0 2023-11-28T13:05:53.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c766d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c015d470 con 0x7fe8c00cbdf0 2023-11-28T13:05:53.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c5c0a700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe8c006b8c0 0x7fe8c0071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32928/0 (socket says 172.21.15.153:32928) 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c5c0a700 1 -- 172.21.15.153:0/2959108612 learned_addr learned my addr 172.21.15.153:0/2959108612 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c4c08700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 0x7fe8c006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8c5409700 1 --2- 172.21.15.153:0/2959108612 >> v2:172.21.15.177:3300/0 conn(0x7fe8c015cef0 0x7fe8c00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623499723 0 0) 0x7fe8c00cc5f0 con 0x7fe8c006b8c0 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8ac0035b0 con 0x7fe8c006b8c0 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933438838 0 0) 0x7fe8ac0035b0 con 0x7fe8c006b8c0 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 msgr2=0x7fe8c006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 0x7fe8c006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 >> v2:172.21.15.177:3300/0 conn(0x7fe8c015cef0 msgr2=0x7fe8c00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 --2- 172.21.15.153:0/2959108612 >> v2:172.21.15.177:3300/0 conn(0x7fe8c015cef0 0x7fe8c00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.519+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c015d750 con 0x7fe8c006b8c0 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fe8b80019f0 con 0x7fe8c006b8c0 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c4c08700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 0x7fe8c006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 >> v1:172.21.15.153:6789/0 conn(0x7fe8c006b8c0 legacy=0x7fe8c0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8ac003a00 0x7fe8ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 0x7fe8ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac004d90 0x7fe8ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c006bd40 con 0x7fe8ac003a00 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c015d470 con 0x7fe8ac004d90 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac00ba10 con 0x7fe8c00cbdf0 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 --2- 172.21.15.153:0/2959108612 >> v2:172.21.15.177:3300/0 conn(0x7fe8c015cef0 0x7fe8c00af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c4c08700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 0x7fe8ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8ac003a00 0x7fe8ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5c0a700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac004d90 0x7fe8ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 -- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 msgr2=0x7fe8ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c00cbdf0 0x7fe8ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 -- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac004d90 msgr2=0x7fe8ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac004d90 0x7fe8ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 -- 172.21.15.153:0/2959108612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00094a0 con 0x7fe8ac003a00 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8c5409700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8ac003a00 0x7fe8ac003e10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009d10 tx=0x7fe8ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31755f55dbb746e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc009e60 con 0x7fe8ac003a00 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc004030 con 0x7fe8ac003a00 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.520+0000 7fe8b7fff700 1 -- 172.21.15.153:0/2959108612 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0043a0 con 0x7fe8ac003a00 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.521+0000 7fe8c766d700 1 -- 172.21.15.153:0/2959108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8ac003a00 msgr2=0x7fe8ac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.521+0000 7fe8c766d700 1 --2- 172.21.15.153:0/2959108612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8ac003a00 0x7fe8ac003e10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009d10 tx=0x7fe8ac005880 comp rx=0 tx=0).stop 2023-11-28T13:05:53.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.521+0000 7fe8c5c0a700 1 -- 172.21.15.153:0/2959108612 reap_dead start 2023-11-28T13:05:53.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.521+0000 7fe8c766d700 1 -- 172.21.15.153:0/2959108612 shutdown_connections 2023-11-28T13:05:53.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.521+0000 7fe8c766d700 1 -- 172.21.15.153:0/2959108612 >> 172.21.15.153:0/2959108612 conn(0x7fe8c00d0000 msgr2=0x7fe8c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:53.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.521+0000 7fe8c766d700 1 -- 172.21.15.153:0/2959108612 shutdown_connections 2023-11-28T13:05:53.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.521+0000 7fe8c766d700 1 -- 172.21.15.153:0/2959108612 wait complete. 2023-11-28T13:05:53.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.522+0000 7fe8c766d700 1 Processor -- start 2023-11-28T13:05:53.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.522+0000 7fe8c766d700 1 -- start start 2023-11-28T13:05:53.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c766d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac003a00 0x7fe8c00b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c766d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c014f500 0x7fe8c00ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c766d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c009b3c0 0x7fe8c009b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c766d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c009bda0 con 0x7fe8c009b3c0 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c766d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00add10 con 0x7fe8ac003a00 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c766d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00aedd0 con 0x7fe8c014f500 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c009b3c0 0x7fe8c009b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c009b3c0 0x7fe8c009b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43010/0 (socket says 172.21.15.153:43010) 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 -- 172.21.15.153:0/1292534022 learned_addr learned my addr 172.21.15.153:0/1292534022 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5409700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac003a00 0x7fe8c00b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 -- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c014f500 msgr2=0x7fe8c00ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c4c08700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c014f500 0x7fe8c00ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c014f500 0x7fe8c00ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 -- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac003a00 msgr2=0x7fe8c00b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac003a00 0x7fe8c00b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5c0a700 1 -- 172.21.15.153:0/1292534022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c015dd10 con 0x7fe8c009b3c0 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.523+0000 7fe8c5409700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac003a00 0x7fe8c00b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.524+0000 7fe8c5c0a700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c009b3c0 0x7fe8c009b860 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe8c006fbb0 tx=0x7fe8b800a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.524+0000 7fe8b5ffb700 1 -- 172.21.15.153:0/1292534022 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800ae80 con 0x7fe8c009b3c0 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.524+0000 7fe8b5ffb700 1 -- 172.21.15.153:0/1292534022 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b80133f0 con 0x7fe8c009b3c0 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.524+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c015da30 con 0x7fe8c009b3c0 2023-11-28T13:05:53.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.524+0000 7fe8b5ffb700 1 -- 172.21.15.153:0/1292534022 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8013780 con 0x7fe8c009b3c0 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.524+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c015dff0 con 0x7fe8c009b3c0 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.525+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c00adea0 con 0x7fe8c009b3c0 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.525+0000 7fe8b5ffb700 1 -- 172.21.15.153:0/1292534022 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe8b80138e0 con 0x7fe8c009b3c0 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.525+0000 7fe8b5ffb700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe8a40682f0 0x7fe8a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.525+0000 7fe8b5ffb700 1 -- 172.21.15.153:0/1292534022 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe8b8086810 con 0x7fe8c009b3c0 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.525+0000 7fe8c5409700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe8a40682f0 0x7fe8a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.526+0000 7fe8c5409700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe8a40682f0 0x7fe8a406a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0090b0 tx=0x7fe8bc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:53.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.530+0000 7fe8b5ffb700 1 -- 172.21.15.153:0/1292534022 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe8b8052fe0 con 0x7fe8c009b3c0 2023-11-28T13:05:53.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.755+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe8c00b5690 con 0x7fe8c009b3c0 2023-11-28T13:05:53.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.756+0000 7fe8b5ffb700 1 -- 172.21.15.153:0/1292534022 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe8b8056600 con 0x7fe8c009b3c0 2023-11-28T13:05:53.757 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:53.757 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:53.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe8a40682f0 msgr2=0x7fe8a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe8a40682f0 0x7fe8a406a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0090b0 tx=0x7fe8bc008030 comp rx=0 tx=0).stop 2023-11-28T13:05:53.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c009b3c0 msgr2=0x7fe8c009b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c009b3c0 0x7fe8c009b860 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe8c006fbb0 tx=0x7fe8b800a640 comp rx=0 tx=0).stop 2023-11-28T13:05:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 shutdown_connections 2023-11-28T13:05:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe8a40682f0 0x7fe8a406a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8c009b3c0 0x7fe8c009b860 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ac003a00 0x7fe8c00b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 --2- 172.21.15.153:0/1292534022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8c014f500 0x7fe8c00ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 >> 172.21.15.153:0/1292534022 conn(0x7fe8c00d0000 msgr2=0x7fe8c006da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:53.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 shutdown_connections 2023-11-28T13:05:53.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:53.758+0000 7fe8c766d700 1 -- 172.21.15.153:0/1292534022 wait complete. 2023-11-28T13:05:53.771 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:58.772 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:58.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.903+0000 7f3aadfa5700 1 Processor -- start 2023-11-28T13:05:58.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.903+0000 7f3aadfa5700 1 -- start start 2023-11-28T13:05:58.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aadfa5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa8071040 0x7f3aa8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aadfa5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3aa8160540 0x7f3aa81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aadfa5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa8071950 con 0x7f3aa806dee0 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aadfa5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa815d060 con 0x7f3aa8160540 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aadfa5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa815d340 con 0x7f3aa8071040 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3aa8160540 0x7f3aa81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3aa8160540 0x7f3aa81609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43016/0 (socket says 172.21.15.153:43016) 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 -- 172.21.15.153:0/3412440477 learned_addr learned my addr 172.21.15.153:0/3412440477 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa77fe700 1 --2- 172.21.15.153:0/3412440477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa8071040 0x7f3aa8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 -- 172.21.15.153:0/3412440477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa8071040 msgr2=0x7f3aa8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 --2- 172.21.15.153:0/3412440477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa8071040 0x7f3aa8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 -- 172.21.15.153:0/3412440477 >> v1:172.21.15.177:6789/0 conn(0x7f3aa806dee0 legacy=0x7f3aa806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:58.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 -- 172.21.15.153:0/3412440477 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa815d620 con 0x7f3aa8160540 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.904+0000 7f3aa7fff700 1 --2- 172.21.15.153:0/3412440477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa81609c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009e50 tx=0x7f3aa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482b4b7b842554d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aa67fc700 1 -- 172.21.15.153:0/3412440477 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000e070 con 0x7f3aa8160540 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aa67fc700 1 -- 172.21.15.153:0/3412440477 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0003a20 con 0x7f3aa8160540 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aa67fc700 1 -- 172.21.15.153:0/3412440477 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0003dd0 con 0x7f3aa8160540 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 -- 172.21.15.153:0/3412440477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 msgr2=0x7f3aa81609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/3412440477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa81609c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009e50 tx=0x7f3aa000b330 comp rx=0 tx=0).stop 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 -- 172.21.15.153:0/3412440477 shutdown_connections 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/3412440477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa81609c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/3412440477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa8071040 0x7f3aa8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 -- 172.21.15.153:0/3412440477 >> 172.21.15.153:0/3412440477 conn(0x7f3aa80d0000 msgr2=0x7f3aa80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:58.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 -- 172.21.15.153:0/3412440477 shutdown_connections 2023-11-28T13:05:58.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.905+0000 7f3aadfa5700 1 -- 172.21.15.153:0/3412440477 wait complete. 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.906+0000 7f3aadfa5700 1 Processor -- start 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aadfa5700 1 -- start start 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aadfa5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa806dee0 0x7f3aa80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aadfa5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aa8071040 0x7f3aa80b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aadfa5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa814f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aadfa5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b54f0 con 0x7f3aa8160540 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aadfa5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa815be70 con 0x7f3aa8071040 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aadfa5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b4b20 con 0x7f3aa806dee0 2023-11-28T13:05:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aa7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa814f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aa7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa814f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43022/0 (socket says 172.21.15.153:43022) 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.907+0000 7f3aa7fff700 1 -- 172.21.15.153:0/1157288934 learned_addr learned my addr 172.21.15.153:0/1157288934 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa77fe700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa806dee0 0x7f3aa80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa6ffd700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aa8071040 0x7f3aa80b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa7fff700 1 -- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa806dee0 msgr2=0x7f3aa80b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa7fff700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa806dee0 0x7f3aa80b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa7fff700 1 -- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aa8071040 msgr2=0x7f3aa80b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa7fff700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aa8071040 0x7f3aa80b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa7fff700 1 -- 172.21.15.153:0/1157288934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa0009b30 con 0x7f3aa8160540 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aa7fff700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa814f2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806c7b0 tx=0x7f3aa0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3a8ffff700 1 -- 172.21.15.153:0/1157288934 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa00092e0 con 0x7f3aa8160540 2023-11-28T13:05:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3a8ffff700 1 -- 172.21.15.153:0/1157288934 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa0004240 con 0x7f3aa8160540 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa815ccd0 con 0x7f3aa8160540 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3a8ffff700 1 -- 172.21.15.153:0/1157288934 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0004610 con 0x7f3aa8160540 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.908+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa815d900 con 0x7f3aa8160540 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.909+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa806a270 con 0x7f3aa8160540 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.909+0000 7f3a8ffff700 1 -- 172.21.15.153:0/1157288934 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3aa002c070 con 0x7f3aa8160540 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.910+0000 7f3a8ffff700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a940682f0 0x7f3a9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.910+0000 7f3a8ffff700 1 -- 172.21.15.153:0/1157288934 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3aa0087000 con 0x7f3aa8160540 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.910+0000 7f3aa77fe700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a940682f0 0x7f3a9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.910+0000 7f3aa77fe700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a940682f0 0x7f3a9406a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006b20 tx=0x7f3a9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:58.915 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:58.914+0000 7f3a8ffff700 1 -- 172.21.15.153:0/1157288934 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3aa00537d0 con 0x7f3aa8160540 2023-11-28T13:05:59.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.141+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3aa809dea0 con 0x7f3aa8160540 2023-11-28T13:05:59.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.142+0000 7f3a8ffff700 1 -- 172.21.15.153:0/1157288934 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3aa00170c0 con 0x7f3aa8160540 2023-11-28T13:05:59.143 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:05:59.143 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:59.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a940682f0 msgr2=0x7f3a9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:59.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a940682f0 0x7f3a9406a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006b20 tx=0x7f3a9c008040 comp rx=0 tx=0).stop 2023-11-28T13:05:59.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 msgr2=0x7f3aa814f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:59.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa814f2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806c7b0 tx=0x7f3aa0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:05:59.145 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 shutdown_connections 2023-11-28T13:05:59.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3a940682f0 0x7f3a9406a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:59.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3aa8160540 0x7f3aa814f2f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:59.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3aa806dee0 0x7f3aa80b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:59.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 --2- 172.21.15.153:0/1157288934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aa8071040 0x7f3aa80b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:59.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 >> 172.21.15.153:0/1157288934 conn(0x7f3aa80d0000 msgr2=0x7f3aa806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:59.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 shutdown_connections 2023-11-28T13:05:59.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:05:59.144+0000 7f3aadfa5700 1 -- 172.21.15.153:0/1157288934 wait complete. 2023-11-28T13:05:59.157 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:04.158 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:04.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.288+0000 7ff3f8723700 1 Processor -- start 2023-11-28T13:06:04.290 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.288+0000 7ff3f8723700 1 -- start start 2023-11-28T13:06:04.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.289+0000 7ff3f8723700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff3f015cef0 0x7ff3f00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.289+0000 7ff3f8723700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff3f00cbdf0 0x7ff3f006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.293 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.289+0000 7ff3f8723700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f00cc5f0 con 0x7ff3f006b8c0 2023-11-28T13:06:04.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.289+0000 7ff3f8723700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f006bd40 con 0x7ff3f00cbdf0 2023-11-28T13:06:04.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.289+0000 7ff3f8723700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f015d470 con 0x7ff3f015cef0 2023-11-28T13:06:04.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f5cbe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff3f00cbdf0 0x7ff3f006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f6cc0700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7ff3f006b8c0 0x7ff3f0071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32960/0 (socket says 172.21.15.153:32960) 2023-11-28T13:06:04.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f6cc0700 1 -- 172.21.15.153:0/45981154 learned_addr learned my addr 172.21.15.153:0/45981154 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f64bf700 1 --2- 172.21.15.153:0/45981154 >> v2:172.21.15.177:3300/0 conn(0x7ff3f015cef0 0x7ff3f00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f5cbe700 1 -- 172.21.15.153:0/45981154 >> v2:172.21.15.177:3300/0 conn(0x7ff3f015cef0 msgr2=0x7ff3f00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f5cbe700 1 --2- 172.21.15.153:0/45981154 >> v2:172.21.15.177:3300/0 conn(0x7ff3f015cef0 0x7ff3f00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f5cbe700 1 -- 172.21.15.153:0/45981154 >> v1:172.21.15.153:6789/0 conn(0x7ff3f006b8c0 legacy=0x7ff3f0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f5cbe700 1 -- 172.21.15.153:0/45981154 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f015d750 con 0x7ff3f00cbdf0 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f54bd700 1 -- 172.21.15.153:0/45981154 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323346 0 0) 0x7ff3f00cc5f0 con 0x7ff3f006b8c0 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f5cbe700 1 --2- 172.21.15.153:0/45981154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f00cbdf0 0x7ff3f006b380 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009840 tx=0x7ff3ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6db529596ab2878 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f54bd700 1 -- 172.21.15.153:0/45981154 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec011830 con 0x7ff3f00cbdf0 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f54bd700 1 -- 172.21.15.153:0/45981154 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec011990 con 0x7ff3f00cbdf0 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.290+0000 7ff3f54bd700 1 -- 172.21.15.153:0/45981154 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec011d20 con 0x7ff3f00cbdf0 2023-11-28T13:06:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 -- 172.21.15.153:0/45981154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f00cbdf0 msgr2=0x7ff3f006b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 --2- 172.21.15.153:0/45981154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f00cbdf0 0x7ff3f006b380 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009840 tx=0x7ff3ec009b50 comp rx=0 tx=0).stop 2023-11-28T13:06:04.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 -- 172.21.15.153:0/45981154 shutdown_connections 2023-11-28T13:06:04.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 --2- 172.21.15.153:0/45981154 >> v2:172.21.15.177:3300/0 conn(0x7ff3f015cef0 0x7ff3f00af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 --2- 172.21.15.153:0/45981154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f00cbdf0 0x7ff3f006b380 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 -- 172.21.15.153:0/45981154 >> 172.21.15.153:0/45981154 conn(0x7ff3f00d0000 msgr2=0x7ff3f00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:04.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 -- 172.21.15.153:0/45981154 shutdown_connections 2023-11-28T13:06:04.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.291+0000 7ff3f8723700 1 -- 172.21.15.153:0/45981154 wait complete. 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.292+0000 7ff3f8723700 1 Processor -- start 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.292+0000 7ff3f8723700 1 -- start start 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f8723700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f006b8c0 0x7ff3f009ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f8723700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f00cbdf0 0x7ff3f0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f8723700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f015cef0 0x7ff3f0153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f8723700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f009ae90 con 0x7ff3f015cef0 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f8723700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f009a5f0 con 0x7ff3f006b8c0 2023-11-28T13:06:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f8723700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f009a8d0 con 0x7ff3f00cbdf0 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f015cef0 0x7ff3f0153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f015cef0 0x7ff3f0153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43034/0 (socket says 172.21.15.153:43034) 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 -- 172.21.15.153:0/2516547612 learned_addr learned my addr 172.21.15.153:0/2516547612 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f64bf700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f006b8c0 0x7ff3f009ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f5cbe700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f00cbdf0 0x7ff3f0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 -- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f00cbdf0 msgr2=0x7ff3f0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f00cbdf0 0x7ff3f0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 -- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f006b8c0 msgr2=0x7ff3f009ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f006b8c0 0x7ff3f009ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 -- 172.21.15.153:0/2516547612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0094a0 con 0x7ff3f015cef0 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f5cbe700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f00cbdf0 0x7ff3f0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f64bf700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f006b8c0 0x7ff3f009ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.293+0000 7ff3f6cc0700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f015cef0 0x7ff3f0153c80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800cb60 tx=0x7ff3e800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.294+0000 7ff3e6ffd700 1 -- 172.21.15.153:0/2516547612 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800ab50 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.294+0000 7ff3e6ffd700 1 -- 172.21.15.153:0/2516547612 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e800acb0 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.294+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f015dd10 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.294+0000 7ff3e6ffd700 1 -- 172.21.15.153:0/2516547612 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8007310 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.294+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f015da30 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.295+0000 7ff3e6ffd700 1 -- 172.21.15.153:0/2516547612 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff3e801a030 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.295+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f015dff0 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.295+0000 7ff3e6ffd700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3d40682f0 0x7ff3d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.295+0000 7ff3e6ffd700 1 -- 172.21.15.153:0/2516547612 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff3e80859d0 con 0x7ff3f015cef0 2023-11-28T13:06:04.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.295+0000 7ff3f64bf700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3d40682f0 0x7ff3d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.296+0000 7ff3f64bf700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3d40682f0 0x7ff3d406a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009a00 tx=0x7ff3e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:04.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.300+0000 7ff3e6ffd700 1 -- 172.21.15.153:0/2516547612 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff3e80521a0 con 0x7ff3f015cef0 2023-11-28T13:06:04.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.527+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff3f015ca60 con 0x7ff3f015cef0 2023-11-28T13:06:04.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.527+0000 7ff3e6ffd700 1 -- 172.21.15.153:0/2516547612 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff3e80180c0 con 0x7ff3f015cef0 2023-11-28T13:06:04.529 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:04.529 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3d40682f0 msgr2=0x7ff3d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3d40682f0 0x7ff3d406a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009a00 tx=0x7ff3e0008040 comp rx=0 tx=0).stop 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f015cef0 msgr2=0x7ff3f0153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f015cef0 0x7ff3f0153c80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800cb60 tx=0x7ff3e800cf20 comp rx=0 tx=0).stop 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 shutdown_connections 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff3d40682f0 0x7ff3d406a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3f015cef0 0x7ff3f0153c80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3f006b8c0 0x7ff3f009ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 --2- 172.21.15.153:0/2516547612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3f00cbdf0 0x7ff3f0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 >> 172.21.15.153:0/2516547612 conn(0x7ff3f00d0000 msgr2=0x7ff3f006e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:04.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 shutdown_connections 2023-11-28T13:06:04.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:04.530+0000 7ff3f8723700 1 -- 172.21.15.153:0/2516547612 wait complete. 2023-11-28T13:06:04.543 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:09.544 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.675+0000 7f1259fd1700 1 Processor -- start 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.675+0000 7f1259fd1700 1 -- start start 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1259fd1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1254160740 0x7f12540cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1259fd1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 0x7f125406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1259fd1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540cc030 con 0x7f1254160b10 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1259fd1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540ca720 con 0x7f1254160740 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1259fd1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540c9e10 con 0x7f125406b060 2023-11-28T13:06:09.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1252ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f1254160b10 0x7f12540cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32970/0 (socket says 172.21.15.153:32970) 2023-11-28T13:06:09.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1252ffd700 1 -- 172.21.15.153:0/1906965948 learned_addr learned my addr 172.21.15.153:0/1906965948 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:06:09.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f1253fff700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 0x7f125406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115486648 0 0) 0x7f12540cc030 con 0x7f1254160b10 2023-11-28T13:06:09.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f12537fe700 1 --2- 172.21.15.153:0/1906965948 >> v2:172.21.15.177:3300/0 conn(0x7f1254160740 0x7f12540cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.676+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f123c0035b0 con 0x7f1254160b10 2023-11-28T13:06:09.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862408969 0 0) 0x7f123c0035b0 con 0x7f1254160b10 2023-11-28T13:06:09.688 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 msgr2=0x7f125406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 0x7f125406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 >> v2:172.21.15.177:3300/0 conn(0x7f1254160740 msgr2=0x7f12540cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 --2- 172.21.15.153:0/1906965948 >> v2:172.21.15.177:3300/0 conn(0x7f1254160740 0x7f12540cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125406efc0 con 0x7f1254160b10 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f1244001670 con 0x7f1254160b10 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 >> v1:172.21.15.153:6789/0 conn(0x7f1254160b10 legacy=0x7f12540cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f123c003a00 0x7f123c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f123c004430 0x7f123c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 0x7f123c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12540c9e10 con 0x7f123c004430 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c005700 con 0x7f123c003a00 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c00ba90 con 0x7f125406b060 2023-11-28T13:06:09.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1252ffd700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f123c004430 0x7f123c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12537fe700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f123c003a00 0x7f123c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1253fff700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 0x7f123c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1252ffd700 1 -- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 msgr2=0x7f123c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1252ffd700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125406b060 0x7f123c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1252ffd700 1 -- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f123c003a00 msgr2=0x7f123c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1252ffd700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f123c003a00 0x7f123c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1252ffd700 1 -- 172.21.15.153:0/1906965948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0035b0 con 0x7f123c004430 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f1252ffd700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f123c004430 0x7f123c0048f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f124400c320 tx=0x7f123c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea516d8566cd8f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.677+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400ce60 con 0x7f123c004430 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244005960 con 0x7f123c004430 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f12527fc700 1 -- 172.21.15.153:0/1906965948 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244005cf0 con 0x7f123c004430 2023-11-28T13:06:09.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f1259fd1700 1 -- 172.21.15.153:0/1906965948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f123c004430 msgr2=0x7f123c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f1259fd1700 1 --2- 172.21.15.153:0/1906965948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f123c004430 0x7f123c0048f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f124400c320 tx=0x7f123c005880 comp rx=0 tx=0).stop 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f1253fff700 1 -- 172.21.15.153:0/1906965948 reap_dead start 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f1259fd1700 1 -- 172.21.15.153:0/1906965948 shutdown_connections 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f1259fd1700 1 -- 172.21.15.153:0/1906965948 >> 172.21.15.153:0/1906965948 conn(0x7f12540b12b0 msgr2=0x7f12540b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f1259fd1700 1 -- 172.21.15.153:0/1906965948 shutdown_connections 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.678+0000 7f1259fd1700 1 -- 172.21.15.153:0/1906965948 wait complete. 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.679+0000 7f1259fd1700 1 Processor -- start 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1259fd1700 1 -- start start 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1259fd1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f123c004430 0x7f1254153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1259fd1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125406e410 0x7f125406cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1259fd1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12541e8360 0x7f1254159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1259fd1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125406d140 con 0x7f12541e8360 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1259fd1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125406d370 con 0x7f125406e410 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1259fd1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125406d650 con 0x7f123c004430 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1253fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12541e8360 0x7f1254159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1253fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12541e8360 0x7f1254159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43054/0 (socket says 172.21.15.153:43054) 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1253fff700 1 -- 172.21.15.153:0/2141994160 learned_addr learned my addr 172.21.15.153:0/2141994160 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f1252ffd700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125406e410 0x7f125406cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.680+0000 7f12537fe700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f123c004430 0x7f1254153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1253fff700 1 -- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f123c004430 msgr2=0x7f1254153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1253fff700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f123c004430 0x7f1254153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1253fff700 1 -- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125406e410 msgr2=0x7f125406cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1253fff700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125406e410 0x7f125406cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1253fff700 1 -- 172.21.15.153:0/2141994160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12540ca3b0 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1253fff700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12541e8360 0x7f1254159060 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1254073c90 tx=0x7f124c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f123bfff700 1 -- 172.21.15.153:0/2141994160 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00e040 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f123bfff700 1 -- 172.21.15.153:0/2141994160 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c00f040 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12540ca0d0 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f123bfff700 1 -- 172.21.15.153:0/2141994160 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00fc40 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.681+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1254160f60 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.682+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1254066110 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.683+0000 7f123bfff700 1 -- 172.21.15.153:0/2141994160 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f124c00fda0 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.683+0000 7f123bfff700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12400682f0 0x7f124006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.683+0000 7f123bfff700 1 -- 172.21.15.153:0/2141994160 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f124c086520 con 0x7f12541e8360 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.683+0000 7f12537fe700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12400682f0 0x7f124006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.684+0000 7f12537fe700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12400682f0 0x7f124006a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1248009c60 tx=0x7f1248009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:09.694 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.687+0000 7f123bfff700 1 -- 172.21.15.153:0/2141994160 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f124c052cf0 con 0x7f12541e8360 2023-11-28T13:06:09.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.911+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f12540b5740 con 0x7f12541e8360 2023-11-28T13:06:09.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.912+0000 7f123bfff700 1 -- 172.21.15.153:0/2141994160 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f124c056310 con 0x7f12541e8360 2023-11-28T13:06:09.914 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:09.914 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12400682f0 msgr2=0x7f124006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12400682f0 0x7f124006a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1248009c60 tx=0x7f1248009210 comp rx=0 tx=0).stop 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12541e8360 msgr2=0x7f1254159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12541e8360 0x7f1254159060 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1254073c90 tx=0x7f124c011ed0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 shutdown_connections 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f12400682f0 0x7f124006a7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12541e8360 0x7f1254159060 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f123c004430 0x7f1254153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 --2- 172.21.15.153:0/2141994160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125406e410 0x7f125406cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 >> 172.21.15.153:0/2141994160 conn(0x7f12540b12b0 msgr2=0x7f125415bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:09.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 shutdown_connections 2023-11-28T13:06:09.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:09.914+0000 7f1259fd1700 1 -- 172.21.15.153:0/2141994160 wait complete. 2023-11-28T13:06:09.928 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:14.930 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:15.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.062+0000 7fda65003700 1 Processor -- start 2023-11-28T13:06:15.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.062+0000 7fda65003700 1 -- start start 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.063+0000 7fda65003700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fda60071040 0x7fda60071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.063+0000 7fda65003700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda60071950 con 0x7fda6006dee0 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.063+0000 7fda65003700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6015d060 con 0x7fda60160540 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.063+0000 7fda65003700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6015d340 con 0x7fda60071040 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.063+0000 7fda5ed9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fda60071040 0x7fda60071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.063+0000 7fda5e59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fda6006dee0 0x7fda6006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:32990/0 (socket says 172.21.15.153:32990) 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.063+0000 7fda5e59c700 1 -- 172.21.15.153:0/3869413916 learned_addr learned my addr 172.21.15.153:0/3869413916 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5ed9d700 1 -- 172.21.15.153:0/3869413916 >> v1:172.21.15.177:6789/0 conn(0x7fda60160540 legacy=0x7fda60160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:15.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5dd9b700 1 -- 172.21.15.153:0/3869413916 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756921325 0 0) 0x7fda60071950 con 0x7fda6006dee0 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5ed9d700 1 -- 172.21.15.153:0/3869413916 >> v1:172.21.15.153:6789/0 conn(0x7fda6006dee0 legacy=0x7fda6006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5ed9d700 1 -- 172.21.15.153:0/3869413916 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6015d620 con 0x7fda60071040 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5ed9d700 1 --2- 172.21.15.153:0/3869413916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda60071410 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fda50009840 tx=0x7fda50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd928affe980915b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5dd9b700 1 -- 172.21.15.153:0/3869413916 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50011830 con 0x7fda60071040 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5dd9b700 1 -- 172.21.15.153:0/3869413916 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda50011990 con 0x7fda60071040 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda5dd9b700 1 -- 172.21.15.153:0/3869413916 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50011d20 con 0x7fda60071040 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda65003700 1 -- 172.21.15.153:0/3869413916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 msgr2=0x7fda60071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.064+0000 7fda65003700 1 --2- 172.21.15.153:0/3869413916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda60071410 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fda50009840 tx=0x7fda50009b50 comp rx=0 tx=0).stop 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.065+0000 7fda65003700 1 -- 172.21.15.153:0/3869413916 shutdown_connections 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.065+0000 7fda65003700 1 --2- 172.21.15.153:0/3869413916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda60071410 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.065+0000 7fda65003700 1 -- 172.21.15.153:0/3869413916 >> 172.21.15.153:0/3869413916 conn(0x7fda600d0000 msgr2=0x7fda600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.065+0000 7fda65003700 1 -- 172.21.15.153:0/3869413916 shutdown_connections 2023-11-28T13:06:15.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.065+0000 7fda65003700 1 -- 172.21.15.153:0/3869413916 wait complete. 2023-11-28T13:06:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.066+0000 7fda65003700 1 Processor -- start 2023-11-28T13:06:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.066+0000 7fda65003700 1 -- start start 2023-11-28T13:06:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda65003700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6006dee0 0x7fda60159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda65003700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda600b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:15.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda65003700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda60160540 0x7fda6014b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda65003700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda600b3790 con 0x7fda60071040 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda65003700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda600b2e30 con 0x7fda60160540 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda65003700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda600c52c0 con 0x7fda6006dee0 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda600b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda600b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43064/0 (socket says 172.21.15.153:43064) 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 -- 172.21.15.153:0/3655345206 learned_addr learned my addr 172.21.15.153:0/3655345206 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5f59e700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda60160540 0x7fda6014b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5ed9d700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6006dee0 0x7fda60159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 -- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6006dee0 msgr2=0x7fda60159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6006dee0 0x7fda60159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 -- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda60160540 msgr2=0x7fda6014b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda60160540 0x7fda6014b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.067+0000 7fda5e59c700 1 -- 172.21.15.153:0/3655345206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500094a0 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.068+0000 7fda5e59c700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda600b5300 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fda5400c520 tx=0x7fda5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.068+0000 7fda477fe700 1 -- 172.21.15.153:0/3655345206 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54004c80 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.068+0000 7fda477fe700 1 -- 172.21.15.153:0/3655345206 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda54004e20 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.068+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6015ccd0 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.068+0000 7fda477fe700 1 -- 172.21.15.153:0/3655345206 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54007310 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.068+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6015d900 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.069+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6006a270 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.069+0000 7fda477fe700 1 -- 172.21.15.153:0/3655345206 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda5401a020 con 0x7fda60071040 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.069+0000 7fda477fe700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fda4c0682f0 0x7fda4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:15.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.070+0000 7fda477fe700 1 -- 172.21.15.153:0/3655345206 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fda540858e0 con 0x7fda60071040 2023-11-28T13:06:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.070+0000 7fda5ed9d700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fda4c0682f0 0x7fda4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:15.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.070+0000 7fda5ed9d700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fda4c0682f0 0x7fda4c06a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fda50009790 tx=0x7fda5001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:15.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.074+0000 7fda477fe700 1 -- 172.21.15.153:0/3655345206 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda540520b0 con 0x7fda60071040 2023-11-28T13:06:15.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.300+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fda600720a0 con 0x7fda60071040 2023-11-28T13:06:15.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.301+0000 7fda477fe700 1 -- 172.21.15.153:0/3655345206 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fda540556d0 con 0x7fda60071040 2023-11-28T13:06:15.303 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:15.303 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:15.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fda4c0682f0 msgr2=0x7fda4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fda4c0682f0 0x7fda4c06a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fda50009790 tx=0x7fda5001c040 comp rx=0 tx=0).stop 2023-11-28T13:06:15.305 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 msgr2=0x7fda600b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda600b5300 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fda5400c520 tx=0x7fda5400c8e0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 shutdown_connections 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda60160540 0x7fda6014b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6006dee0 0x7fda60159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fda4c0682f0 0x7fda4c06a7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 --2- 172.21.15.153:0/3655345206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda60071040 0x7fda600b5300 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 >> 172.21.15.153:0/3655345206 conn(0x7fda600d0000 msgr2=0x7fda6015e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 shutdown_connections 2023-11-28T13:06:15.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:15.303+0000 7fda65003700 1 -- 172.21.15.153:0/3655345206 wait complete. 2023-11-28T13:06:15.317 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:16.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:16.991 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:20.317 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:20.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.448+0000 7f22da49d700 1 Processor -- start 2023-11-28T13:06:20.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22da49d700 1 -- start start 2023-11-28T13:06:20.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22da49d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d40cd8c0 0x7f22d4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22da49d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f22d406f850 0x7f22d406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22da49d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d413ff00 con 0x7f22d40cdc90 2023-11-28T13:06:20.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22da49d700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d41404c0 con 0x7f22d406f850 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22da49d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d415aa80 con 0x7f22d40cd8c0 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22d8a3a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f22d406f850 0x7f22d406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22d37fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f22d40cdc90 0x7f22d40b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33004/0 (socket says 172.21.15.153:33004) 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.449+0000 7f22d37fe700 1 -- 172.21.15.153:0/1356953518 learned_addr learned my addr 172.21.15.153:0/1356953518 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d8a3a700 1 -- 172.21.15.153:0/1356953518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d40cd8c0 msgr2=0x7f22d4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d3fff700 1 --2- 172.21.15.153:0/1356953518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d40cd8c0 0x7f22d4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d8a3a700 1 --2- 172.21.15.153:0/1356953518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d40cd8c0 0x7f22d4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.459 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d2ffd700 1 -- 172.21.15.153:0/1356953518 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686746177 0 0) 0x7f22d413ff00 con 0x7f22d40cdc90 2023-11-28T13:06:20.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d8a3a700 1 -- 172.21.15.153:0/1356953518 >> v1:172.21.15.153:6789/0 conn(0x7f22d40cdc90 legacy=0x7f22d40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d8a3a700 1 -- 172.21.15.153:0/1356953518 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40b3040 con 0x7f22d406f850 2023-11-28T13:06:20.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d3fff700 1 --2- 172.21.15.153:0/1356953518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d40cd8c0 0x7f22d4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:20.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d8a3a700 1 --2- 172.21.15.153:0/1356953518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d406f850 0x7f22d406fcc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009d90 tx=0x7f22cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8791d1134e257f18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:20.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d2ffd700 1 -- 172.21.15.153:0/1356953518 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00e070 con 0x7f22d406f850 2023-11-28T13:06:20.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d2ffd700 1 -- 172.21.15.153:0/1356953518 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc00bbd0 con 0x7f22d406f850 2023-11-28T13:06:20.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.450+0000 7f22d2ffd700 1 -- 172.21.15.153:0/1356953518 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc012430 con 0x7f22d406f850 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 -- 172.21.15.153:0/1356953518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d406f850 msgr2=0x7f22d406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 --2- 172.21.15.153:0/1356953518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d406f850 0x7f22d406fcc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009d90 tx=0x7f22cc00b330 comp rx=0 tx=0).stop 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 -- 172.21.15.153:0/1356953518 shutdown_connections 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 --2- 172.21.15.153:0/1356953518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d40cd8c0 0x7f22d4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 --2- 172.21.15.153:0/1356953518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d406f850 0x7f22d406fcc0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 -- 172.21.15.153:0/1356953518 >> 172.21.15.153:0/1356953518 conn(0x7f22d406aed0 msgr2=0x7f22d406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 -- 172.21.15.153:0/1356953518 shutdown_connections 2023-11-28T13:06:20.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.451+0000 7f22da49d700 1 -- 172.21.15.153:0/1356953518 wait complete. 2023-11-28T13:06:20.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.452+0000 7f22da49d700 1 Processor -- start 2023-11-28T13:06:20.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.452+0000 7f22da49d700 1 -- start start 2023-11-28T13:06:20.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22da49d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d406f850 0x7f22d406b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22da49d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d40cd8c0 0x7f22d413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22da49d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22d40cdc90 0x7f22d413d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22da49d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d40cec80 con 0x7f22d40cd8c0 2023-11-28T13:06:20.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22da49d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406b9f0 con 0x7f22d40cdc90 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22da49d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406bcd0 con 0x7f22d406f850 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d40cd8c0 0x7f22d413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d40cd8c0 0x7f22d413ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43078/0 (socket says 172.21.15.153:43078) 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 -- 172.21.15.153:0/1371043521 learned_addr learned my addr 172.21.15.153:0/1371043521 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d3fff700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d406f850 0x7f22d406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d8a3a700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22d40cdc90 0x7f22d413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 -- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d406f850 msgr2=0x7f22d406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.463 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d406f850 0x7f22d406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 -- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22d40cdc90 msgr2=0x7f22d413d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22d40cdc90 0x7f22d413d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 -- 172.21.15.153:0/1371043521 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc009a70 con 0x7f22d40cd8c0 2023-11-28T13:06:20.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d3fff700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d406f850 0x7f22d406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:06:20.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.453+0000 7f22d37fe700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d40cd8c0 0x7f22d413ccf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c520 tx=0x7f22c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:20.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.454+0000 7f22d0ff9700 1 -- 172.21.15.153:0/1371043521 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8004c80 con 0x7f22d40cd8c0 2023-11-28T13:06:20.464 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.454+0000 7f22d0ff9700 1 -- 172.21.15.153:0/1371043521 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8004de0 con 0x7f22d40cd8c0 2023-11-28T13:06:20.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.454+0000 7f22d0ff9700 1 -- 172.21.15.153:0/1371043521 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80072d0 con 0x7f22d40cd8c0 2023-11-28T13:06:20.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.454+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d40b35f0 con 0x7f22d40cd8c0 2023-11-28T13:06:20.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.454+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d40b3310 con 0x7f22d40cd8c0 2023-11-28T13:06:20.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.455+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d4140150 con 0x7f22d40cd8c0 2023-11-28T13:06:20.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.455+0000 7f22d0ff9700 1 -- 172.21.15.153:0/1371043521 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f22c801a020 con 0x7f22d40cd8c0 2023-11-28T13:06:20.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.456+0000 7f22d0ff9700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22c00682f0 0x7f22c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.465 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.456+0000 7f22d0ff9700 1 -- 172.21.15.153:0/1371043521 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f22c8085960 con 0x7f22d40cd8c0 2023-11-28T13:06:20.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.456+0000 7f22d3fff700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22c00682f0 0x7f22c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.456+0000 7f22d3fff700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22c00682f0 0x7f22c006a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22c400aaf0 tx=0x7f22c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:20.466 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.460+0000 7f22d0ff9700 1 -- 172.21.15.153:0/1371043521 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f22c80521e0 con 0x7f22d40cd8c0 2023-11-28T13:06:20.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.685+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f22d4068690 con 0x7f22d40cd8c0 2023-11-28T13:06:20.687 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.685+0000 7f22d0ff9700 1 -- 172.21.15.153:0/1371043521 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f22c8055800 con 0x7f22d40cd8c0 2023-11-28T13:06:20.687 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:20.688 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:20.689 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22c00682f0 msgr2=0x7f22c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22c00682f0 0x7f22c006a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22c400aaf0 tx=0x7f22c4008040 comp rx=0 tx=0).stop 2023-11-28T13:06:20.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d40cd8c0 msgr2=0x7f22d413ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d40cd8c0 0x7f22d413ccf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c520 tx=0x7f22c800c8e0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.690 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 shutdown_connections 2023-11-28T13:06:20.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22d40cdc90 0x7f22d413d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22d406f850 0x7f22d406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.691 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f22c00682f0 0x7f22c006a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 --2- 172.21.15.153:0/1371043521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22d40cd8c0 0x7f22d413ccf0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 >> 172.21.15.153:0/1371043521 conn(0x7f22d406aed0 msgr2=0x7f22d413ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 shutdown_connections 2023-11-28T13:06:20.693 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:20.687+0000 7f22da49d700 1 -- 172.21.15.153:0/1371043521 wait complete. 2023-11-28T13:06:20.701 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:25.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:25.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.833+0000 7fa4aff42700 1 Processor -- start 2023-11-28T13:06:25.834 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.833+0000 7fa4aff42700 1 -- start start 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4aff42700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4a806ba90 0x7fa4a80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4aff42700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa4a80cbdf0 0x7fa4a80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4aff42700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a80cc5f0 con 0x7fa4a815cef0 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4aff42700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a815d470 con 0x7fa4a806ba90 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4aff42700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a815d750 con 0x7fa4a80cbdf0 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4ad4dd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4a806ba90 0x7fa4a80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4ad4dd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa4a806ba90 0x7fa4a80af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43084/0 (socket says 172.21.15.153:43084) 2023-11-28T13:06:25.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.834+0000 7fa4ad4dd700 1 -- 172.21.15.153:0/4126086612 learned_addr learned my addr 172.21.15.153:0/4126086612 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4ae4df700 1 --2- 172.21.15.153:0/4126086612 >> v2:172.21.15.177:3300/0 conn(0x7fa4a80cbdf0 0x7fa4a80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4ad4dd700 1 -- 172.21.15.153:0/4126086612 >> v2:172.21.15.177:3300/0 conn(0x7fa4a80cbdf0 msgr2=0x7fa4a80ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4ad4dd700 1 --2- 172.21.15.153:0/4126086612 >> v2:172.21.15.177:3300/0 conn(0x7fa4a80cbdf0 0x7fa4a80ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4ad4dd700 1 -- 172.21.15.153:0/4126086612 >> v1:172.21.15.177:6789/0 conn(0x7fa4a815cef0 legacy=0x7fa4a806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4ad4dd700 1 -- 172.21.15.153:0/4126086612 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a815da30 con 0x7fa4a806ba90 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4ad4dd700 1 --2- 172.21.15.153:0/4126086612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a806ba90 0x7fa4a80af400 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8073b10 tx=0x7fa4a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f06caa131c1a00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4accdc700 1 -- 172.21.15.153:0/4126086612 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400b870 con 0x7fa4a806ba90 2023-11-28T13:06:25.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4accdc700 1 -- 172.21.15.153:0/4126086612 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a400b9d0 con 0x7fa4a806ba90 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.835+0000 7fa4accdc700 1 -- 172.21.15.153:0/4126086612 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400bd60 con 0x7fa4a806ba90 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 -- 172.21.15.153:0/4126086612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a806ba90 msgr2=0x7fa4a80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 --2- 172.21.15.153:0/4126086612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a806ba90 0x7fa4a80af400 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8073b10 tx=0x7fa4a4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 -- 172.21.15.153:0/4126086612 shutdown_connections 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 --2- 172.21.15.153:0/4126086612 >> v2:172.21.15.177:3300/0 conn(0x7fa4a80cbdf0 0x7fa4a80ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 --2- 172.21.15.153:0/4126086612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a806ba90 0x7fa4a80af400 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 -- 172.21.15.153:0/4126086612 >> 172.21.15.153:0/4126086612 conn(0x7fa4a80d0000 msgr2=0x7fa4a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 -- 172.21.15.153:0/4126086612 shutdown_connections 2023-11-28T13:06:25.839 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.836+0000 7fa4aff42700 1 -- 172.21.15.153:0/4126086612 wait complete. 2023-11-28T13:06:25.842 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.837+0000 7fa4aff42700 1 Processor -- start 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.837+0000 7fa4aff42700 1 -- start start 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4aff42700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a806ba90 0x7fa4a81f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4aff42700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a80cbdf0 0x7fa4a815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4aff42700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a815cef0 0x7fa4a80b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4aff42700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a81e4b20 con 0x7fa4a815cef0 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4aff42700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a81f1b50 con 0x7fa4a806ba90 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4aff42700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a815b930 con 0x7fa4a80cbdf0 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a815cef0 0x7fa4a80b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a815cef0 0x7fa4a80b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43094/0 (socket says 172.21.15.153:43094) 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 -- 172.21.15.153:0/3335502770 learned_addr learned my addr 172.21.15.153:0/3335502770 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:25.843 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ad4dd700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a80cbdf0 0x7fa4a815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 -- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a80cbdf0 msgr2=0x7fa4a815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a80cbdf0 0x7fa4a815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 -- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a806ba90 msgr2=0x7fa4a81f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4adcde700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a806ba90 0x7fa4a81f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a806ba90 0x7fa4a81f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 -- 172.21.15.153:0/3335502770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4009550 con 0x7fa4a815cef0 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4adcde700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a806ba90 0x7fa4a81f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa4ae4df700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a815cef0 0x7fa4a80b5040 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0007e40 tx=0x7fa4a000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.838+0000 7fa49e7fc700 1 -- 172.21.15.153:0/3335502770 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000ee40 con 0x7fa4a815cef0 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.839+0000 7fa49e7fc700 1 -- 172.21.15.153:0/3335502770 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a0014420 con 0x7fa4a815cef0 2023-11-28T13:06:25.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.839+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a815dff0 con 0x7fa4a815cef0 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.839+0000 7fa49e7fc700 1 -- 172.21.15.153:0/3335502770 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00147d0 con 0x7fa4a815cef0 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.839+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a815dd10 con 0x7fa4a815cef0 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.839+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a80cab90 con 0x7fa4a815cef0 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.840+0000 7fa49e7fc700 1 -- 172.21.15.153:0/3335502770 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4a0012050 con 0x7fa4a815cef0 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.840+0000 7fa49e7fc700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa48c0682f0 0x7fa48c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.840+0000 7fa49e7fc700 1 -- 172.21.15.153:0/3335502770 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa4a00863d0 con 0x7fa4a815cef0 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.840+0000 7fa4adcde700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa48c0682f0 0x7fa48c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.841+0000 7fa4adcde700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa48c0682f0 0x7fa48c06a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa498000f60 tx=0x7fa498009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:25.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:25.844+0000 7fa49e7fc700 1 -- 172.21.15.153:0/3335502770 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4a0052ba0 con 0x7fa4a815cef0 2023-11-28T13:06:26.072 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.072+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa4a80ce820 con 0x7fa4a815cef0 2023-11-28T13:06:26.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.072+0000 7fa49e7fc700 1 -- 172.21.15.153:0/3335502770 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa4a00561c0 con 0x7fa4a815cef0 2023-11-28T13:06:26.073 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:26.073 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa48c0682f0 msgr2=0x7fa48c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa48c0682f0 0x7fa48c06a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa498000f60 tx=0x7fa498009040 comp rx=0 tx=0).stop 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a815cef0 msgr2=0x7fa4a80b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a815cef0 0x7fa4a80b5040 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0007e40 tx=0x7fa4a000e3c0 comp rx=0 tx=0).stop 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 shutdown_connections 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa48c0682f0 0x7fa48c06a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a815cef0 0x7fa4a80b5040 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a806ba90 0x7fa4a81f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:26.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 --2- 172.21.15.153:0/3335502770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a80cbdf0 0x7fa4a815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:26.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 >> 172.21.15.153:0/3335502770 conn(0x7fa4a80d0000 msgr2=0x7fa4a80aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:26.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 shutdown_connections 2023-11-28T13:06:26.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:26.074+0000 7fa4aff42700 1 -- 172.21.15.153:0/3335502770 wait complete. 2023-11-28T13:06:26.087 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:31.087 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:31.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.219+0000 7f2ecdb85700 1 Processor -- start 2023-11-28T13:06:31.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.219+0000 7f2ecdb85700 1 -- start start 2023-11-28T13:06:31.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ecdb85700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2ec806ba90 0x7f2ec80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ecdb85700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ec80cbdf0 0x7f2ec80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ecdb85700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec80cc5f0 con 0x7f2ec815cef0 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ecdb85700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec815d470 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ecdb85700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec815d750 con 0x7f2ec806ba90 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ec80cbdf0 0x7f2ec80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ec80cbdf0 0x7f2ec80ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43102/0 (socket says 172.21.15.153:43102) 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec7fff700 1 -- 172.21.15.153:0/3235008449 learned_addr learned my addr 172.21.15.153:0/3235008449 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec77fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2ec815cef0 0x7f2ec806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33030/0 (socket says 172.21.15.153:33030) 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec7fff700 1 -- 172.21.15.153:0/3235008449 >> v2:172.21.15.177:3300/0 conn(0x7f2ec806ba90 msgr2=0x7f2ec80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:31.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec6ffd700 1 --2- 172.21.15.153:0/3235008449 >> v2:172.21.15.177:3300/0 conn(0x7f2ec806ba90 0x7f2ec80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec7fff700 1 --2- 172.21.15.153:0/3235008449 >> v2:172.21.15.177:3300/0 conn(0x7f2ec806ba90 0x7f2ec80af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec7fff700 1 -- 172.21.15.153:0/3235008449 >> v1:172.21.15.153:6789/0 conn(0x7f2ec815cef0 legacy=0x7f2ec806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec67fc700 1 -- 172.21.15.153:0/3235008449 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921616289 0 0) 0x7f2ec80cc5f0 con 0x7f2ec815cef0 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.220+0000 7f2ec7fff700 1 -- 172.21.15.153:0/3235008449 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec815da30 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ec7fff700 1 --2- 172.21.15.153:0/3235008449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80ca650 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2ec806fbb0 tx=0x7f2ec000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e199a044d394084 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ec67fc700 1 -- 172.21.15.153:0/3235008449 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0010070 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ec67fc700 1 -- 172.21.15.153:0/3235008449 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec000dd90 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ec67fc700 1 -- 172.21.15.153:0/3235008449 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec000b310 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 -- 172.21.15.153:0/3235008449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 msgr2=0x7f2ec80ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/3235008449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80ca650 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2ec806fbb0 tx=0x7f2ec000d430 comp rx=0 tx=0).stop 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 -- 172.21.15.153:0/3235008449 shutdown_connections 2023-11-28T13:06:31.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/3235008449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80ca650 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/3235008449 >> v2:172.21.15.177:3300/0 conn(0x7f2ec806ba90 0x7f2ec80af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 -- 172.21.15.153:0/3235008449 >> 172.21.15.153:0/3235008449 conn(0x7f2ec80d0000 msgr2=0x7f2ec80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:31.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 -- 172.21.15.153:0/3235008449 shutdown_connections 2023-11-28T13:06:31.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.221+0000 7f2ecdb85700 1 -- 172.21.15.153:0/3235008449 wait complete. 2023-11-28T13:06:31.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.223+0000 7f2ecdb85700 1 Processor -- start 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.223+0000 7f2ecdb85700 1 -- start start 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.223+0000 7f2ecdb85700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec806ba90 0x7f2ec8151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.223+0000 7f2ecdb85700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.223+0000 7f2ecdb85700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec815cef0 0x7f2ec80c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.223+0000 7f2ecdb85700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80b2ec0 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.223+0000 7f2ecdb85700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80c6050 con 0x7f2ec806ba90 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ecdb85700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80c6330 con 0x7f2ec815cef0 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43106/0 (socket says 172.21.15.153:43106) 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 -- 172.21.15.153:0/2835998184 learned_addr learned my addr 172.21.15.153:0/2835998184 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec77fe700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec806ba90 0x7f2ec8151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec7fff700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec815cef0 0x7f2ec80c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 -- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec815cef0 msgr2=0x7f2ec80c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec815cef0 0x7f2ec80c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 -- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec806ba90 msgr2=0x7f2ec8151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec806ba90 0x7f2ec8151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 -- 172.21.15.153:0/2835998184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec000bd40 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec77fe700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec806ba90 0x7f2ec8151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec7fff700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec815cef0 0x7f2ec80c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ec6ffd700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80b54a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c7f0 tx=0x7f2ebc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2eaffff700 1 -- 172.21.15.153:0/2835998184 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011800 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2eaffff700 1 -- 172.21.15.153:0/2835998184 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc011960 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec815dff0 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2eaffff700 1 -- 172.21.15.153:0/2835998184 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011cf0 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.224+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec815dd10 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.225+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec80cab90 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.226+0000 7f2eaffff700 1 -- 172.21.15.153:0/2835998184 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ebc021070 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.226+0000 7f2eaffff700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb40682f0 0x7f2eb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.226+0000 7f2eaffff700 1 -- 172.21.15.153:0/2835998184 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2ebc085c50 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.226+0000 7f2ec77fe700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb40682f0 0x7f2eb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.226+0000 7f2ec77fe700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb40682f0 0x7f2eb406a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8000f60 tx=0x7f2eb8007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:31.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.231+0000 7f2eaffff700 1 -- 172.21.15.153:0/2835998184 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ebc0523a0 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.454+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2ec80b3000 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.454+0000 7f2eaffff700 1 -- 172.21.15.153:0/2835998184 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f2ebc01f080 con 0x7f2ec80cbdf0 2023-11-28T13:06:31.455 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:31.455 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb40682f0 msgr2=0x7f2eb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb40682f0 0x7f2eb406a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8000f60 tx=0x7f2eb8007d30 comp rx=0 tx=0).stop 2023-11-28T13:06:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 msgr2=0x7f2ec80b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80b54a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c7f0 tx=0x7f2ebc00cbb0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 shutdown_connections 2023-11-28T13:06:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ec815cef0 0x7f2ec80c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec806ba90 0x7f2ec8151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb40682f0 0x7f2eb406a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 --2- 172.21.15.153:0/2835998184 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ec80cbdf0 0x7f2ec80b54a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 >> 172.21.15.153:0/2835998184 conn(0x7f2ec80d0000 msgr2=0x7f2ec80caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 shutdown_connections 2023-11-28T13:06:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:31.456+0000 7f2ecdb85700 1 -- 172.21.15.153:0/2835998184 wait complete. 2023-11-28T13:06:31.469 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:36.470 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:36.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.603+0000 7ff0d07b2700 1 Processor -- start 2023-11-28T13:06:36.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.603+0000 7ff0d07b2700 1 -- start start 2023-11-28T13:06:36.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0d07b2700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0c80cd8c0 0x7ff0c8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0d07b2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c806f850 0x7ff0c806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0d07b2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c813ff00 con 0x7ff0c80cdc90 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0d07b2700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c81404c0 con 0x7ff0c80cd8c0 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0d07b2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c815aa80 con 0x7ff0c806f850 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0ce54e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0c80cd8c0 0x7ff0c8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0ce54e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff0c80cd8c0 0x7ff0c8140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43112/0 (socket says 172.21.15.153:43112) 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0ce54e700 1 -- 172.21.15.153:0/367997480 learned_addr learned my addr 172.21.15.153:0/367997480 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.604+0000 7ff0ced4f700 1 --2- 172.21.15.153:0/367997480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c806f850 0x7ff0c806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0ce54e700 1 -- 172.21.15.153:0/367997480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c806f850 msgr2=0x7ff0c806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0ce54e700 1 --2- 172.21.15.153:0/367997480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c806f850 0x7ff0c806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0ce54e700 1 -- 172.21.15.153:0/367997480 >> v1:172.21.15.177:6789/0 conn(0x7ff0c80cdc90 legacy=0x7ff0c80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0ce54e700 1 -- 172.21.15.153:0/367997480 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80b3040 con 0x7ff0c80cd8c0 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0ced4f700 1 --2- 172.21.15.153:0/367997480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c806f850 0x7ff0c806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0ce54e700 1 --2- 172.21.15.153:0/367997480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c80cd8c0 0x7ff0c8140950 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8006fd0 tx=0x7ff0b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2293b3df9689bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0cd54c700 1 -- 172.21.15.153:0/367997480 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8005e30 con 0x7ff0c80cd8c0 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0cd54c700 1 -- 172.21.15.153:0/367997480 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8004030 con 0x7ff0c80cd8c0 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0cd54c700 1 -- 172.21.15.153:0/367997480 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80043e0 con 0x7ff0c80cd8c0 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0d07b2700 1 -- 172.21.15.153:0/367997480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c80cd8c0 msgr2=0x7ff0c8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.605+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/367997480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c80cd8c0 0x7ff0c8140950 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8006fd0 tx=0x7ff0b8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.606+0000 7ff0d07b2700 1 -- 172.21.15.153:0/367997480 shutdown_connections 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.606+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/367997480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c806f850 0x7ff0c806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.606+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/367997480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c80cd8c0 0x7ff0c8140950 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.606+0000 7ff0d07b2700 1 -- 172.21.15.153:0/367997480 >> 172.21.15.153:0/367997480 conn(0x7ff0c806aed0 msgr2=0x7ff0c806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:36.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.606+0000 7ff0d07b2700 1 -- 172.21.15.153:0/367997480 shutdown_connections 2023-11-28T13:06:36.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.606+0000 7ff0d07b2700 1 -- 172.21.15.153:0/367997480 wait complete. 2023-11-28T13:06:36.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.607+0000 7ff0d07b2700 1 Processor -- start 2023-11-28T13:06:36.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.607+0000 7ff0d07b2700 1 -- start start 2023-11-28T13:06:36.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0d07b2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c806f850 0x7ff0c80c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0d07b2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c80cd8c0 0x7ff0c8158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0d07b2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c80cdc90 0x7ff0c80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0d07b2700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c8020 con 0x7ff0c806f850 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0d07b2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c8300 con 0x7ff0c80cdc90 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0d07b2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c85e0 con 0x7ff0c80cd8c0 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c806f850 0x7ff0c80c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c806f850 0x7ff0c80c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43116/0 (socket says 172.21.15.153:43116) 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 -- 172.21.15.153:0/280040050 learned_addr learned my addr 172.21.15.153:0/280040050 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ced4f700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c80cdc90 0x7ff0c80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0cdd4d700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c80cd8c0 0x7ff0c8158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 -- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c80cd8c0 msgr2=0x7ff0c8158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c80cd8c0 0x7ff0c8158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 -- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c80cdc90 msgr2=0x7ff0c80b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c80cdc90 0x7ff0c80b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 -- 172.21.15.153:0/280040050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8009500 con 0x7ff0c806f850 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0ce54e700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c806f850 0x7ff0c80c4c30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009c10 tx=0x7ff0b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0beffd700 1 -- 172.21.15.153:0/280040050 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8003c30 con 0x7ff0c806f850 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0beffd700 1 -- 172.21.15.153:0/280040050 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8003db0 con 0x7ff0c806f850 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80b35f0 con 0x7ff0c806f850 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.608+0000 7ff0beffd700 1 -- 172.21.15.153:0/280040050 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8011660 con 0x7ff0c806f850 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.609+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c80b3310 con 0x7ff0c806f850 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.609+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c8140150 con 0x7ff0c806f850 2023-11-28T13:06:36.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.610+0000 7ff0beffd700 1 -- 172.21.15.153:0/280040050 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff0b802c070 con 0x7ff0c806f850 2023-11-28T13:06:36.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.610+0000 7ff0beffd700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0ac0682f0 0x7ff0ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.610+0000 7ff0beffd700 1 -- 172.21.15.153:0/280040050 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff0b8086d10 con 0x7ff0c806f850 2023-11-28T13:06:36.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.610+0000 7ff0cdd4d700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0ac0682f0 0x7ff0ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.614 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.611+0000 7ff0cdd4d700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0ac0682f0 0x7ff0ac06a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8159a70 tx=0x7ff0c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:36.615 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.615+0000 7ff0beffd700 1 -- 172.21.15.153:0/280040050 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0b80534e0 con 0x7ff0c806f850 2023-11-28T13:06:36.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.837+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff0c80d1d60 con 0x7ff0c806f850 2023-11-28T13:06:36.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.838+0000 7ff0beffd700 1 -- 172.21.15.153:0/280040050 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff0b80160c0 con 0x7ff0c806f850 2023-11-28T13:06:36.839 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:36.839 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0ac0682f0 msgr2=0x7ff0ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0ac0682f0 0x7ff0ac06a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8159a70 tx=0x7ff0c4009040 comp rx=0 tx=0).stop 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c806f850 msgr2=0x7ff0c80c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c806f850 0x7ff0c80c4c30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009c10 tx=0x7ff0b8004750 comp rx=0 tx=0).stop 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 shutdown_connections 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff0ac0682f0 0x7ff0ac06a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff0c806f850 0x7ff0c80c4c30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c80cdc90 0x7ff0c80b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 --2- 172.21.15.153:0/280040050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0c80cd8c0 0x7ff0c8158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 >> 172.21.15.153:0/280040050 conn(0x7ff0c806aed0 msgr2=0x7ff0c806fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 shutdown_connections 2023-11-28T13:06:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:36.840+0000 7ff0d07b2700 1 -- 172.21.15.153:0/280040050 wait complete. 2023-11-28T13:06:36.852 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:41.854 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.986+0000 7f88ca195700 1 Processor -- start 2023-11-28T13:06:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.986+0000 7f88ca195700 1 -- start start 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.986+0000 7f88ca195700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.986+0000 7f88ca195700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f88c406dee0 0x7f88c406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88ca195700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c4071950 con 0x7f88c406cdc0 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88ca195700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c4161550 con 0x7f88c406dee0 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88ca195700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c415d060 con 0x7f88c4071040 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f88c406dee0 0x7f88c406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f88c406dee0 0x7f88c406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43126/0 (socket says 172.21.15.153:43126) 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 -- 172.21.15.153:0/4155607152 learned_addr learned my addr 172.21.15.153:0/4155607152 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c37fe700 1 --2- 172.21.15.153:0/4155607152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 -- 172.21.15.153:0/4155607152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 msgr2=0x7f88c4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 --2- 172.21.15.153:0/4155607152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 -- 172.21.15.153:0/4155607152 >> v1:172.21.15.177:6789/0 conn(0x7f88c406cdc0 legacy=0x7f88c406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 -- 172.21.15.153:0/4155607152 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c415d340 con 0x7f88c406dee0 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c2ffd700 1 --2- 172.21.15.153:0/4155607152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c406e350 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88b80098f0 tx=0x7f88b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fe7114b16a828f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c27fc700 1 -- 172.21.15.153:0/4155607152 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8011830 con 0x7f88c406dee0 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c27fc700 1 -- 172.21.15.153:0/4155607152 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b8011990 con 0x7f88c406dee0 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.987+0000 7f88c27fc700 1 -- 172.21.15.153:0/4155607152 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8011d20 con 0x7f88c406dee0 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 -- 172.21.15.153:0/4155607152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 msgr2=0x7f88c406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 --2- 172.21.15.153:0/4155607152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c406e350 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88b80098f0 tx=0x7f88b8009c00 comp rx=0 tx=0).stop 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 -- 172.21.15.153:0/4155607152 shutdown_connections 2023-11-28T13:06:41.990 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 --2- 172.21.15.153:0/4155607152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 --2- 172.21.15.153:0/4155607152 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c406e350 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 -- 172.21.15.153:0/4155607152 >> 172.21.15.153:0/4155607152 conn(0x7f88c40d0000 msgr2=0x7f88c40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:41.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 -- 172.21.15.153:0/4155607152 shutdown_connections 2023-11-28T13:06:41.991 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.988+0000 7f88ca195700 1 -- 172.21.15.153:0/4155607152 wait complete. 2023-11-28T13:06:41.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.989+0000 7f88ca195700 1 Processor -- start 2023-11-28T13:06:41.993 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.989+0000 7f88ca195700 1 -- start start 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88ca195700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88c406cdc0 0x7f88c41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88ca195700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c40c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88ca195700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c40b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88ca195700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c41e7c50 con 0x7f88c406dee0 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88ca195700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40afd80 con 0x7f88c406cdc0 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88ca195700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40affd0 con 0x7f88c4071040 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88c2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c40c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88c2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c40c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43132/0 (socket says 172.21.15.153:43132) 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88c2ffd700 1 -- 172.21.15.153:0/1436375767 learned_addr learned my addr 172.21.15.153:0/1436375767 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88c3fff700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c40b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88c37fe700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88c406cdc0 0x7f88c41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.994 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.990+0000 7f88c2ffd700 1 -- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 msgr2=0x7f88c40b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88c2ffd700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c40b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88c2ffd700 1 -- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88c406cdc0 msgr2=0x7f88c41543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88c2ffd700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88c406cdc0 0x7f88c41543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88c2ffd700 1 -- 172.21.15.153:0/1436375767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8009550 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88c2ffd700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c40c3aa0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009c40 tx=0x7f88b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88abfff700 1 -- 172.21.15.153:0/1436375767 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80036a0 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88abfff700 1 -- 172.21.15.153:0/1436375767 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b801e070 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88abfff700 1 -- 172.21.15.153:0/1436375767 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8003a40 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c415d900 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.991+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c415d620 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.992+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c406e890 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.992+0000 7f88abfff700 1 -- 172.21.15.153:0/1436375767 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88b800f050 con 0x7f88c406dee0 2023-11-28T13:06:41.995 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.993+0000 7f88abfff700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88b00682f0 0x7f88b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.993+0000 7f88abfff700 1 -- 172.21.15.153:0/1436375767 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f88b8086cc0 con 0x7f88c406dee0 2023-11-28T13:06:41.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.993+0000 7f88c37fe700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88b00682f0 0x7f88b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.996 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.993+0000 7f88c37fe700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88b00682f0 0x7f88b006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009ad0 tx=0x7f88b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:41.998 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:41.997+0000 7f88abfff700 1 -- 172.21.15.153:0/1436375767 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88b8053490 con 0x7f88c406dee0 2023-11-28T13:06:42.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.217+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f88c40720a0 con 0x7f88c406dee0 2023-11-28T13:06:42.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.218+0000 7f88abfff700 1 -- 172.21.15.153:0/1436375767 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f88b8020080 con 0x7f88c406dee0 2023-11-28T13:06:42.219 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:42.219 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:42.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88b00682f0 msgr2=0x7f88b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:42.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88b00682f0 0x7f88b006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009ad0 tx=0x7f88b4008040 comp rx=0 tx=0).stop 2023-11-28T13:06:42.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 msgr2=0x7f88c40c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:42.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c40c3aa0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009c40 tx=0x7f88b8004750 comp rx=0 tx=0).stop 2023-11-28T13:06:42.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 shutdown_connections 2023-11-28T13:06:42.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88c4071040 0x7f88c40b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88c406cdc0 0x7f88c41543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f88b00682f0 0x7f88b006a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 --2- 172.21.15.153:0/1436375767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f88c406dee0 0x7f88c40c3aa0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 >> 172.21.15.153:0/1436375767 conn(0x7f88c40d0000 msgr2=0x7f88c406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:42.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 shutdown_connections 2023-11-28T13:06:42.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:42.220+0000 7f88ca195700 1 -- 172.21.15.153:0/1436375767 wait complete. 2023-11-28T13:06:42.232 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:47.055 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:47.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:47.233 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:47.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.364+0000 7f1977c3a700 1 Processor -- start 2023-11-28T13:06:47.365 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.365+0000 7f1977c3a700 1 -- start start 2023-11-28T13:06:47.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.365+0000 7f1977c3a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.367 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.365+0000 7f1977c3a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19700d2320 0x7f19700cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.365+0000 7f1977c3a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19700d1950 con 0x7f197015c9b0 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.365+0000 7f1977c3a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19700b5800 con 0x7f19700d2320 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.365+0000 7f1977c3a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700b5300 con 0x7f1970161340 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19761d7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f19700d2320 0x7f19700cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19759d6700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f197015c9b0 0x7f19700b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33070/0 (socket says 172.21.15.153:33070) 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19759d6700 1 -- 172.21.15.153:0/1859672771 learned_addr learned my addr 172.21.15.153:0/1859672771 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19751d5700 1 --2- 172.21.15.153:0/1859672771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19761d7700 1 -- 172.21.15.153:0/1859672771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 msgr2=0x7f19700d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19761d7700 1 --2- 172.21.15.153:0/1859672771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19749d4700 1 -- 172.21.15.153:0/1859672771 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888699542 0 0) 0x7f19700d1950 con 0x7f197015c9b0 2023-11-28T13:06:47.368 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19761d7700 1 -- 172.21.15.153:0/1859672771 >> v1:172.21.15.153:6789/0 conn(0x7f197015c9b0 legacy=0x7f19700b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19761d7700 1 -- 172.21.15.153:0/1859672771 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700b5520 con 0x7f19700d2320 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19761d7700 1 --2- 172.21.15.153:0/1859672771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19700d2320 0x7f19700cd7c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1968009dc0 tx=0x7f196800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601cfa131c59ad53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19749d4700 1 -- 172.21.15.153:0/1859672771 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800e070 con 0x7f19700d2320 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19749d4700 1 -- 172.21.15.153:0/1859672771 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196800bae0 con 0x7f19700d2320 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.366+0000 7f19749d4700 1 -- 172.21.15.153:0/1859672771 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800be90 con 0x7f19700d2320 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 -- 172.21.15.153:0/1859672771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19700d2320 msgr2=0x7f19700cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1859672771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19700d2320 0x7f19700cd7c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1968009dc0 tx=0x7f196800b330 comp rx=0 tx=0).stop 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 -- 172.21.15.153:0/1859672771 shutdown_connections 2023-11-28T13:06:47.369 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1859672771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f19700d2320 0x7f19700cd7c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1859672771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 -- 172.21.15.153:0/1859672771 >> 172.21.15.153:0/1859672771 conn(0x7f1970072600 msgr2=0x7f19700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:47.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 -- 172.21.15.153:0/1859672771 shutdown_connections 2023-11-28T13:06:47.370 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.367+0000 7f1977c3a700 1 -- 172.21.15.153:0/1859672771 wait complete. 2023-11-28T13:06:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.368+0000 7f1977c3a700 1 Processor -- start 2023-11-28T13:06:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.368+0000 7f1977c3a700 1 -- start start 2023-11-28T13:06:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f1977c3a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19700d2320 0x7f19701474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f1977c3a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f197015c9b0 0x7f19700ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f1977c3a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f1977c3a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700c7f70 con 0x7f197015c9b0 2023-11-28T13:06:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f1977c3a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1970147a00 con 0x7f19700d2320 2023-11-28T13:06:47.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f1977c3a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700cd840 con 0x7f1970161340 2023-11-28T13:06:47.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f197015c9b0 0x7f19700ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f197015c9b0 0x7f19700ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43146/0 (socket says 172.21.15.153:43146) 2023-11-28T13:06:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 -- 172.21.15.153:0/1568330359 learned_addr learned my addr 172.21.15.153:0/1568330359 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19761d7700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 -- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 msgr2=0x7f19700b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 -- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19700d2320 msgr2=0x7f19701474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19759d6700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19700d2320 0x7f19701474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19700d2320 0x7f19701474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19751d5700 1 -- 172.21.15.153:0/1568330359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968009a70 con 0x7f197015c9b0 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.369+0000 7f19759d6700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19700d2320 0x7f19701474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.370+0000 7f19751d5700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f197015c9b0 0x7f19700ccd60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9b0 tx=0x7f196000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.370+0000 7f19667fc700 1 -- 172.21.15.153:0/1568330359 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960009710 con 0x7f197015c9b0 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.370+0000 7f19667fc700 1 -- 172.21.15.153:0/1568330359 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000c850 con 0x7f197015c9b0 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.370+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700d15e0 con 0x7f197015c9b0 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.370+0000 7f19667fc700 1 -- 172.21.15.153:0/1568330359 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000cbc0 con 0x7f197015c9b0 2023-11-28T13:06:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.370+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19700d1300 con 0x7f197015c9b0 2023-11-28T13:06:47.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.371+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197015cd80 con 0x7f197015c9b0 2023-11-28T13:06:47.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.371+0000 7f19667fc700 1 -- 172.21.15.153:0/1568330359 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1960023070 con 0x7f197015c9b0 2023-11-28T13:06:47.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.372+0000 7f19667fc700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19540682f0 0x7f195406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.372+0000 7f19667fc700 1 -- 172.21.15.153:0/1568330359 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1960086050 con 0x7f197015c9b0 2023-11-28T13:06:47.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.372+0000 7f19759d6700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19540682f0 0x7f195406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.372+0000 7f19759d6700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19540682f0 0x7f195406a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f196c007fd0 tx=0x7f196c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:47.380 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.376+0000 7f19667fc700 1 -- 172.21.15.153:0/1568330359 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1960052820 con 0x7f197015c9b0 2023-11-28T13:06:47.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.603+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f19700745c0 con 0x7f197015c9b0 2023-11-28T13:06:47.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.603+0000 7f19667fc700 1 -- 172.21.15.153:0/1568330359 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1960055e40 con 0x7f197015c9b0 2023-11-28T13:06:47.604 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:47.604 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19540682f0 msgr2=0x7f195406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19540682f0 0x7f195406a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f196c007fd0 tx=0x7f196c009040 comp rx=0 tx=0).stop 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f197015c9b0 msgr2=0x7f19700ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f197015c9b0 0x7f19700ccd60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f196000e9b0 tx=0x7f196000ed70 comp rx=0 tx=0).stop 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 shutdown_connections 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1970161340 0x7f19700b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19700d2320 0x7f19701474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f19540682f0 0x7f195406a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 --2- 172.21.15.153:0/1568330359 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f197015c9b0 0x7f19700ccd60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 >> 172.21.15.153:0/1568330359 conn(0x7f1970072600 msgr2=0x7f197006a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 shutdown_connections 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:47.605+0000 7f1977c3a700 1 -- 172.21.15.153:0/1568330359 wait complete. 2023-11-28T13:06:47.617 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:52.618 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:52.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe16d8d6700 1 Processor -- start 2023-11-28T13:06:52.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe16d8d6700 1 -- start start 2023-11-28T13:06:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe16d8d6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe168070c50 0x7fe1680b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe16d8d6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe1680b0bb0 0x7fe16815add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe16d8d6700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16815e1a0 con 0x7fe168158200 2023-11-28T13:06:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe16d8d6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16815f070 con 0x7fe168070c50 2023-11-28T13:06:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe16d8d6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1680b0fd0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe1667fc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fe168070c50 0x7fe1680b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe166ffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fe168158200 0x7fe1680caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33084/0 (socket says 172.21.15.153:33084) 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.750+0000 7fe166ffd700 1 -- 172.21.15.153:0/1662650677 learned_addr learned my addr 172.21.15.153:0/1662650677 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe1677fe700 1 --2- 172.21.15.153:0/1662650677 >> v2:172.21.15.177:3300/0 conn(0x7fe1680b0bb0 0x7fe16815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe1667fc700 1 -- 172.21.15.153:0/1662650677 >> v2:172.21.15.177:3300/0 conn(0x7fe1680b0bb0 msgr2=0x7fe16815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe1667fc700 1 --2- 172.21.15.153:0/1662650677 >> v2:172.21.15.177:3300/0 conn(0x7fe1680b0bb0 0x7fe16815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe1667fc700 1 -- 172.21.15.153:0/1662650677 >> v1:172.21.15.153:6789/0 conn(0x7fe168158200 legacy=0x7fe1680caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe165ffb700 1 -- 172.21.15.153:0/1662650677 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642183604 0 0) 0x7fe16815e1a0 con 0x7fe168158200 2023-11-28T13:06:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe1667fc700 1 -- 172.21.15.153:0/1662650677 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680cb180 con 0x7fe168070c50 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe1677fe700 1 --2- 172.21.15.153:0/1662650677 >> v2:172.21.15.177:3300/0 conn(0x7fe1680b0bb0 0x7fe16815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe1667fc700 1 --2- 172.21.15.153:0/1662650677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe168070c50 0x7fe1680b0670 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe16815ef70 tx=0x7fe158009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7972ca95d8db162e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe165ffb700 1 -- 172.21.15.153:0/1662650677 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158011830 con 0x7fe168070c50 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe165ffb700 1 -- 172.21.15.153:0/1662650677 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158011990 con 0x7fe168070c50 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.751+0000 7fe165ffb700 1 -- 172.21.15.153:0/1662650677 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158011d20 con 0x7fe168070c50 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1662650677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe168070c50 msgr2=0x7fe1680b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1662650677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe168070c50 0x7fe1680b0670 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe16815ef70 tx=0x7fe158009b20 comp rx=0 tx=0).stop 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1662650677 shutdown_connections 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1662650677 >> v2:172.21.15.177:3300/0 conn(0x7fe1680b0bb0 0x7fe16815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1662650677 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe168070c50 0x7fe1680b0670 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1662650677 >> 172.21.15.153:0/1662650677 conn(0x7fe1680cb670 msgr2=0x7fe1680cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:52.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1662650677 shutdown_connections 2023-11-28T13:06:52.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.752+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1662650677 wait complete. 2023-11-28T13:06:52.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.753+0000 7fe16d8d6700 1 Processor -- start 2023-11-28T13:06:52.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.753+0000 7fe16d8d6700 1 -- start start 2023-11-28T13:06:52.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe16d8d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe168070c50 0x7fe1680c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe16d8d6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1680b0bb0 0x7fe1680ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe16d8d6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe168158200 0x7fe1680caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe16d8d6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680b1a20 con 0x7fe1680b0bb0 2023-11-28T13:06:52.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe16d8d6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680b1d00 con 0x7fe168158200 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe16d8d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680b1fe0 con 0x7fe168070c50 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1680b0bb0 0x7fe1680ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1680b0bb0 0x7fe1680ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43160/0 (socket says 172.21.15.153:43160) 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 -- 172.21.15.153:0/1962586622 learned_addr learned my addr 172.21.15.153:0/1962586622 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe166ffd700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe168070c50 0x7fe1680c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1677fe700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe168158200 0x7fe1680caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 -- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe168070c50 msgr2=0x7fe1680c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe168070c50 0x7fe1680c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 -- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe168158200 msgr2=0x7fe1680caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe168158200 0x7fe1680caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 -- 172.21.15.153:0/1962586622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580094a0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1677fe700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe168158200 0x7fe1680caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe166ffd700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe168070c50 0x7fe1680c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.754+0000 7fe1667fc700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1680b0bb0 0x7fe1680ca4f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe158004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.755+0000 7fe14ffff700 1 -- 172.21.15.153:0/1962586622 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580038a0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.755+0000 7fe14ffff700 1 -- 172.21.15.153:0/1962586622 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15801f070 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.755+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1681585d0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.755+0000 7fe14ffff700 1 -- 172.21.15.153:0/1962586622 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158003c40 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.755+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1680710f0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.755+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16815e3f0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.756+0000 7fe14ffff700 1 -- 172.21.15.153:0/1962586622 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe15800f050 con 0x7fe1680b0bb0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.756+0000 7fe14ffff700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.756+0000 7fe14ffff700 1 -- 172.21.15.153:0/1962586622 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe158021070 con 0x7fe1680b0bb0 2023-11-28T13:06:52.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.756+0000 7fe166ffd700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.757+0000 7fe166ffd700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1540682f0 0x7fe15406a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000f60 tx=0x7fe15c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:52.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.761+0000 7fe14ffff700 1 -- 172.21.15.153:0/1962586622 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe158053fc0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.984+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe16806fd60 con 0x7fe1680b0bb0 2023-11-28T13:06:52.985 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.984+0000 7fe14ffff700 1 -- 172.21.15.153:0/1962586622 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe1580150c0 con 0x7fe1680b0bb0 2023-11-28T13:06:52.986 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:52.986 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:52.987 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1540682f0 msgr2=0x7fe15406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1540682f0 0x7fe15406a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000f60 tx=0x7fe15c009040 comp rx=0 tx=0).stop 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1680b0bb0 msgr2=0x7fe1680ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1680b0bb0 0x7fe1680ca4f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe158004750 comp rx=0 tx=0).stop 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 shutdown_connections 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe168158200 0x7fe1680caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe168070c50 0x7fe1680c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fe1540682f0 0x7fe15406a7a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 --2- 172.21.15.153:0/1962586622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1680b0bb0 0x7fe1680ca4f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.986+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 >> 172.21.15.153:0/1962586622 conn(0x7fe1680cb670 msgr2=0x7fe168158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.987+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 shutdown_connections 2023-11-28T13:06:52.988 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:52.987+0000 7fe16d8d6700 1 -- 172.21.15.153:0/1962586622 wait complete. 2023-11-28T13:06:52.999 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:58.000 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.130+0000 7fecb8599700 1 Processor -- start 2023-11-28T13:06:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.130+0000 7fecb8599700 1 -- start start 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.131+0000 7fecb8599700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb0140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.131+0000 7fecb8599700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fecb00cdc90 0x7fecb00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.131+0000 7fecb8599700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fecb00b2e70 0x7fecb006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.131+0000 7fecb8599700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb013ff00 con 0x7fecb00b2e70 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.131+0000 7fecb8599700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb01404c0 con 0x7fecb00cdc90 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.131+0000 7fecb8599700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb015aa80 con 0x7fecb00cd8c0 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.131+0000 7fecb6b36700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fecb00b2e70 0x7fecb006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fecb00b2e70 0x7fecb006f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43170/0 (socket says 172.21.15.153:43170) 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 -- 172.21.15.153:0/374896347 learned_addr learned my addr 172.21.15.153:0/374896347 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6335700 1 --2- 172.21.15.153:0/374896347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 -- 172.21.15.153:0/374896347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 msgr2=0x7fecb0140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 --2- 172.21.15.153:0/374896347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb0140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb5b34700 1 --2- 172.21.15.153:0/374896347 >> v2:172.21.15.177:3300/0 conn(0x7fecb00cdc90 0x7fecb00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 -- 172.21.15.153:0/374896347 >> v2:172.21.15.177:3300/0 conn(0x7fecb00cdc90 msgr2=0x7fecb00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 --2- 172.21.15.153:0/374896347 >> v2:172.21.15.177:3300/0 conn(0x7fecb00cdc90 0x7fecb00b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 -- 172.21.15.153:0/374896347 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00b3310 con 0x7fecb00b2e70 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb6b36700 1 --2- 172.21.15.153:0/374896347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00b2e70 0x7fecb006f850 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fecb0067c60 tx=0x7feca800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444dad197e096288 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:58.136 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb5333700 1 -- 172.21.15.153:0/374896347 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca800e070 con 0x7fecb00b2e70 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb5333700 1 -- 172.21.15.153:0/374896347 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca800b9c0 con 0x7fecb00b2e70 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.132+0000 7fecb5333700 1 -- 172.21.15.153:0/374896347 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca800bd70 con 0x7fecb00b2e70 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 -- 172.21.15.153:0/374896347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00b2e70 msgr2=0x7fecb006f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 --2- 172.21.15.153:0/374896347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00b2e70 0x7fecb006f850 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fecb0067c60 tx=0x7feca800b330 comp rx=0 tx=0).stop 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 -- 172.21.15.153:0/374896347 shutdown_connections 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 --2- 172.21.15.153:0/374896347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb0140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 --2- 172.21.15.153:0/374896347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00b2e70 0x7fecb006f850 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 --2- 172.21.15.153:0/374896347 >> v2:172.21.15.177:3300/0 conn(0x7fecb00cdc90 0x7fecb00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 -- 172.21.15.153:0/374896347 >> 172.21.15.153:0/374896347 conn(0x7fecb006aed0 msgr2=0x7fecb006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:58.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 -- 172.21.15.153:0/374896347 shutdown_connections 2023-11-28T13:06:58.138 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.133+0000 7fecb8599700 1 -- 172.21.15.153:0/374896347 wait complete. 2023-11-28T13:06:58.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.134+0000 7fecb8599700 1 Processor -- start 2023-11-28T13:06:58.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.134+0000 7fecb8599700 1 -- start start 2023-11-28T13:06:58.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb8599700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecb00b2e70 0x7fecb013ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.141 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb8599700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb013d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb8599700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00cdc90 0x7fecb00ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb8599700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cf0b0 con 0x7fecb00cdc90 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb8599700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cfc30 con 0x7fecb00b2e70 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb8599700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cf300 con 0x7fecb00cd8c0 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00cdc90 0x7fecb00ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00cdc90 0x7fecb00ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43176/0 (socket says 172.21.15.153:43176) 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 -- 172.21.15.153:0/3287892064 learned_addr learned my addr 172.21.15.153:0/3287892064 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb5b34700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6335700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecb00b2e70 0x7fecb013ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 -- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 msgr2=0x7fecb013d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.142 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb013d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 -- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecb00b2e70 msgr2=0x7fecb013ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecb00b2e70 0x7fecb013ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 -- 172.21.15.153:0/3287892064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca8009b30 con 0x7fecb00cdc90 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb5b34700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb013d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6335700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecb00b2e70 0x7fecb013ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.135+0000 7fecb6b36700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00cdc90 0x7fecb00ca2a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feca800b370 tx=0x7feca8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.136+0000 7feca6ffd700 1 -- 172.21.15.153:0/3287892064 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8004310 con 0x7fecb00cdc90 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.136+0000 7feca6ffd700 1 -- 172.21.15.153:0/3287892064 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca8004470 con 0x7fecb00cdc90 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.136+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb0140150 con 0x7fecb00cdc90 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.136+0000 7feca6ffd700 1 -- 172.21.15.153:0/3287892064 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8004820 con 0x7fecb00cdc90 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.136+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb00b35f0 con 0x7fecb00cdc90 2023-11-28T13:06:58.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.137+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb00ce100 con 0x7fecb00cdc90 2023-11-28T13:06:58.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.137+0000 7feca6ffd700 1 -- 172.21.15.153:0/3287892064 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feca801b430 con 0x7fecb00cdc90 2023-11-28T13:06:58.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.137+0000 7feca6ffd700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fec940682f0 0x7fec9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.137+0000 7feca6ffd700 1 -- 172.21.15.153:0/3287892064 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7feca80870f0 con 0x7fecb00cdc90 2023-11-28T13:06:58.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.137+0000 7fecb6335700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fec940682f0 0x7fec9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.138+0000 7fecb6335700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fec940682f0 0x7fec9406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fecb00d1600 tx=0x7feca0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:58.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.142+0000 7feca6ffd700 1 -- 172.21.15.153:0/3287892064 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feca8053910 con 0x7fecb00cdc90 2023-11-28T13:06:58.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.370+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fecb00d08c0 con 0x7fecb00cdc90 2023-11-28T13:06:58.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.370+0000 7feca6ffd700 1 -- 172.21.15.153:0/3287892064 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7feca8056f30 con 0x7fecb00cdc90 2023-11-28T13:06:58.372 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:06:58.372 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:58.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fec940682f0 msgr2=0x7fec9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.374 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fec940682f0 0x7fec9406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fecb00d1600 tx=0x7feca0008040 comp rx=0 tx=0).stop 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00cdc90 msgr2=0x7fecb00ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00cdc90 0x7fecb00ca2a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feca800b370 tx=0x7feca8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 shutdown_connections 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fec940682f0 0x7fec9406a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fecb00cdc90 0x7fecb00ca2a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecb00b2e70 0x7fecb013ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 --2- 172.21.15.153:0/3287892064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecb00cd8c0 0x7fecb013d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 >> 172.21.15.153:0/3287892064 conn(0x7fecb006aed0 msgr2=0x7fecb0158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 shutdown_connections 2023-11-28T13:06:58.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:06:58.372+0000 7fecb8599700 1 -- 172.21.15.153:0/3287892064 wait complete. 2023-11-28T13:06:58.386 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:03.387 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:03.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.517+0000 7f44aa584700 1 Processor -- start 2023-11-28T13:07:03.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.517+0000 7f44aa584700 1 -- start start 2023-11-28T13:07:03.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44aa584700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a406dee0 0x7f44a406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44aa584700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44a4160540 0x7f44a41609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44aa584700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a415d060 con 0x7f44a4071040 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44aa584700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a415d340 con 0x7f44a4160540 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44aa584700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a415d620 con 0x7f44a406dee0 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44a8b21700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44a4160540 0x7f44a41609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44a8b21700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f44a4160540 0x7f44a41609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43184/0 (socket says 172.21.15.153:43184) 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44a8b21700 1 -- 172.21.15.153:0/495469921 learned_addr learned my addr 172.21.15.153:0/495469921 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.518+0000 7f44a37fe700 1 --2- 172.21.15.153:0/495469921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a406dee0 0x7f44a406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a8b21700 1 -- 172.21.15.153:0/495469921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a406dee0 msgr2=0x7f44a406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a8b21700 1 --2- 172.21.15.153:0/495469921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a406dee0 0x7f44a406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a8b21700 1 -- 172.21.15.153:0/495469921 >> v1:172.21.15.177:6789/0 conn(0x7f44a4071040 legacy=0x7f44a4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a8b21700 1 -- 172.21.15.153:0/495469921 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a415d900 con 0x7f44a4160540 2023-11-28T13:07:03.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a8b21700 1 --2- 172.21.15.153:0/495469921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4160540 0x7f44a41609c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f449c009e50 tx=0x7f449c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cbea6e8fe8fa89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a2ffd700 1 -- 172.21.15.153:0/495469921 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00e070 con 0x7f44a4160540 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a2ffd700 1 -- 172.21.15.153:0/495469921 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c00b9c0 con 0x7f44a4160540 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44a2ffd700 1 -- 172.21.15.153:0/495469921 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00bd70 con 0x7f44a4160540 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44aa584700 1 -- 172.21.15.153:0/495469921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4160540 msgr2=0x7f44a41609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44aa584700 1 --2- 172.21.15.153:0/495469921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4160540 0x7f44a41609c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f449c009e50 tx=0x7f449c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44aa584700 1 -- 172.21.15.153:0/495469921 shutdown_connections 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44aa584700 1 --2- 172.21.15.153:0/495469921 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4160540 0x7f44a41609c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44aa584700 1 --2- 172.21.15.153:0/495469921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a406dee0 0x7f44a406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.519+0000 7f44aa584700 1 -- 172.21.15.153:0/495469921 >> 172.21.15.153:0/495469921 conn(0x7f44a40d0000 msgr2=0x7f44a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.520+0000 7f44aa584700 1 -- 172.21.15.153:0/495469921 shutdown_connections 2023-11-28T13:07:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.520+0000 7f44aa584700 1 -- 172.21.15.153:0/495469921 wait complete. 2023-11-28T13:07:03.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.521+0000 7f44aa584700 1 Processor -- start 2023-11-28T13:07:03.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.521+0000 7f44aa584700 1 -- start start 2023-11-28T13:07:03.526 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44aa584700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a406dee0 0x7f44a41f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44aa584700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4071040 0x7f44a40c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44aa584700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a4160540 0x7f44a40b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44aa584700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41f3510 con 0x7f44a4071040 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44aa584700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40c6320 con 0x7f44a406dee0 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44aa584700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40b3260 con 0x7f44a4160540 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4071040 0x7f44a40c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4071040 0x7f44a40c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43188/0 (socket says 172.21.15.153:43188) 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 -- 172.21.15.153:0/3734687613 learned_addr learned my addr 172.21.15.153:0/3734687613 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a8b21700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a4160540 0x7f44a40b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a3fff700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a406dee0 0x7f44a41f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 -- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a4160540 msgr2=0x7f44a40b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a4160540 0x7f44a40b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 -- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a406dee0 msgr2=0x7f44a41f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a406dee0 0x7f44a41f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 -- 172.21.15.153:0/3734687613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c009b30 con 0x7f44a4071040 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a8b21700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a4160540 0x7f44a40b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a37fe700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4071040 0x7f44a40c5100 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f449400d800 tx=0x7f449400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a0ff9700 1 -- 172.21.15.153:0/3734687613 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44940125f0 con 0x7f44a4071040 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a0ff9700 1 -- 172.21.15.153:0/3734687613 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4494012750 con 0x7f44a4071040 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a415ccd0 con 0x7f44a4071040 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.522+0000 7f44a0ff9700 1 -- 172.21.15.153:0/3734687613 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494012b00 con 0x7f44a4071040 2023-11-28T13:07:03.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.523+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a406e870 con 0x7f44a4071040 2023-11-28T13:07:03.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.523+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a406a270 con 0x7f44a4071040 2023-11-28T13:07:03.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.524+0000 7f44a0ff9700 1 -- 172.21.15.153:0/3734687613 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4494012c60 con 0x7f44a4071040 2023-11-28T13:07:03.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.524+0000 7f44a0ff9700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f44900682f0 0x7f449006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.524+0000 7f44a0ff9700 1 -- 172.21.15.153:0/3734687613 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4494086300 con 0x7f44a4071040 2023-11-28T13:07:03.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.524+0000 7f44a3fff700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f44900682f0 0x7f449006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.525+0000 7f44a3fff700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f44900682f0 0x7f449006a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4498000f60 tx=0x7f4498009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:03.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.528+0000 7f44a0ff9700 1 -- 172.21.15.153:0/3734687613 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4494052a50 con 0x7f44a4071040 2023-11-28T13:07:03.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.757+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f44a40720a0 con 0x7f44a4071040 2023-11-28T13:07:03.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.757+0000 7f44a0ff9700 1 -- 172.21.15.153:0/3734687613 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4494056070 con 0x7f44a4071040 2023-11-28T13:07:03.759 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:03.760 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:03.761 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.759+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f44900682f0 msgr2=0x7f449006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.759+0000 7f44aa584700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f44900682f0 0x7f449006a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4498000f60 tx=0x7f4498009040 comp rx=0 tx=0).stop 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.759+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4071040 msgr2=0x7f44a40c5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.759+0000 7f44aa584700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4071040 0x7f44a40c5100 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f449400d800 tx=0x7f449400dbc0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.759+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 shutdown_connections 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.760+0000 7f44aa584700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44a4160540 0x7f44a40b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.760+0000 7f44aa584700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44a406dee0 0x7f44a41f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.760+0000 7f44aa584700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f44900682f0 0x7f449006a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.760+0000 7f44aa584700 1 --2- 172.21.15.153:0/3734687613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44a4071040 0x7f44a40c5100 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.760+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 >> 172.21.15.153:0/3734687613 conn(0x7f44a40d0000 msgr2=0x7f44a415e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.760+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 shutdown_connections 2023-11-28T13:07:03.762 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:03.760+0000 7f44aa584700 1 -- 172.21.15.153:0/3734687613 wait complete. 2023-11-28T13:07:03.773 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:08.774 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:08.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.905+0000 7fa3ccf78700 1 Processor -- start 2023-11-28T13:07:08.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.905+0000 7fa3ccf78700 1 -- start start 2023-11-28T13:07:08.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3ccf78700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa3c8071040 0x7fa3c8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3ccf78700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c8071950 con 0x7fa3c806dee0 2023-11-28T13:07:08.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3ccf78700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c815d060 con 0x7fa3c8160540 2023-11-28T13:07:08.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3ccf78700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c815d340 con 0x7fa3c8071040 2023-11-28T13:07:08.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c5d9b700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa3c806dee0 0x7fa3c806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33128/0 (socket says 172.21.15.153:33128) 2023-11-28T13:07:08.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c5d9b700 1 -- 172.21.15.153:0/4050913550 learned_addr learned my addr 172.21.15.153:0/4050913550 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:08.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c659c700 1 --2- 172.21.15.153:0/4050913550 >> v2:172.21.15.177:3300/0 conn(0x7fa3c8071040 0x7fa3c8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097181649 0 0) 0x7fa3c8071950 con 0x7fa3c806dee0 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b00035b0 con 0x7fa3c806dee0 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492727588 0 0) 0x7fa3c815d060 con 0x7fa3c8160540 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c8071950 con 0x7fa3c8160540 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1331281108 0 0) 0x7fa3b00035b0 con 0x7fa3c806dee0 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 >> v2:172.21.15.177:3300/0 conn(0x7fa3c8071040 msgr2=0x7fa3c8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 --2- 172.21.15.153:0/4050913550 >> v2:172.21.15.177:3300/0 conn(0x7fa3c8071040 0x7fa3c8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 >> v1:172.21.15.177:6789/0 conn(0x7fa3c8160540 legacy=0x7fa3c8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c815d620 con 0x7fa3c806dee0 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.906+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fa3bc001670 con 0x7fa3c806dee0 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 >> v1:172.21.15.153:6789/0 conn(0x7fa3c806dee0 legacy=0x7fa3c806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3b0003be0 0x7fa3b0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b0004850 0x7fa3b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3b00052c0 0x7fa3b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c815d340 con 0x7fa3b00052c0 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b000c200 con 0x7fa3b0004850 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b000c3c0 con 0x7fa3b0003be0 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3b00052c0 0x7fa3b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c659c700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3b0003be0 0x7fa3b0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c5d9b700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b0004850 0x7fa3b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3b0003be0 msgr2=0x7fa3b0003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3b0003be0 0x7fa3b0003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b0004850 msgr2=0x7fa3b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b0004850 0x7fa3b0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/4050913550 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b00035b0 con 0x7fa3b00052c0 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/4050913550 reap_dead start 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c6d9d700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3b00052c0 0x7fa3b000ba10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c4c0 tx=0x7fa3b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724b6dfaa316b764 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0003a00 con 0x7fa3b00052c0 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c0003b80 con 0x7fa3b00052c0 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.907+0000 7fa3c559a700 1 -- 172.21.15.153:0/4050913550 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00050d0 con 0x7fa3b00052c0 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.908+0000 7fa3ccf78700 1 -- 172.21.15.153:0/4050913550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3b00052c0 msgr2=0x7fa3b000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.908+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3b00052c0 0x7fa3b000ba10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c4c0 tx=0x7fa3b0005880 comp rx=0 tx=0).stop 2023-11-28T13:07:08.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.908+0000 7fa3ccf78700 1 -- 172.21.15.153:0/4050913550 shutdown_connections 2023-11-28T13:07:08.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.908+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/4050913550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3b00052c0 0x7fa3b000ba10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.908+0000 7fa3ccf78700 1 -- 172.21.15.153:0/4050913550 >> 172.21.15.153:0/4050913550 conn(0x7fa3c80d0000 msgr2=0x7fa3c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:08.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.908+0000 7fa3ccf78700 1 -- 172.21.15.153:0/4050913550 shutdown_connections 2023-11-28T13:07:08.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.908+0000 7fa3ccf78700 1 -- 172.21.15.153:0/4050913550 wait complete. 2023-11-28T13:07:08.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.909+0000 7fa3ccf78700 1 Processor -- start 2023-11-28T13:07:08.916 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.909+0000 7fa3ccf78700 1 -- start start 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3ccf78700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b00052c0 0x7fa3c80b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3ccf78700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3c80cec10 0x7fa3c80cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3ccf78700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3c81485d0 0x7fa3c8148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3ccf78700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8145ed0 con 0x7fa3c81485d0 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3ccf78700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80cf5c0 con 0x7fa3b00052c0 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3ccf78700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80b5d90 con 0x7fa3c80cec10 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3c81485d0 0x7fa3c8148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3c81485d0 0x7fa3c8148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43210/0 (socket says 172.21.15.153:43210) 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/90818562 learned_addr learned my addr 172.21.15.153:0/90818562 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3c659c700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b00052c0 0x7fa3c80b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.910+0000 7fa3c5d9b700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3c80cec10 0x7fa3c80cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3c80cec10 msgr2=0x7fa3c80cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.917 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3c6d9d700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3c80cec10 0x7fa3c80cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b00052c0 msgr2=0x7fa3c80b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3c6d9d700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b00052c0 0x7fa3c80b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3c6d9d700 1 -- 172.21.15.153:0/90818562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c815ccd0 con 0x7fa3c81485d0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3c659c700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b00052c0 0x7fa3c80b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3c6d9d700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3c81485d0 0x7fa3c8148a50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8071d40 tx=0x7fa3c0001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3b6ffd700 1 -- 172.21.15.153:0/90818562 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0003a00 con 0x7fa3c81485d0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3b6ffd700 1 -- 172.21.15.153:0/90818562 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c0005710 con 0x7fa3c81485d0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c815d900 con 0x7fa3c81485d0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3b6ffd700 1 -- 172.21.15.153:0/90818562 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c001c680 con 0x7fa3c81485d0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.911+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c806a270 con 0x7fa3c81485d0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.912+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c80cfde0 con 0x7fa3c81485d0 2023-11-28T13:07:08.918 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.912+0000 7fa3b6ffd700 1 -- 172.21.15.153:0/90818562 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa3c0010050 con 0x7fa3c81485d0 2023-11-28T13:07:08.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.913+0000 7fa3b6ffd700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa3a80682f0 0x7fa3a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.913+0000 7fa3b6ffd700 1 -- 172.21.15.153:0/90818562 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa3c0087200 con 0x7fa3c81485d0 2023-11-28T13:07:08.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.913+0000 7fa3c659c700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa3a80682f0 0x7fa3a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.913+0000 7fa3c659c700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa3a80682f0 0x7fa3a806a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009be0 tx=0x7fa3b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:08.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:08.917+0000 7fa3b6ffd700 1 -- 172.21.15.153:0/90818562 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa3c0053970 con 0x7fa3c81485d0 2023-11-28T13:07:09.143 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.141+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa3c80720a0 con 0x7fa3c81485d0 2023-11-28T13:07:09.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.142+0000 7fa3b6ffd700 1 -- 172.21.15.153:0/90818562 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa3c0056f90 con 0x7fa3c81485d0 2023-11-28T13:07:09.144 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:09.144 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:09.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa3a80682f0 msgr2=0x7fa3a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:09.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa3a80682f0 0x7fa3a806a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009be0 tx=0x7fa3b8008040 comp rx=0 tx=0).stop 2023-11-28T13:07:09.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3c81485d0 msgr2=0x7fa3c8148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:09.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3c81485d0 0x7fa3c8148a50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8071d40 tx=0x7fa3c0001690 comp rx=0 tx=0).stop 2023-11-28T13:07:09.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 shutdown_connections 2023-11-28T13:07:09.146 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa3a80682f0 0x7fa3a806a7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa3c81485d0 0x7fa3c8148a50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3b00052c0 0x7fa3c80b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 --2- 172.21.15.153:0/90818562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3c80cec10 0x7fa3c80cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 >> 172.21.15.153:0/90818562 conn(0x7fa3c80d0000 msgr2=0x7fa3c815f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:09.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 shutdown_connections 2023-11-28T13:07:09.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:09.144+0000 7fa3ccf78700 1 -- 172.21.15.153:0/90818562 wait complete. 2023-11-28T13:07:09.157 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:14.158 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.289+0000 7fcf3c899700 1 Processor -- start 2023-11-28T13:07:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.289+0000 7fcf3c899700 1 -- start start 2023-11-28T13:07:14.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3c899700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf38161340 0x7fcf380d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3c899700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf380d1950 con 0x7fcf380d2320 2023-11-28T13:07:14.294 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3c899700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf380b5520 con 0x7fcf3815c9b0 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3c899700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf380b5800 con 0x7fcf38161340 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf36d9d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fcf380d2320 0x7fcf380cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33150/0 (socket says 172.21.15.153:33150) 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf36d9d700 1 -- 172.21.15.153:0/2460949006 learned_addr learned my addr 172.21.15.153:0/2460949006 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf35d9b700 1 --2- 172.21.15.153:0/2460949006 >> v2:172.21.15.177:3300/0 conn(0x7fcf38161340 0x7fcf380d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933353591 0 0) 0x7fcf380d1950 con 0x7fcf380d2320 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf200035b0 con 0x7fcf380d2320 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783840212 0 0) 0x7fcf200035b0 con 0x7fcf380d2320 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 >> v2:172.21.15.177:3300/0 conn(0x7fcf38161340 msgr2=0x7fcf380d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 --2- 172.21.15.153:0/2460949006 >> v2:172.21.15.177:3300/0 conn(0x7fcf38161340 0x7fcf380d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 >> v1:172.21.15.177:6789/0 conn(0x7fcf3815c9b0 legacy=0x7fcf380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.295 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf380d1300 con 0x7fcf380d2320 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.290+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fcf300019f0 con 0x7fcf380d2320 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 >> v1:172.21.15.153:6789/0 conn(0x7fcf380d2320 legacy=0x7fcf380cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf20003a00 0x7fcf20003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf20004850 0x7fcf20004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20005200 0x7fcf2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf380b5800 con 0x7fcf20003a00 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2000c110 con 0x7fcf20005200 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2000c2d0 con 0x7fcf20004850 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3659c700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf20003a00 0x7fcf20003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf35d9b700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf20004850 0x7fcf20004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf36d9d700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20005200 0x7fcf2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.296 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3659c700 1 -- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf20004850 msgr2=0x7fcf20004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3659c700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf20004850 0x7fcf20004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3659c700 1 -- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20005200 msgr2=0x7fcf2000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3659c700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20005200 0x7fcf2000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3659c700 1 -- 172.21.15.153:0/2460949006 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf200035b0 con 0x7fcf20003a00 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf36d9d700 1 -- 172.21.15.153:0/2460949006 reap_dead start 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3659c700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf20003a00 0x7fcf20003e10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcf280017c0 tx=0x7fcf20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6937afe9e91f4bd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280052d0 con 0x7fcf20003a00 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf28005450 con 0x7fcf20003a00 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.291+0000 7fcf3559a700 1 -- 172.21.15.153:0/2460949006 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280038c0 con 0x7fcf20003a00 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.292+0000 7fcf3c899700 1 -- 172.21.15.153:0/2460949006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf20003a00 msgr2=0x7fcf20003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.292+0000 7fcf3c899700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf20003a00 0x7fcf20003e10 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcf280017c0 tx=0x7fcf20005880 comp rx=0 tx=0).stop 2023-11-28T13:07:14.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.292+0000 7fcf3c899700 1 -- 172.21.15.153:0/2460949006 shutdown_connections 2023-11-28T13:07:14.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.292+0000 7fcf3c899700 1 --2- 172.21.15.153:0/2460949006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf20003a00 0x7fcf20003e10 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.292+0000 7fcf3c899700 1 -- 172.21.15.153:0/2460949006 >> 172.21.15.153:0/2460949006 conn(0x7fcf38072600 msgr2=0x7fcf380cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:14.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.292+0000 7fcf3c899700 1 -- 172.21.15.153:0/2460949006 shutdown_connections 2023-11-28T13:07:14.298 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.292+0000 7fcf3c899700 1 -- 172.21.15.153:0/2460949006 wait complete. 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.293+0000 7fcf3c899700 1 Processor -- start 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.293+0000 7fcf3c899700 1 -- start start 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf3c899700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20003a00 0x7fcf3815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf3c899700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf381f6690 0x7fcf381f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf3c899700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf381f2180 0x7fcf38156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf3c899700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381f26c0 con 0x7fcf381f6690 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf3c899700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381f29a0 con 0x7fcf20003a00 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf3c899700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381f2c80 con 0x7fcf381f2180 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf35d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf381f6690 0x7fcf381f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf35d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf381f6690 0x7fcf381f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43228/0 (socket says 172.21.15.153:43228) 2023-11-28T13:07:14.301 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf35d9b700 1 -- 172.21.15.153:0/3737785797 learned_addr learned my addr 172.21.15.153:0/3737785797 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf36d9d700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf381f2180 0x7fcf38156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.294+0000 7fcf3659c700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20003a00 0x7fcf3815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf35d9b700 1 -- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf381f2180 msgr2=0x7fcf38156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf35d9b700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf381f2180 0x7fcf38156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf35d9b700 1 -- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20003a00 msgr2=0x7fcf3815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf35d9b700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20003a00 0x7fcf3815e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf35d9b700 1 -- 172.21.15.153:0/3737785797 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3815cd80 con 0x7fcf381f6690 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf36d9d700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf381f2180 0x7fcf38156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf35d9b700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf381f6690 0x7fcf381f1c40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c011aa0 tx=0x7fcf2c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf26ffd700 1 -- 172.21.15.153:0/3737785797 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00f980 con 0x7fcf381f6690 2023-11-28T13:07:14.302 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf26ffd700 1 -- 172.21.15.153:0/3737785797 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00fb00 con 0x7fcf381f6690 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf380d15e0 con 0x7fcf381f6690 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf26ffd700 1 -- 172.21.15.153:0/3737785797 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00feb0 con 0x7fcf381f6690 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.295+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3815cf10 con 0x7fcf381f6690 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.296+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf381f2800 con 0x7fcf381f6690 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.296+0000 7fcf26ffd700 1 -- 172.21.15.153:0/3737785797 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf2c014440 con 0x7fcf381f6690 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.297+0000 7fcf26ffd700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf180682f0 0x7fcf1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.297+0000 7fcf26ffd700 1 -- 172.21.15.153:0/3737785797 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fcf2c086530 con 0x7fcf381f6690 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.297+0000 7fcf3659c700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf180682f0 0x7fcf1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.297+0000 7fcf3659c700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf180682f0 0x7fcf1806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcf28015040 tx=0x7fcf28009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.301+0000 7fcf26ffd700 1 -- 172.21.15.153:0/3737785797 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf381f2800 con 0x7fcf381f6690 2023-11-28T13:07:14.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.526+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcf381f2800 con 0x7fcf381f6690 2023-11-28T13:07:14.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.526+0000 7fcf26ffd700 1 -- 172.21.15.153:0/3737785797 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fcf381f2800 con 0x7fcf381f6690 2023-11-28T13:07:14.528 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:14.528 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:14.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf180682f0 msgr2=0x7fcf1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf180682f0 0x7fcf1806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcf28015040 tx=0x7fcf28009030 comp rx=0 tx=0).stop 2023-11-28T13:07:14.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf381f6690 msgr2=0x7fcf381f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf381f6690 0x7fcf381f1c40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c011aa0 tx=0x7fcf2c011e60 comp rx=0 tx=0).stop 2023-11-28T13:07:14.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 shutdown_connections 2023-11-28T13:07:14.530 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf381f2180 0x7fcf38156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf20003a00 0x7fcf3815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fcf180682f0 0x7fcf1806a7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 --2- 172.21.15.153:0/3737785797 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf381f6690 0x7fcf381f1c40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 >> 172.21.15.153:0/3737785797 conn(0x7fcf38072600 msgr2=0x7fcf3815fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:14.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 shutdown_connections 2023-11-28T13:07:14.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.528+0000 7fcf3c899700 1 -- 172.21.15.153:0/3737785797 wait complete. 2023-11-28T13:07:14.542 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:07:14.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:07:14.578 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:07:14.627 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:07:14.628 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:07:14.628 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:14.645 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:07:14.646 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:14.703 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:14.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.861+0000 7fa7ff64a700 1 Processor -- start 2023-11-28T13:07:14.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.861+0000 7fa7ff64a700 1 -- start start 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.862+0000 7fa7ff64a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa7f8071040 0x7fa7f8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.862+0000 7fa7ff64a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f8071950 con 0x7fa7f806dee0 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.862+0000 7fa7fcbe5700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa7f806dee0 0x7fa7f806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33166/0 (socket says 172.21.15.153:33166) 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.862+0000 7fa7fcbe5700 1 -- 172.21.15.153:0/238608578 learned_addr learned my addr 172.21.15.153:0/238608578 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.862+0000 7fa7ff64a700 1 -- 172.21.15.153:0/238608578 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f815d060 con 0x7fa7f8160540 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.862+0000 7fa7ff64a700 1 -- 172.21.15.153:0/238608578 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f815d340 con 0x7fa7f8071040 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.862+0000 7fa7fd3e6700 1 --2- 172.21.15.153:0/238608578 >> v2:172.21.15.177:3300/0 conn(0x7fa7f8071040 0x7fa7f8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7effff700 1 -- 172.21.15.153:0/238608578 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587902805 0 0) 0x7fa7f8071950 con 0x7fa7f806dee0 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7effff700 1 -- 172.21.15.153:0/238608578 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7e40035b0 con 0x7fa7f806dee0 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7fd3e6700 1 -- 172.21.15.153:0/238608578 >> v1:172.21.15.177:6789/0 conn(0x7fa7f8160540 legacy=0x7fa7f8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7effff700 1 -- 172.21.15.153:0/238608578 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363995636 0 0) 0x7fa7e40035b0 con 0x7fa7f806dee0 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7fd3e6700 1 -- 172.21.15.153:0/238608578 >> v1:172.21.15.153:6789/0 conn(0x7fa7f806dee0 legacy=0x7fa7f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7fd3e6700 1 -- 172.21.15.153:0/238608578 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f815d620 con 0x7fa7f8071040 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7fd3e6700 1 --2- 172.21.15.153:0/238608578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f8071040 0x7fa7f8071410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009840 tx=0x7fa7e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1832ece10df34c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7effff700 1 -- 172.21.15.153:0/238608578 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4001670 con 0x7fa7f8071040 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7effff700 1 -- 172.21.15.153:0/238608578 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e8004030 con 0x7fa7f8071040 2023-11-28T13:07:14.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.863+0000 7fa7effff700 1 -- 172.21.15.153:0/238608578 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80043a0 con 0x7fa7f8071040 2023-11-28T13:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.864+0000 7fa7ff64a700 1 -- 172.21.15.153:0/238608578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f8071040 msgr2=0x7fa7f8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.864+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/238608578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f8071040 0x7fa7f8071410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009840 tx=0x7fa7e8009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.864+0000 7fa7ff64a700 1 -- 172.21.15.153:0/238608578 shutdown_connections 2023-11-28T13:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.864+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/238608578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f8071040 0x7fa7f8071410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.864+0000 7fa7ff64a700 1 -- 172.21.15.153:0/238608578 >> 172.21.15.153:0/238608578 conn(0x7fa7f80d0000 msgr2=0x7fa7f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.864+0000 7fa7ff64a700 1 -- 172.21.15.153:0/238608578 shutdown_connections 2023-11-28T13:07:14.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.864+0000 7fa7ff64a700 1 -- 172.21.15.153:0/238608578 wait complete. 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.865+0000 7fa7ff64a700 1 Processor -- start 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.865+0000 7fa7ff64a700 1 -- start start 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7ff64a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f806dee0 0x7fa7f8159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7ff64a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7f8071040 0x7fa7f8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7ff64a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f8160540 0x7fa7f80b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7ff64a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8159640 con 0x7fa7f8160540 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7ff64a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80c4c50 con 0x7fa7f806dee0 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7ff64a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80c4f30 con 0x7fa7f8071040 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f8160540 0x7fa7f80b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f8160540 0x7fa7f80b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43244/0 (socket says 172.21.15.153:43244) 2023-11-28T13:07:14.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 -- 172.21.15.153:0/4132954233 learned_addr learned my addr 172.21.15.153:0/4132954233 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fcbe5700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7f8071040 0x7fa7f8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fd3e6700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f806dee0 0x7fa7f8159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 -- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7f8071040 msgr2=0x7fa7f8155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7f8071040 0x7fa7f8155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 -- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f806dee0 msgr2=0x7fa7f8159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f806dee0 0x7fa7f8159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 -- 172.21.15.153:0/4132954233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80094a0 con 0x7fa7f8160540 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fcbe5700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7f8071040 0x7fa7f8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fd3e6700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f806dee0 0x7fa7f8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:14.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.866+0000 7fa7fdbe7700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f8160540 0x7fa7f80b5780 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c870 tx=0x7fa7f000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.867+0000 7fa7edffb700 1 -- 172.21.15.153:0/4132954233 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0003e30 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.867+0000 7fa7edffb700 1 -- 172.21.15.153:0/4132954233 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f000ce10 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.867+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f815ccd0 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.867+0000 7fa7edffb700 1 -- 172.21.15.153:0/4132954233 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00052b0 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.867+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f815d900 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.868+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f806a270 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.868+0000 7fa7edffb700 1 -- 172.21.15.153:0/4132954233 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7f0019030 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.868+0000 7fa7edffb700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa7dc0682f0 0x7fa7dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.868+0000 7fa7edffb700 1 -- 172.21.15.153:0/4132954233 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa7f0085300 con 0x7fa7f8160540 2023-11-28T13:07:14.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.868+0000 7fa7fd3e6700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa7dc0682f0 0x7fa7dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.869+0000 7fa7fd3e6700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa7dc0682f0 0x7fa7dc06a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa7f815fba0 tx=0x7fa7e8008000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:14.872+0000 7fa7edffb700 1 -- 172.21.15.153:0/4132954233 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7f0051ad0 con 0x7fa7f8160540 2023-11-28T13:07:15.056 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.054+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7f806c2e0 con 0x7fa7f8160540 2023-11-28T13:07:15.057 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.056+0000 7fa7edffb700 1 -- 172.21.15.153:0/4132954233 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18209 (secure 0 0 0) 0x7fa7f0017090 con 0x7fa7f8160540 2023-11-28T13:07:15.058 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:15.058 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":65,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:04:00.505453+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:52.747638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:53.909181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.3a","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:15.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa7dc0682f0 msgr2=0x7fa7dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa7dc0682f0 0x7fa7dc06a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa7f815fba0 tx=0x7fa7e8008000 comp rx=0 tx=0).stop 2023-11-28T13:07:15.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f8160540 msgr2=0x7fa7f80b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f8160540 0x7fa7f80b5780 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c870 tx=0x7fa7f000cc30 comp rx=0 tx=0).stop 2023-11-28T13:07:15.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 shutdown_connections 2023-11-28T13:07:15.060 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa7dc0682f0 0x7fa7dc06a7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7f8160540 0x7fa7f80b5780 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7f806dee0 0x7fa7f8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 --2- 172.21.15.153:0/4132954233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7f8071040 0x7fa7f8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 >> 172.21.15.153:0/4132954233 conn(0x7fa7f80d0000 msgr2=0x7fa7f815e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:15.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 shutdown_connections 2023-11-28T13:07:15.061 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.058+0000 7fa7ff64a700 1 -- 172.21.15.153:0/4132954233 wait complete. 2023-11-28T13:07:15.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:02:52.747638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:02:53.909181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:07:15.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:07:15.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.193+0000 7f46d8528700 1 Processor -- start 2023-11-28T13:07:15.195 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.193+0000 7f46d8528700 1 -- start start 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.194+0000 7f46d8528700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.194+0000 7f46d8528700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.194+0000 7f46d8528700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.194+0000 7f46d8528700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d0071950 con 0x7f46d0071040 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.194+0000 7f46d8528700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d015d060 con 0x7f46d006dee0 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.194+0000 7f46d8528700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d015d340 con 0x7f46d006cdc0 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.194+0000 7f46d62c4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43252/0 (socket says 172.21.15.153:43252) 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 -- 172.21.15.153:0/2181191639 learned_addr learned my addr 172.21.15.153:0/2181191639 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d5ac3700 1 --2- 172.21.15.153:0/2181191639 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d6ac5700 1 --2- 172.21.15.153:0/2181191639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 -- 172.21.15.153:0/2181191639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 msgr2=0x7f46d006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 --2- 172.21.15.153:0/2181191639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 -- 172.21.15.153:0/2181191639 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 msgr2=0x7f46d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 --2- 172.21.15.153:0/2181191639 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 -- 172.21.15.153:0/2181191639 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d015d620 con 0x7f46d0071040 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d62c4700 1 --2- 172.21.15.153:0/2181191639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d0071410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f46d006fb00 tx=0x7f46c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fc47a83d9a56a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d52c2700 1 -- 172.21.15.153:0/2181191639 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000b870 con 0x7f46d0071040 2023-11-28T13:07:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d52c2700 1 -- 172.21.15.153:0/2181191639 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c000b9d0 con 0x7f46d0071040 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.195+0000 7f46d52c2700 1 -- 172.21.15.153:0/2181191639 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000bd60 con 0x7f46d0071040 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 -- 172.21.15.153:0/2181191639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 msgr2=0x7f46d0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 --2- 172.21.15.153:0/2181191639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d0071410 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f46d006fb00 tx=0x7f46c0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 -- 172.21.15.153:0/2181191639 shutdown_connections 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 --2- 172.21.15.153:0/2181191639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 --2- 172.21.15.153:0/2181191639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 --2- 172.21.15.153:0/2181191639 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 -- 172.21.15.153:0/2181191639 >> 172.21.15.153:0/2181191639 conn(0x7f46d00d0000 msgr2=0x7f46d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 -- 172.21.15.153:0/2181191639 shutdown_connections 2023-11-28T13:07:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.196+0000 7f46d8528700 1 -- 172.21.15.153:0/2181191639 wait complete. 2023-11-28T13:07:15.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.197+0000 7f46d8528700 1 Processor -- start 2023-11-28T13:07:15.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.197+0000 7f46d8528700 1 -- start start 2023-11-28T13:07:15.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006cdc0 0x7f46d0154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006dee0 0x7f46d0152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d00b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c3aa0 con 0x7f46d0071040 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c3d80 con 0x7f46d006cdc0 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c4060 con 0x7f46d006dee0 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d00b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d00b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43258/0 (socket says 172.21.15.153:43258) 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 -- 172.21.15.153:0/1764432202 learned_addr learned my addr 172.21.15.153:0/1764432202 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d5ac3700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006dee0 0x7f46d0152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d62c4700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006cdc0 0x7f46d0154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 -- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006dee0 msgr2=0x7f46d0152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006dee0 0x7f46d0152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 -- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006cdc0 msgr2=0x7f46d0154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006cdc0 0x7f46d0154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 -- 172.21.15.153:0/1764432202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009550 con 0x7f46d0071040 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d62c4700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006cdc0 0x7f46d0154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d6ac5700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d00b4780 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f46c8007d20 tx=0x7f46c800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46c6ffd700 1 -- 172.21.15.153:0/1764432202 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c800ede0 con 0x7f46d0071040 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46c6ffd700 1 -- 172.21.15.153:0/1764432202 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c8007e70 con 0x7f46d0071040 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d006e870 con 0x7f46d0071040 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46c6ffd700 1 -- 172.21.15.153:0/1764432202 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8014640 con 0x7f46d0071040 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.198+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d015d900 con 0x7f46d0071040 2023-11-28T13:07:15.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.199+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d015ccd0 con 0x7f46d0071040 2023-11-28T13:07:15.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.200+0000 7f46c6ffd700 1 -- 172.21.15.153:0/1764432202 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46c8012050 con 0x7f46d0071040 2023-11-28T13:07:15.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.200+0000 7f46c6ffd700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.200+0000 7f46c6ffd700 1 -- 172.21.15.153:0/1764432202 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f46c80861e0 con 0x7f46d0071040 2023-11-28T13:07:15.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.200+0000 7f46d62c4700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.201+0000 7f46d62c4700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009840 tx=0x7f46c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.204+0000 7f46c6ffd700 1 -- 172.21.15.153:0/1764432202 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46c8052950 con 0x7f46d0071040 2023-11-28T13:07:15.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.385+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46d00c6d70 con 0x7f46d0071040 2023-11-28T13:07:15.387 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.386+0000 7f46c6ffd700 1 -- 172.21.15.153:0/1764432202 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46c8055f70 con 0x7f46d0071040 2023-11-28T13:07:15.388 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 msgr2=0x7f46b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009840 tx=0x7f46c0008030 comp rx=0 tx=0).stop 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 msgr2=0x7f46d00b4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d00b4780 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f46c8007d20 tx=0x7f46c800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 shutdown_connections 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d00b4780 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006cdc0 0x7f46d0154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 --2- 172.21.15.153:0/1764432202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006dee0 0x7f46d0152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 >> 172.21.15.153:0/1764432202 conn(0x7f46d00d0000 msgr2=0x7f46d015e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:15.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 shutdown_connections 2023-11-28T13:07:15.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.388+0000 7f46d8528700 1 -- 172.21.15.153:0/1764432202 wait complete. 2023-11-28T13:07:15.402 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:07:15.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.523+0000 7f26e5944700 1 Processor -- start 2023-11-28T13:07:15.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.523+0000 7f26e5944700 1 -- start start 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.523+0000 7f26e5944700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26e0158ce0 0x7f26e015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.523+0000 7f26e5944700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f26e00ce4b0 0x7f26e006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.523+0000 7f26e5944700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e006bf10 con 0x7f26e006fb30 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.523+0000 7f26e5944700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e015d730 con 0x7f26e0158ce0 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.523+0000 7f26e5944700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e01590b0 con 0x7f26e00ce4b0 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26e0158ce0 0x7f26e015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f26e0158ce0 0x7f26e015d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43262/0 (socket says 172.21.15.153:43262) 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 -- 172.21.15.153:0/2124999782 learned_addr learned my addr 172.21.15.153:0/2124999782 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:15.527 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 -- 172.21.15.153:0/2124999782 >> v2:172.21.15.177:3300/0 conn(0x7f26e00ce4b0 msgr2=0x7f26e006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 --2- 172.21.15.153:0/2124999782 >> v2:172.21.15.177:3300/0 conn(0x7f26e00ce4b0 0x7f26e006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 -- 172.21.15.153:0/2124999782 >> v1:172.21.15.177:6789/0 conn(0x7f26e006fb30 legacy=0x7f26e00cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 -- 172.21.15.153:0/2124999782 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e00cb0d0 con 0x7f26e0158ce0 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26deffd700 1 --2- 172.21.15.153:0/2124999782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e015d1f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f26e00c2630 tx=0x7f26d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad281fe215ec9c22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26ddffb700 1 -- 172.21.15.153:0/2124999782 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004230 con 0x7f26e0158ce0 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26ddffb700 1 -- 172.21.15.153:0/2124999782 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d0004390 con 0x7f26e0158ce0 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.524+0000 7f26ddffb700 1 -- 172.21.15.153:0/2124999782 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d00036a0 con 0x7f26e0158ce0 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 -- 172.21.15.153:0/2124999782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 msgr2=0x7f26e015d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 --2- 172.21.15.153:0/2124999782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e015d1f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f26e00c2630 tx=0x7f26d0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 -- 172.21.15.153:0/2124999782 shutdown_connections 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 --2- 172.21.15.153:0/2124999782 >> v2:172.21.15.177:3300/0 conn(0x7f26e00ce4b0 0x7f26e006c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 --2- 172.21.15.153:0/2124999782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e015d1f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.528 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 -- 172.21.15.153:0/2124999782 >> 172.21.15.153:0/2124999782 conn(0x7f26e00cb670 msgr2=0x7f26e00cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:15.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 -- 172.21.15.153:0/2124999782 shutdown_connections 2023-11-28T13:07:15.529 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.525+0000 7f26e5944700 1 -- 172.21.15.153:0/2124999782 wait complete. 2023-11-28T13:07:15.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.526+0000 7f26e5944700 1 Processor -- start 2023-11-28T13:07:15.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.526+0000 7f26e5944700 1 -- start start 2023-11-28T13:07:15.531 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26e5944700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006fb30 0x7f26e015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26e5944700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e00ce4b0 0x7f26e00c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26e5944700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26e5944700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e00c50a0 con 0x7f26e0158ce0 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26e5944700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e015be60 con 0x7f26e006fb30 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26e5944700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e00c3d60 con 0x7f26e00ce4b0 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e00701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43272/0 (socket says 172.21.15.153:43272) 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 -- 172.21.15.153:0/3073639073 learned_addr learned my addr 172.21.15.153:0/3073639073 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26de7fc700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e00ce4b0 0x7f26e00c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26deffd700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006fb30 0x7f26e015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 -- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e00ce4b0 msgr2=0x7f26e00c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.532 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e00ce4b0 0x7f26e00c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 -- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006fb30 msgr2=0x7f26e015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006fb30 0x7f26e015b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 -- 172.21.15.153:0/3073639073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d0009550 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.527+0000 7f26df7fe700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e00701d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b9e0 tx=0x7f26d800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.528+0000 7f26c7fff700 1 -- 172.21.15.153:0/3073639073 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800c9b0 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.528+0000 7f26c7fff700 1 -- 172.21.15.153:0/3073639073 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d800cb30 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.528+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e006b8c0 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.528+0000 7f26c7fff700 1 -- 172.21.15.153:0/3073639073 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8014440 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.528+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e00cb3b0 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.529+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e006bba0 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.529+0000 7f26c7fff700 1 -- 172.21.15.153:0/3073639073 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f26d801b030 con 0x7f26e0158ce0 2023-11-28T13:07:15.533 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.529+0000 7f26c7fff700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f26cc0682f0 0x7f26cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.530+0000 7f26deffd700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f26cc0682f0 0x7f26cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.530+0000 7f26c7fff700 1 -- 172.21.15.153:0/3073639073 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f26d8006e40 con 0x7f26e0158ce0 2023-11-28T13:07:15.534 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.530+0000 7f26deffd700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f26cc0682f0 0x7f26cc06a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009130 tx=0x7f26d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.535 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.534+0000 7f26c7fff700 1 -- 172.21.15.153:0/3073639073 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f26d8052130 con 0x7f26e0158ce0 2023-11-28T13:07:15.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.714+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f26e00745c0 con 0x7f26e0158ce0 2023-11-28T13:07:15.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.715+0000 7f26c7fff700 1 -- 172.21.15.153:0/3073639073 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f26d80190a0 con 0x7f26e0158ce0 2023-11-28T13:07:15.717 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f26cc0682f0 msgr2=0x7f26cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f26cc0682f0 0x7f26cc06a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009130 tx=0x7f26d0008030 comp rx=0 tx=0).stop 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 msgr2=0x7f26e00701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e00701d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b9e0 tx=0x7f26d800bda0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 shutdown_connections 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f26cc0682f0 0x7f26cc06a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26e0158ce0 0x7f26e00701d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006fb30 0x7f26e015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 --2- 172.21.15.153:0/3073639073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e00ce4b0 0x7f26e00c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 >> 172.21.15.153:0/3073639073 conn(0x7f26e00cb670 msgr2=0x7f26e0158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 shutdown_connections 2023-11-28T13:07:15.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.717+0000 7f26e5944700 1 -- 172.21.15.153:0/3073639073 wait complete. 2023-11-28T13:07:15.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:07:15.852 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce51fab700 1 Processor -- start 2023-11-28T13:07:15.853 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce51fab700 1 -- start start 2023-11-28T13:07:15.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce51fab700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fce4c06ec90 0x7fce4c065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce51fab700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fce4c0cec70 0x7fce4c0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce51fab700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c06e240 con 0x7fce4c0c9f10 2023-11-28T13:07:15.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce51fab700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c066580 con 0x7fce4c0cec70 2023-11-28T13:07:15.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce51fab700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0cded0 con 0x7fce4c06ec90 2023-11-28T13:07:15.855 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.851+0000 7fce4affd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fce4c0cec70 0x7fce4c0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4bfff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fce4c0c9f10 0x7fce4c15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33210/0 (socket says 172.21.15.153:33210) 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4bfff700 1 -- 172.21.15.153:0/2087592145 learned_addr learned my addr 172.21.15.153:0/2087592145 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4b7fe700 1 --2- 172.21.15.153:0/2087592145 >> v2:172.21.15.177:3300/0 conn(0x7fce4c06ec90 0x7fce4c065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4a7fc700 1 -- 172.21.15.153:0/2087592145 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369540951 0 0) 0x7fce44001490 con 0x7fce4c0c9f10 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4a7fc700 1 -- 172.21.15.153:0/2087592145 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce340035b0 con 0x7fce4c0c9f10 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4affd700 1 -- 172.21.15.153:0/2087592145 >> v2:172.21.15.177:3300/0 conn(0x7fce4c06ec90 msgr2=0x7fce4c065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4affd700 1 --2- 172.21.15.153:0/2087592145 >> v2:172.21.15.177:3300/0 conn(0x7fce4c06ec90 0x7fce4c065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4affd700 1 -- 172.21.15.153:0/2087592145 >> v1:172.21.15.153:6789/0 conn(0x7fce4c0c9f10 legacy=0x7fce4c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4affd700 1 -- 172.21.15.153:0/2087592145 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0ce1b0 con 0x7fce4c0cec70 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4b7fe700 1 --2- 172.21.15.153:0/2087592145 >> v2:172.21.15.177:3300/0 conn(0x7fce4c06ec90 0x7fce4c065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4affd700 1 --2- 172.21.15.153:0/2087592145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c0cec70 0x7fce4c0ca9d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fce4c073d40 tx=0x7fce3c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0868c0140b5383 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.856 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4a7fc700 1 -- 172.21.15.153:0/2087592145 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c011830 con 0x7fce4c0cec70 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4a7fc700 1 -- 172.21.15.153:0/2087592145 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c011990 con 0x7fce4c0cec70 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.852+0000 7fce4a7fc700 1 -- 172.21.15.153:0/2087592145 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c011d00 con 0x7fce4c0cec70 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 -- 172.21.15.153:0/2087592145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c0cec70 msgr2=0x7fce4c0ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 --2- 172.21.15.153:0/2087592145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c0cec70 0x7fce4c0ca9d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fce4c073d40 tx=0x7fce3c009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 -- 172.21.15.153:0/2087592145 shutdown_connections 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 --2- 172.21.15.153:0/2087592145 >> v2:172.21.15.177:3300/0 conn(0x7fce4c06ec90 0x7fce4c065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 --2- 172.21.15.153:0/2087592145 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c0cec70 0x7fce4c0ca9d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 -- 172.21.15.153:0/2087592145 >> 172.21.15.153:0/2087592145 conn(0x7fce4c0b12b0 msgr2=0x7fce4c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 -- 172.21.15.153:0/2087592145 shutdown_connections 2023-11-28T13:07:15.857 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.853+0000 7fce51fab700 1 -- 172.21.15.153:0/2087592145 wait complete. 2023-11-28T13:07:15.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.854+0000 7fce51fab700 1 Processor -- start 2023-11-28T13:07:15.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.854+0000 7fce51fab700 1 -- start start 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce51fab700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c06ec90 0x7fce4c0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce51fab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce4c0c9f10 0x7fce4c0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce51fab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4c0cec70 0x7fce4c14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce51fab700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b3e50 con 0x7fce4c06ec90 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce51fab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b3900 con 0x7fce4c0cec70 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce51fab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b40f0 con 0x7fce4c0c9f10 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c06ec90 0x7fce4c0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c06ec90 0x7fce4c0c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43282/0 (socket says 172.21.15.153:43282) 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 -- 172.21.15.153:0/4014635043 learned_addr learned my addr 172.21.15.153:0/4014635043 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4bfff700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4c0cec70 0x7fce4c14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4affd700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce4c0c9f10 0x7fce4c0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 -- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce4c0c9f10 msgr2=0x7fce4c0b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce4c0c9f10 0x7fce4c0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 -- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4c0cec70 msgr2=0x7fce4c14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4c0cec70 0x7fce4c14ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.855+0000 7fce4b7fe700 1 -- 172.21.15.153:0/4014635043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0094a0 con 0x7fce4c06ec90 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.856+0000 7fce4b7fe700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c06ec90 0x7fce4c0c6890 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c7f0 tx=0x7fce4000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.856+0000 7fce33fff700 1 -- 172.21.15.153:0/4014635043 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40011800 con 0x7fce4c06ec90 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.856+0000 7fce33fff700 1 -- 172.21.15.153:0/4014635043 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40011960 con 0x7fce4c06ec90 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.856+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c0ce770 con 0x7fce4c06ec90 2023-11-28T13:07:15.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.856+0000 7fce33fff700 1 -- 172.21.15.153:0/4014635043 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40011d10 con 0x7fce4c06ec90 2023-11-28T13:07:15.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.856+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c0ce490 con 0x7fce4c06ec90 2023-11-28T13:07:15.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.857+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c0cb6c0 con 0x7fce4c06ec90 2023-11-28T13:07:15.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.857+0000 7fce33fff700 1 -- 172.21.15.153:0/4014635043 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce40021070 con 0x7fce4c06ec90 2023-11-28T13:07:15.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.857+0000 7fce33fff700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fce380682f0 0x7fce3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:15.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.858+0000 7fce33fff700 1 -- 172.21.15.153:0/4014635043 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fce40085d70 con 0x7fce4c06ec90 2023-11-28T13:07:15.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.858+0000 7fce4affd700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fce380682f0 0x7fce3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:15.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.858+0000 7fce4affd700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fce380682f0 0x7fce3806a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fce4c14d470 tx=0x7fce3c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:15.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:15.862+0000 7fce33fff700 1 -- 172.21.15.153:0/4014635043 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce400523d0 con 0x7fce4c06ec90 2023-11-28T13:07:16.045 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.044+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce4c0ae830 con 0x7fce4c06ec90 2023-11-28T13:07:16.046 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.044+0000 7fce33fff700 1 -- 172.21.15.153:0/4014635043 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce4001f020 con 0x7fce4c06ec90 2023-11-28T13:07:16.046 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:07:16.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fce380682f0 msgr2=0x7fce3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fce380682f0 0x7fce3806a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fce4c14d470 tx=0x7fce3c008030 comp rx=0 tx=0).stop 2023-11-28T13:07:16.048 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c06ec90 msgr2=0x7fce4c0c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c06ec90 0x7fce4c0c6890 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c7f0 tx=0x7fce4000cbb0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 shutdown_connections 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fce380682f0 0x7fce3806a7a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce4c06ec90 0x7fce4c0c6890 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4c0cec70 0x7fce4c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 --2- 172.21.15.153:0/4014635043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce4c0c9f10 0x7fce4c0b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 >> 172.21.15.153:0/4014635043 conn(0x7fce4c0b12b0 msgr2=0x7fce4c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 shutdown_connections 2023-11-28T13:07:16.049 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.046+0000 7fce51fab700 1 -- 172.21.15.153:0/4014635043 wait complete. 2023-11-28T13:07:16.061 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:07:16.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.181+0000 7f8e92636700 1 Processor -- start 2023-11-28T13:07:16.183 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.181+0000 7f8e92636700 1 -- start start 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e92636700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e8c154e70 0x7f8e8c0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e92636700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c0cf360 0x7f8e8c071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e92636700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8e8c071580 0x7f8e8c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e92636700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c0719e0 con 0x7f8e8c154e70 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e92636700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c155240 con 0x7f8e8c071580 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e92636700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c15b100 con 0x7f8e8c0cf360 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e8bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e8c154e70 0x7f8e8c0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e8bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e8c154e70 0x7f8e8c0cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43290/0 (socket says 172.21.15.153:43290) 2023-11-28T13:07:16.186 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.182+0000 7f8e8bfff700 1 -- 172.21.15.153:0/2476501082 learned_addr learned my addr 172.21.15.153:0/2476501082 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8b7fe700 1 --2- 172.21.15.153:0/2476501082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c0cf360 0x7f8e8c071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e90bd3700 1 --2- 172.21.15.153:0/2476501082 >> v2:172.21.15.177:3300/0 conn(0x7f8e8c071580 0x7f8e8c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8bfff700 1 -- 172.21.15.153:0/2476501082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c0cf360 msgr2=0x7f8e8c071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8bfff700 1 --2- 172.21.15.153:0/2476501082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c0cf360 0x7f8e8c071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8bfff700 1 -- 172.21.15.153:0/2476501082 >> v2:172.21.15.177:3300/0 conn(0x7f8e8c071580 msgr2=0x7f8e8c15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8bfff700 1 --2- 172.21.15.153:0/2476501082 >> v2:172.21.15.177:3300/0 conn(0x7f8e8c071580 0x7f8e8c15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8bfff700 1 -- 172.21.15.153:0/2476501082 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c15b3e0 con 0x7f8e8c154e70 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8bfff700 1 --2- 172.21.15.153:0/2476501082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c154e70 0x7f8e8c0cdbd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0af0c0 tx=0x7f8e80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249d22cefe3ca5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8affd700 1 -- 172.21.15.153:0/2476501082 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000b870 con 0x7f8e8c154e70 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8affd700 1 -- 172.21.15.153:0/2476501082 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8000b9d0 con 0x7f8e8c154e70 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e8affd700 1 -- 172.21.15.153:0/2476501082 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000bd60 con 0x7f8e8c154e70 2023-11-28T13:07:16.187 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e92636700 1 -- 172.21.15.153:0/2476501082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c154e70 msgr2=0x7f8e8c0cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.183+0000 7f8e92636700 1 --2- 172.21.15.153:0/2476501082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c154e70 0x7f8e8c0cdbd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0af0c0 tx=0x7f8e80009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.184+0000 7f8e92636700 1 -- 172.21.15.153:0/2476501082 shutdown_connections 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.184+0000 7f8e92636700 1 --2- 172.21.15.153:0/2476501082 >> v2:172.21.15.177:3300/0 conn(0x7f8e8c071580 0x7f8e8c15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.184+0000 7f8e92636700 1 --2- 172.21.15.153:0/2476501082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c154e70 0x7f8e8c0cdbd0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.184+0000 7f8e92636700 1 --2- 172.21.15.153:0/2476501082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c0cf360 0x7f8e8c071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.184+0000 7f8e92636700 1 -- 172.21.15.153:0/2476501082 >> 172.21.15.153:0/2476501082 conn(0x7f8e8c0d0000 msgr2=0x7f8e8c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.184+0000 7f8e92636700 1 -- 172.21.15.153:0/2476501082 shutdown_connections 2023-11-28T13:07:16.188 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.184+0000 7f8e92636700 1 -- 172.21.15.153:0/2476501082 wait complete. 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.185+0000 7f8e92636700 1 Processor -- start 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.185+0000 7f8e92636700 1 -- start start 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e8c071580 0x7f8e8c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c0cf360 0x7f8e8c0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c154e70 0x7f8e8c06e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0c7170 con 0x7f8e8c0cf360 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0c7450 con 0x7f8e8c071580 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0c7730 con 0x7f8e8c154e70 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c0cf360 0x7f8e8c0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c0cf360 0x7f8e8c0b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43298/0 (socket says 172.21.15.153:43298) 2023-11-28T13:07:16.191 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 -- 172.21.15.153:0/2706080891 learned_addr learned my addr 172.21.15.153:0/2706080891 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8bfff700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e8c071580 0x7f8e8c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e90bd3700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c154e70 0x7f8e8c06e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 -- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c154e70 msgr2=0x7f8e8c06e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c154e70 0x7f8e8c06e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 -- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e8c071580 msgr2=0x7f8e8c15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e8c071580 0x7f8e8c15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 -- 172.21.15.153:0/2706080891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80009550 con 0x7f8e8c0cf360 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e8b7fe700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c0cf360 0x7f8e8c0b5820 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e810 tx=0x7f8e7c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e88ff9700 1 -- 172.21.15.153:0/2706080891 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00c070 con 0x7f8e8c0cf360 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e88ff9700 1 -- 172.21.15.153:0/2706080891 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c013610 con 0x7f8e8c0cf360 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c15b9a0 con 0x7f8e8c0cf360 2023-11-28T13:07:16.192 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e88ff9700 1 -- 172.21.15.153:0/2706080891 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0139c0 con 0x7f8e8c0cf360 2023-11-28T13:07:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.186+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c15b6c0 con 0x7f8e8c0cf360 2023-11-28T13:07:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.187+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c0af520 con 0x7f8e8c0cf360 2023-11-28T13:07:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.188+0000 7f8e88ff9700 1 -- 172.21.15.153:0/2706080891 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8e7c023070 con 0x7f8e8c0cf360 2023-11-28T13:07:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.188+0000 7f8e88ff9700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e780682f0 0x7f8e7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.188+0000 7f8e88ff9700 1 -- 172.21.15.153:0/2706080891 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8e7c085a80 con 0x7f8e8c0cf360 2023-11-28T13:07:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.188+0000 7f8e8bfff700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e780682f0 0x7f8e7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.189+0000 7f8e8bfff700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e780682f0 0x7f8e7806a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e80014040 tx=0x7f8e80008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:16.194 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.192+0000 7f8e88ff9700 1 -- 172.21.15.153:0/2706080891 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8e7c0521d0 con 0x7f8e8c0cf360 2023-11-28T13:07:16.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.374+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e8c072260 con 0x7f8e8c0cf360 2023-11-28T13:07:16.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.374+0000 7f8e88ff9700 1 -- 172.21.15.153:0/2706080891 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e7c00f040 con 0x7f8e8c0cf360 2023-11-28T13:07:16.376 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:07:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e780682f0 msgr2=0x7f8e7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e780682f0 0x7f8e7806a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e80014040 tx=0x7f8e80008030 comp rx=0 tx=0).stop 2023-11-28T13:07:16.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c0cf360 msgr2=0x7f8e8c0b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c0cf360 0x7f8e8c0b5820 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e810 tx=0x7f8e7c00eb20 comp rx=0 tx=0).stop 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 shutdown_connections 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e8c154e70 0x7f8e8c06e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e8c071580 0x7f8e8c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e780682f0 0x7f8e7806a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 --2- 172.21.15.153:0/2706080891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e8c0cf360 0x7f8e8c0b5820 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 >> 172.21.15.153:0/2706080891 conn(0x7f8e8c0d0000 msgr2=0x7f8e8c0adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 shutdown_connections 2023-11-28T13:07:16.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.376+0000 7f8e92636700 1 -- 172.21.15.153:0/2706080891 wait complete. 2023-11-28T13:07:16.390 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:07:16.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.509+0000 7f2ed1ecc700 1 Processor -- start 2023-11-28T13:07:16.511 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.509+0000 7f2ed1ecc700 1 -- start start 2023-11-28T13:07:16.512 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ed1ecc700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ecc071040 0x7f2ecc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ed1ecc700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc071950 con 0x7f2ecc06dee0 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ed1ecc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc15d060 con 0x7f2ecc160540 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ed1ecc700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc15d340 con 0x7f2ecc071040 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ecb7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ecc071040 0x7f2ecc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ecaffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2ecc06dee0 0x7f2ecc06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33236/0 (socket says 172.21.15.153:33236) 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ecb7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ecc071040 0x7f2ecc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43304/0 (socket says 172.21.15.153:43304) 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ecaffd700 1 -- 172.21.15.153:0/3310932369 learned_addr learned my addr 172.21.15.153:0/3310932369 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ecb7fe700 1 -- 172.21.15.153:0/3310932369 >> v1:172.21.15.177:6789/0 conn(0x7f2ecc160540 legacy=0x7f2ecc160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ecb7fe700 1 -- 172.21.15.153:0/3310932369 >> v1:172.21.15.153:6789/0 conn(0x7f2ecc06dee0 legacy=0x7f2ecc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.510+0000 7f2ecb7fe700 1 -- 172.21.15.153:0/3310932369 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc15d620 con 0x7f2ecc071040 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ecb7fe700 1 --2- 172.21.15.153:0/3310932369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc071040 0x7f2ecc071410 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009840 tx=0x7f2ebc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100cba25a6ef5174 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2eca7fc700 1 -- 172.21.15.153:0/3310932369 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc004230 con 0x7f2ecc071040 2023-11-28T13:07:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2eca7fc700 1 -- 172.21.15.153:0/3310932369 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc004390 con 0x7f2ecc071040 2023-11-28T13:07:16.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2eca7fc700 1 -- 172.21.15.153:0/3310932369 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0036a0 con 0x7f2ecc071040 2023-11-28T13:07:16.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3310932369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc071040 msgr2=0x7f2ecc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3310932369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc071040 0x7f2ecc071410 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009840 tx=0x7f2ebc009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:16.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3310932369 shutdown_connections 2023-11-28T13:07:16.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3310932369 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc071040 0x7f2ecc071410 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3310932369 >> 172.21.15.153:0/3310932369 conn(0x7f2ecc0d0000 msgr2=0x7f2ecc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3310932369 shutdown_connections 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.511+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3310932369 wait complete. 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.512+0000 7f2ed1ecc700 1 Processor -- start 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.513+0000 7f2ed1ecc700 1 -- start start 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.513+0000 7f2ed1ecc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc06dee0 0x7f2ecc15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.513+0000 7f2ed1ecc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc071040 0x7f2ecc158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.513+0000 7f2ed1ecc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc160540 0x7f2ecc1f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.513+0000 7f2ed1ecc700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0b5ee0 con 0x7f2ecc06dee0 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.513+0000 7f2ed1ecc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0b3100 con 0x7f2ecc071040 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.513+0000 7f2ed1ecc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0b33e0 con 0x7f2ecc160540 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc06dee0 0x7f2ecc15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc06dee0 0x7f2ecc15a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43310/0 (socket says 172.21.15.153:43310) 2023-11-28T13:07:16.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 -- 172.21.15.153:0/3500869199 learned_addr learned my addr 172.21.15.153:0/3500869199 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecaffd700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc071040 0x7f2ecc158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecbfff700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc160540 0x7f2ecc1f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 -- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc160540 msgr2=0x7f2ecc1f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc160540 0x7f2ecc1f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 -- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc071040 msgr2=0x7f2ecc158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc071040 0x7f2ecc158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 -- 172.21.15.153:0/3500869199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0094a0 con 0x7f2ecc06dee0 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecbfff700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc160540 0x7f2ecc1f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecaffd700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc071040 0x7f2ecc158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ecb7fe700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc06dee0 0x7f2ecc15a6c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009b90 tx=0x7f2ebc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2eb3fff700 1 -- 172.21.15.153:0/3500869199 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011630 con 0x7f2ecc06dee0 2023-11-28T13:07:16.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2eb3fff700 1 -- 172.21.15.153:0/3500869199 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc011790 con 0x7f2ecc06dee0 2023-11-28T13:07:16.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc15ccd0 con 0x7f2ecc06dee0 2023-11-28T13:07:16.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2eb3fff700 1 -- 172.21.15.153:0/3500869199 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011b00 con 0x7f2ecc06dee0 2023-11-28T13:07:16.519 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.514+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc15d900 con 0x7f2ecc06dee0 2023-11-28T13:07:16.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.515+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc06a270 con 0x7f2ecc06dee0 2023-11-28T13:07:16.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.516+0000 7f2eb3fff700 1 -- 172.21.15.153:0/3500869199 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ebc017020 con 0x7f2ecc06dee0 2023-11-28T13:07:16.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.516+0000 7f2eb3fff700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb80682f0 0x7f2eb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:16.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.516+0000 7f2eb3fff700 1 -- 172.21.15.153:0/3500869199 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2ebc087550 con 0x7f2ecc06dee0 2023-11-28T13:07:16.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.516+0000 7f2ecaffd700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb80682f0 0x7f2eb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:16.520 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.516+0000 7f2ecaffd700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb80682f0 0x7f2eb806a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0007fd0 tx=0x7f2ec0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:16.521 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.520+0000 7f2eb3fff700 1 -- 172.21.15.153:0/3500869199 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ebc053d20 con 0x7f2ecc06dee0 2023-11-28T13:07:16.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.719+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ecc160010 con 0x7f2ecc06dee0 2023-11-28T13:07:16.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.719+0000 7f2eb3fff700 1 -- 172.21.15.153:0/3500869199 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ebc057340 con 0x7f2ecc06dee0 2023-11-28T13:07:16.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb80682f0 msgr2=0x7f2eb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb80682f0 0x7f2eb806a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0007fd0 tx=0x7f2ec0009040 comp rx=0 tx=0).stop 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc06dee0 msgr2=0x7f2ecc15a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc06dee0 0x7f2ecc15a6c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009b90 tx=0x7f2ebc0047c0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 shutdown_connections 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2eb80682f0 0x7f2eb806a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ecc06dee0 0x7f2ecc15a6c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc160540 0x7f2ecc1f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 --2- 172.21.15.153:0/3500869199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc071040 0x7f2ecc158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 >> 172.21.15.153:0/3500869199 conn(0x7f2ecc0d0000 msgr2=0x7f2ecc15e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 shutdown_connections 2023-11-28T13:07:16.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:16.722+0000 7f2ed1ecc700 1 -- 172.21.15.153:0/3500869199 wait complete. 2023-11-28T13:07:16.725 INFO:teuthology.orchestra.run.smithi153.stderr:listed 0 entries 2023-11-28T13:07:16.735 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T13:07:16.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:16.736 DEBUG:teuthology.orchestra.run.smithi153:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T13:07:16.788 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:07:16.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:16.789 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:07:16.816 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:07:16.816 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:16.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:16.835 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:07:16.835 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:16.874 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:07:16.874 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:07:16.874 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:16.894 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:16.898 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:17.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.032+0000 7f976ed37700 1 Processor -- start 2023-11-28T13:07:17.034 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.032+0000 7f976ed37700 1 -- start start 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.033+0000 7f976ed37700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f97680ce040 0x7f976815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.033+0000 7f976ed37700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97680b2930 0x7f97680b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.033+0000 7f976ed37700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976806b420 con 0x7f97680d0a00 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.033+0000 7f976ed37700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97680b32f0 con 0x7f97680b2930 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.033+0000 7f976ed37700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976806ff60 con 0x7f97680ce040 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.033+0000 7f976d2d4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f97680b2930 0x7f97680b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.033+0000 7f9767fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f97680d0a00 0x7f976806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33250/0 (socket says 172.21.15.153:33250) 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f9767fff700 1 -- 172.21.15.153:0/2990587841 learned_addr learned my addr 172.21.15.153:0/2990587841 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f976cad3700 1 --2- 172.21.15.153:0/2990587841 >> v2:172.21.15.177:3300/0 conn(0x7f97680ce040 0x7f976815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f976d2d4700 1 -- 172.21.15.153:0/2990587841 >> v2:172.21.15.177:3300/0 conn(0x7f97680ce040 msgr2=0x7f976815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f976d2d4700 1 --2- 172.21.15.153:0/2990587841 >> v2:172.21.15.177:3300/0 conn(0x7f97680ce040 0x7f976815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f976d2d4700 1 -- 172.21.15.153:0/2990587841 >> v1:172.21.15.153:6789/0 conn(0x7f97680d0a00 legacy=0x7f976806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f976d2d4700 1 -- 172.21.15.153:0/2990587841 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976806b700 con 0x7f97680b2930 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f97677fe700 1 -- 172.21.15.153:0/2990587841 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734454364 0 0) 0x7f976806b420 con 0x7f97680d0a00 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f976d2d4700 1 --2- 172.21.15.153:0/2990587841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680b2930 0x7f97680b2db0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f976806eab0 tx=0x7f976000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6603b17ee0e6baf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f97677fe700 1 -- 172.21.15.153:0/2990587841 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976000e070 con 0x7f97680b2930 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f97677fe700 1 -- 172.21.15.153:0/2990587841 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976000bbd0 con 0x7f97680b2930 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.034+0000 7f97677fe700 1 -- 172.21.15.153:0/2990587841 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760012430 con 0x7f97680b2930 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 -- 172.21.15.153:0/2990587841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680b2930 msgr2=0x7f97680b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 --2- 172.21.15.153:0/2990587841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680b2930 0x7f97680b2db0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f976806eab0 tx=0x7f976000b330 comp rx=0 tx=0).stop 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 -- 172.21.15.153:0/2990587841 shutdown_connections 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 --2- 172.21.15.153:0/2990587841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680b2930 0x7f97680b2db0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 --2- 172.21.15.153:0/2990587841 >> v2:172.21.15.177:3300/0 conn(0x7f97680ce040 0x7f976815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 -- 172.21.15.153:0/2990587841 >> 172.21.15.153:0/2990587841 conn(0x7f976806aed0 msgr2=0x7f976806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 -- 172.21.15.153:0/2990587841 shutdown_connections 2023-11-28T13:07:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.035+0000 7f976ed37700 1 -- 172.21.15.153:0/2990587841 wait complete. 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.036+0000 7f976ed37700 1 Processor -- start 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.036+0000 7f976ed37700 1 -- start start 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976ed37700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97680b2930 0x7f97681f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976ed37700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680ce040 0x7f97680c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976ed37700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97680d0a00 0x7f97680cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976ed37700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768147bf0 con 0x7f97680ce040 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976ed37700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97680cad20 con 0x7f97680b2930 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976ed37700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97680c9c80 con 0x7f97680d0a00 2023-11-28T13:07:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680ce040 0x7f97680c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680ce040 0x7f97680c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43326/0 (socket says 172.21.15.153:43326) 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 -- 172.21.15.153:0/3377732555 learned_addr learned my addr 172.21.15.153:0/3377732555 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976cad3700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97680b2930 0x7f97681f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f976d2d4700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97680d0a00 0x7f97680cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 -- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97680d0a00 msgr2=0x7f97680cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97680d0a00 0x7f97680cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 -- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97680b2930 msgr2=0x7f97681f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97680b2930 0x7f97681f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 -- 172.21.15.153:0/3377732555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9760009a70 con 0x7f97680ce040 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.037+0000 7f9767fff700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680ce040 0x7f97680c9230 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f975400c550 tx=0x7f975400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.038+0000 7f97657fa700 1 -- 172.21.15.153:0/3377732555 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754004c80 con 0x7f97680ce040 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.038+0000 7f97657fa700 1 -- 172.21.15.153:0/3377732555 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754004e00 con 0x7f97680ce040 2023-11-28T13:07:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.038+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976806bcc0 con 0x7f97680ce040 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.038+0000 7f97657fa700 1 -- 172.21.15.153:0/3377732555 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754007310 con 0x7f97680ce040 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.038+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976806b9e0 con 0x7f97680ce040 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.039+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976806bfa0 con 0x7f97680ce040 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.039+0000 7f97657fa700 1 -- 172.21.15.153:0/3377732555 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f975401a020 con 0x7f97680ce040 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.039+0000 7f97657fa700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f974c0682f0 0x7f974c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.040+0000 7f97657fa700 1 -- 172.21.15.153:0/3377732555 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9754085970 con 0x7f97680ce040 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.040+0000 7f976cad3700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f974c0682f0 0x7f974c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.040+0000 7f976cad3700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f974c0682f0 0x7f974c06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f975c006fd0 tx=0x7f975c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.045 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.044+0000 7f97657fa700 1 -- 172.21.15.153:0/3377732555 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97540521f0 con 0x7f97680ce040 2023-11-28T13:07:17.135 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:17.138 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:17.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.272+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9768068690 con 0x7f97680ce040 2023-11-28T13:07:17.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.272+0000 7f97657fa700 1 -- 172.21.15.153:0/3377732555 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f9754055810 con 0x7f97680ce040 2023-11-28T13:07:17.274 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:17.275 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:03:11.406430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:17.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.274+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f974c0682f0 msgr2=0x7f974c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.274+0000 7f976ed37700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f974c0682f0 0x7f974c06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f975c006fd0 tx=0x7f975c008040 comp rx=0 tx=0).stop 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.274+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680ce040 msgr2=0x7f97680c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.274+0000 7f976ed37700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680ce040 0x7f97680c9230 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f975400c550 tx=0x7f975400c910 comp rx=0 tx=0).stop 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 shutdown_connections 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97680d0a00 0x7f97680cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97680b2930 0x7f97681f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f974c0682f0 0x7f974c06a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 --2- 172.21.15.153:0/3377732555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97680ce040 0x7f97680c9230 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 >> 172.21.15.153:0/3377732555 conn(0x7f976806aed0 msgr2=0x7f976815ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 shutdown_connections 2023-11-28T13:07:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.275+0000 7f976ed37700 1 -- 172.21.15.153:0/3377732555 wait complete. 2023-11-28T13:07:17.278 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2023-11-28T13:07:17.288 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:17.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.407+0000 7f4d46b6a700 1 Processor -- start 2023-11-28T13:07:17.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.407+0000 7f4d46b6a700 1 -- start start 2023-11-28T13:07:17.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.407+0000 7f4d46b6a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d4006f850 0x7f4d4006fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.407+0000 7f4d46b6a700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4013ff00 con 0x7f4d40140950 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.407+0000 7f4d46b6a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d401404c0 con 0x7f4d400cd8c0 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.407+0000 7f4d46b6a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d40140dc0 con 0x7f4d4006f850 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d45107700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4d4006f850 0x7f4d4006fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d3ffff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4d40140950 0x7f4d400b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33266/0 (socket says 172.21.15.153:33266) 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d3ffff700 1 -- 172.21.15.153:0/3272406354 learned_addr learned my addr 172.21.15.153:0/3272406354 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d45107700 1 -- 172.21.15.153:0/3272406354 >> v1:172.21.15.177:6789/0 conn(0x7f4d400cd8c0 legacy=0x7f4d400cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d45107700 1 -- 172.21.15.153:0/3272406354 >> v1:172.21.15.153:6789/0 conn(0x7f4d40140950 legacy=0x7f4d400b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d3f7fe700 1 -- 172.21.15.153:0/3272406354 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544369562 0 0) 0x7f4d4013ff00 con 0x7f4d40140950 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d45107700 1 -- 172.21.15.153:0/3272406354 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4015aa80 con 0x7f4d4006f850 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d45107700 1 --2- 172.21.15.153:0/3272406354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d4006fcf0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009d90 tx=0x7f4d3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad38a7ae1034550 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d3f7fe700 1 -- 172.21.15.153:0/3272406354 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800e070 con 0x7f4d4006f850 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d3f7fe700 1 -- 172.21.15.153:0/3272406354 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3800bb90 con 0x7f4d4006f850 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.408+0000 7f4d3f7fe700 1 -- 172.21.15.153:0/3272406354 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38003a20 con 0x7f4d4006f850 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.409+0000 7f4d46b6a700 1 -- 172.21.15.153:0/3272406354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 msgr2=0x7f4d4006fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.409+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/3272406354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d4006fcf0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009d90 tx=0x7f4d3800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.409+0000 7f4d46b6a700 1 -- 172.21.15.153:0/3272406354 shutdown_connections 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.409+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/3272406354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d4006fcf0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.409+0000 7f4d46b6a700 1 -- 172.21.15.153:0/3272406354 >> 172.21.15.153:0/3272406354 conn(0x7f4d4006aed0 msgr2=0x7f4d4006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:17.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.409+0000 7f4d46b6a700 1 -- 172.21.15.153:0/3272406354 shutdown_connections 2023-11-28T13:07:17.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.409+0000 7f4d46b6a700 1 -- 172.21.15.153:0/3272406354 wait complete. 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.410+0000 7f4d46b6a700 1 Processor -- start 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.410+0000 7f4d46b6a700 1 -- start start 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d46b6a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d400b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d46b6a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d400cd8c0 0x7f4d401f0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d46b6a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d40140950 0x7f4d400c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d46b6a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400b6140 con 0x7f4d4006f850 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d46b6a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401f0dc0 con 0x7f4d40140950 2023-11-28T13:07:17.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d46b6a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400c4dc0 con 0x7f4d400cd8c0 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d400b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d400b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43338/0 (socket says 172.21.15.153:43338) 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 -- 172.21.15.153:0/2433451446 learned_addr learned my addr 172.21.15.153:0/2433451446 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d45107700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d40140950 0x7f4d400c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d3ffff700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d400cd8c0 0x7f4d401f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 -- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d400cd8c0 msgr2=0x7f4d401f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d400cd8c0 0x7f4d401f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 -- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d40140950 msgr2=0x7f4d400c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d40140950 0x7f4d400c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 -- 172.21.15.153:0/2433451446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d38009a70 con 0x7f4d4006f850 2023-11-28T13:07:17.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d45107700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d40140950 0x7f4d400c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d3ffff700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d400cd8c0 0x7f4d401f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.411+0000 7f4d44906700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d400b5c00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c4b0 tx=0x7f4d3400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.412+0000 7f4d3d7fa700 1 -- 172.21.15.153:0/2433451446 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34011790 con 0x7f4d4006f850 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.412+0000 7f4d3d7fa700 1 -- 172.21.15.153:0/2433451446 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d340118f0 con 0x7f4d4006f850 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.412+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d400b3310 con 0x7f4d4006f850 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.412+0000 7f4d3d7fa700 1 -- 172.21.15.153:0/2433451446 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34011ca0 con 0x7f4d4006f850 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.412+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d400b3040 con 0x7f4d4006f850 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.412+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d400b35f0 con 0x7f4d4006f850 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.413+0000 7f4d3d7fa700 1 -- 172.21.15.153:0/2433451446 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d34010050 con 0x7f4d4006f850 2023-11-28T13:07:17.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.413+0000 7f4d3d7fa700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d240682f0 0x7f4d2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.413+0000 7f4d3d7fa700 1 -- 172.21.15.153:0/2433451446 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4d34085790 con 0x7f4d4006f850 2023-11-28T13:07:17.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.413+0000 7f4d3ffff700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d240682f0 0x7f4d2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.414+0000 7f4d3ffff700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d240682f0 0x7f4d2406a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c007fd0 tx=0x7f4d2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.418+0000 7f4d3d7fa700 1 -- 172.21.15.153:0/2433451446 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d34051f00 con 0x7f4d4006f850 2023-11-28T13:07:17.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.602+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d400c6630 con 0x7f4d4006f850 2023-11-28T13:07:17.605 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.603+0000 7f4d3d7fa700 1 -- 172.21.15.153:0/2433451446 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f4d34055520 con 0x7f4d4006f850 2023-11-28T13:07:17.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:17.605 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":65,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:04:00.505453+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:52.747638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:53.909181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.3a","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:17.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d240682f0 msgr2=0x7f4d2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d240682f0 0x7f4d2406a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c007fd0 tx=0x7f4d2c009040 comp rx=0 tx=0).stop 2023-11-28T13:07:17.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 msgr2=0x7f4d400b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d400b5c00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c4b0 tx=0x7f4d3400c870 comp rx=0 tx=0).stop 2023-11-28T13:07:17.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 shutdown_connections 2023-11-28T13:07:17.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4d240682f0 0x7f4d2406a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d4006f850 0x7f4d400b5c00 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d40140950 0x7f4d400c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 --2- 172.21.15.153:0/2433451446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d400cd8c0 0x7f4d401f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 >> 172.21.15.153:0/2433451446 conn(0x7f4d4006aed0 msgr2=0x7f4d400cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 shutdown_connections 2023-11-28T13:07:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.605+0000 7f4d46b6a700 1 -- 172.21.15.153:0/2433451446 wait complete. 2023-11-28T13:07:17.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:04.175316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:10.995932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:02:52.747638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:02:53.909181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:07:17.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:07:17.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.742+0000 7f40cdde5700 1 Processor -- start 2023-11-28T13:07:17.744 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.742+0000 7f40cdde5700 1 -- start start 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40cdde5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 0x7f40c8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40cdde5700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40c806dee0 0x7f40c806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40cdde5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c8071950 con 0x7f40c806cdc0 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40cdde5700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c8161550 con 0x7f40c806dee0 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40cdde5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c815d060 con 0x7f40c8071040 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40c806dee0 0x7f40c806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f40c806dee0 0x7f40c806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43346/0 (socket says 172.21.15.153:43346) 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c7fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f40c806cdc0 0x7f40c806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33282/0 (socket says 172.21.15.153:33282) 2023-11-28T13:07:17.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 -- 172.21.15.153:0/1312701086 learned_addr learned my addr 172.21.15.153:0/1312701086 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c77fe700 1 --2- 172.21.15.153:0/1312701086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 0x7f40c8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 -- 172.21.15.153:0/1312701086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 msgr2=0x7f40c8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 --2- 172.21.15.153:0/1312701086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 0x7f40c8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 -- 172.21.15.153:0/1312701086 >> v1:172.21.15.153:6789/0 conn(0x7f40c806cdc0 legacy=0x7f40c806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 -- 172.21.15.153:0/1312701086 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c815d340 con 0x7f40c806dee0 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c67fc700 1 -- 172.21.15.153:0/1312701086 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007000257 0 0) 0x7f40c8071950 con 0x7f40c806cdc0 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c6ffd700 1 --2- 172.21.15.153:0/1312701086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c806e350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0098f0 tx=0x7f40bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e63d6491fa8fcd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c67fc700 1 -- 172.21.15.153:0/1312701086 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00b8a0 con 0x7f40c806dee0 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c67fc700 1 -- 172.21.15.153:0/1312701086 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00ba00 con 0x7f40c806dee0 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.743+0000 7f40c67fc700 1 -- 172.21.15.153:0/1312701086 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00bd90 con 0x7f40c806dee0 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 -- 172.21.15.153:0/1312701086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 msgr2=0x7f40c806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 --2- 172.21.15.153:0/1312701086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c806e350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0098f0 tx=0x7f40bc009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 -- 172.21.15.153:0/1312701086 shutdown_connections 2023-11-28T13:07:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 --2- 172.21.15.153:0/1312701086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 0x7f40c8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 --2- 172.21.15.153:0/1312701086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c806e350 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 -- 172.21.15.153:0/1312701086 >> 172.21.15.153:0/1312701086 conn(0x7f40c80d0000 msgr2=0x7f40c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 -- 172.21.15.153:0/1312701086 shutdown_connections 2023-11-28T13:07:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.744+0000 7f40cdde5700 1 -- 172.21.15.153:0/1312701086 wait complete. 2023-11-28T13:07:17.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.745+0000 7f40cdde5700 1 Processor -- start 2023-11-28T13:07:17.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.745+0000 7f40cdde5700 1 -- start start 2023-11-28T13:07:17.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40cdde5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c806cdc0 0x7f40c815abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40cdde5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c81f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40cdde5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 0x7f40c81f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40cdde5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c815a150 con 0x7f40c806dee0 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40cdde5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c4860 con 0x7f40c806cdc0 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40cdde5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c4b40 con 0x7f40c8071040 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c81f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c81f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43354/0 (socket says 172.21.15.153:43354) 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 -- 172.21.15.153:0/2598306688 learned_addr learned my addr 172.21.15.153:0/2598306688 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 -- 172.21.15.153:0/2598306688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 msgr2=0x7f40c81f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 0x7f40c81f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 -- 172.21.15.153:0/2598306688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c806cdc0 msgr2=0x7f40c815abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c806cdc0 0x7f40c815abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.746+0000 7f40c6ffd700 1 -- 172.21.15.153:0/2598306688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc009550 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.747+0000 7f40c6ffd700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c81f33d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f40bc014040 tx=0x7f40bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.747+0000 7f40affff700 1 -- 172.21.15.153:0/2598306688 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc0036a0 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.747+0000 7f40affff700 1 -- 172.21.15.153:0/2598306688 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc01e070 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.747+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c815d900 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.747+0000 7f40affff700 1 -- 172.21.15.153:0/2598306688 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc003a20 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.747+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c815d620 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.748+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c806e890 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.748+0000 7f40affff700 1 -- 172.21.15.153:0/2598306688 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f40bc00f050 con 0x7f40c806dee0 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.748+0000 7f40affff700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40b40682f0 0x7f40b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.748+0000 7f40affff700 1 -- 172.21.15.153:0/2598306688 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f40bc020070 con 0x7f40c806dee0 2023-11-28T13:07:17.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.748+0000 7f40c77fe700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40b40682f0 0x7f40b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:17.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.749+0000 7f40c77fe700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40b40682f0 0x7f40b406a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f40c806efe0 tx=0x7f40b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:17.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.753+0000 7f40affff700 1 -- 172.21.15.153:0/2598306688 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40bc053a40 con 0x7f40c806dee0 2023-11-28T13:07:17.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.934+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40c815bf10 con 0x7f40c806dee0 2023-11-28T13:07:17.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.935+0000 7f40affff700 1 -- 172.21.15.153:0/2598306688 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40bc02a020 con 0x7f40c806dee0 2023-11-28T13:07:17.937 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2023-11-28T13:07:17.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40b40682f0 msgr2=0x7f40b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40b40682f0 0x7f40b406a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f40c806efe0 tx=0x7f40b8008040 comp rx=0 tx=0).stop 2023-11-28T13:07:17.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 msgr2=0x7f40c81f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:17.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c81f33d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f40bc014040 tx=0x7f40bc004750 comp rx=0 tx=0).stop 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 shutdown_connections 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40c8071040 0x7f40c81f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40c806cdc0 0x7f40c815abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f40b40682f0 0x7f40b406a7a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 --2- 172.21.15.153:0/2598306688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f40c806dee0 0x7f40c81f33d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 >> 172.21.15.153:0/2598306688 conn(0x7f40c80d0000 msgr2=0x7f40c815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 shutdown_connections 2023-11-28T13:07:17.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:17.937+0000 7f40cdde5700 1 -- 172.21.15.153:0/2598306688 wait complete. 2023-11-28T13:07:17.951 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:07:18.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.071+0000 7f4f69d3c700 1 Processor -- start 2023-11-28T13:07:18.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.072+0000 7f4f69d3c700 1 -- start start 2023-11-28T13:07:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.072+0000 7f4f69d3c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4f64071040 0x7f4f64071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.072+0000 7f4f69d3c700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f6406dee0 0x7f4f6406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.072+0000 7f4f69d3c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f64071950 con 0x7f4f6406cdc0 2023-11-28T13:07:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.072+0000 7f4f69d3c700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f64161550 con 0x7f4f6406dee0 2023-11-28T13:07:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.072+0000 7f4f69d3c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6415d060 con 0x7f4f64071040 2023-11-28T13:07:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.072+0000 7f4f62ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f6406dee0 0x7f4f6406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f62ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4f6406dee0 0x7f4f6406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43360/0 (socket says 172.21.15.153:43360) 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f62ffd700 1 -- 172.21.15.153:0/3885185931 learned_addr learned my addr 172.21.15.153:0/3885185931 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f63fff700 1 --1- 172.21.15.153:0/3885185931 >> v1:172.21.15.153:6789/0 conn(0x7f4f6406cdc0 0x7f4f6406d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33292/0 (socket says 172.21.15.153:33292) 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f637fe700 1 --2- 172.21.15.153:0/3885185931 >> v2:172.21.15.177:3300/0 conn(0x7f4f64071040 0x7f4f64071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f62ffd700 1 -- 172.21.15.153:0/3885185931 >> v2:172.21.15.177:3300/0 conn(0x7f4f64071040 msgr2=0x7f4f64071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f62ffd700 1 --2- 172.21.15.153:0/3885185931 >> v2:172.21.15.177:3300/0 conn(0x7f4f64071040 0x7f4f64071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f62ffd700 1 -- 172.21.15.153:0/3885185931 >> v1:172.21.15.153:6789/0 conn(0x7f4f6406cdc0 legacy=0x7f4f6406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f62ffd700 1 -- 172.21.15.153:0/3885185931 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6415d340 con 0x7f4f6406dee0 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f627fc700 1 -- 172.21.15.153:0/3885185931 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831912705 0 0) 0x7f4f64071950 con 0x7f4f6406cdc0 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f62ffd700 1 --2- 172.21.15.153:0/3885185931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f6406dee0 0x7f4f6406e330 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4f640d0f50 tx=0x7f4f54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c23a3b0bf11f737 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f627fc700 1 -- 172.21.15.153:0/3885185931 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400b870 con 0x7f4f6406dee0 2023-11-28T13:07:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f627fc700 1 -- 172.21.15.153:0/3885185931 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400b9d0 con 0x7f4f6406dee0 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.073+0000 7f4f627fc700 1 -- 172.21.15.153:0/3885185931 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400bd60 con 0x7f4f6406dee0 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3885185931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f6406dee0 msgr2=0x7f4f6406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3885185931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f6406dee0 0x7f4f6406e330 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4f640d0f50 tx=0x7f4f54009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3885185931 shutdown_connections 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3885185931 >> v2:172.21.15.177:3300/0 conn(0x7f4f64071040 0x7f4f64071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3885185931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f6406dee0 0x7f4f6406e330 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3885185931 >> 172.21.15.153:0/3885185931 conn(0x7f4f640d0000 msgr2=0x7f4f640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3885185931 shutdown_connections 2023-11-28T13:07:18.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.074+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3885185931 wait complete. 2023-11-28T13:07:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.075+0000 7f4f69d3c700 1 Processor -- start 2023-11-28T13:07:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.076+0000 7f4f69d3c700 1 -- start start 2023-11-28T13:07:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.076+0000 7f4f69d3c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f6406cdc0 0x7f4f6406a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.076+0000 7f4f69d3c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f6406dee0 0x7f4f640c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.076+0000 7f4f69d3c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64071040 0x7f4f640b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.076+0000 7f4f69d3c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640b3e80 con 0x7f4f64071040 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f69d3c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640b4160 con 0x7f4f6406dee0 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f69d3c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c3e10 con 0x7f4f6406cdc0 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64071040 0x7f4f640b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64071040 0x7f4f640b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43368/0 (socket says 172.21.15.153:43368) 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 -- 172.21.15.153:0/3965941245 learned_addr learned my addr 172.21.15.153:0/3965941245 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f62ffd700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f6406dee0 0x7f4f640c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f637fe700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f6406cdc0 0x7f4f6406a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 -- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f6406cdc0 msgr2=0x7f4f6406a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f6406cdc0 0x7f4f6406a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 -- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f6406dee0 msgr2=0x7f4f640c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f6406dee0 0x7f4f640c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 -- 172.21.15.153:0/3965941245 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f54009550 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f63fff700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64071040 0x7f4f640b45f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00cb90 tx=0x7f4f5c00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f5bfff700 1 -- 172.21.15.153:0/3965941245 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00aa60 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f5bfff700 1 -- 172.21.15.153:0/3965941245 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c00abc0 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f5bfff700 1 -- 172.21.15.153:0/3965941245 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0070d0 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6415d900 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.077+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6415d620 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.078+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6406e870 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.079+0000 7f4f5bfff700 1 -- 172.21.15.153:0/3965941245 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f5c01a030 con 0x7f4f64071040 2023-11-28T13:07:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.079+0000 7f4f5bfff700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f440683c0 0x7f4f4406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.079+0000 7f4f5bfff700 1 -- 172.21.15.153:0/3965941245 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4f5c085a50 con 0x7f4f64071040 2023-11-28T13:07:18.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.079+0000 7f4f637fe700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f440683c0 0x7f4f4406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.084 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.080+0000 7f4f637fe700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f440683c0 0x7f4f4406a870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c006b20 tx=0x7f4f4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.085 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.083+0000 7f4f5bfff700 1 -- 172.21.15.153:0/3965941245 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4f5c0521b0 con 0x7f4f64071040 2023-11-28T13:07:18.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.267+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4f640b42a0 con 0x7f4f64071040 2023-11-28T13:07:18.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.267+0000 7f4f5bfff700 1 -- 172.21.15.153:0/3965941245 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4f5c018020 con 0x7f4f64071040 2023-11-28T13:07:18.269 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f440683c0 msgr2=0x7f4f4406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f440683c0 0x7f4f4406a870 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c006b20 tx=0x7f4f4c008040 comp rx=0 tx=0).stop 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64071040 msgr2=0x7f4f640b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64071040 0x7f4f640b45f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00cb90 tx=0x7f4f5c00cea0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 shutdown_connections 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4f440683c0 0x7f4f4406a870 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f64071040 0x7f4f640b45f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.272 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4f6406cdc0 0x7f4f6406a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 --2- 172.21.15.153:0/3965941245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f6406dee0 0x7f4f640c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 >> 172.21.15.153:0/3965941245 conn(0x7f4f640d0000 msgr2=0x7f4f6406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:18.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 shutdown_connections 2023-11-28T13:07:18.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.269+0000 7f4f69d3c700 1 -- 172.21.15.153:0/3965941245 wait complete. 2023-11-28T13:07:18.283 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:07:18.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.404+0000 7ff6addcd700 1 Processor -- start 2023-11-28T13:07:18.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.405+0000 7ff6addcd700 1 -- start start 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.405+0000 7ff6addcd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.405+0000 7ff6addcd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff6a806dee0 0x7ff6a806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.405+0000 7ff6addcd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a8071950 con 0x7ff6a806cdc0 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.405+0000 7ff6addcd700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a8161550 con 0x7ff6a806dee0 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.405+0000 7ff6addcd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a815d060 con 0x7ff6a8071040 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a6ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff6a806dee0 0x7ff6a806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.409 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a7fff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff6a806cdc0 0x7ff6a806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:38154/0 (socket says 172.21.15.153:38154) 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a6ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff6a806dee0 0x7ff6a806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.153:52838/0 (socket says 172.21.15.153:52838) 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:60982/0 (socket says 172.21.15.153:60982) 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a7fff700 1 -- 172.21.15.153:0/3221483032 learned_addr learned my addr 172.21.15.153:0/3221483032 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a77fe700 1 -- 172.21.15.153:0/3221483032 >> v2:172.21.15.177:3300/0 conn(0x7ff6a806dee0 msgr2=0x7ff6a806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a77fe700 1 --2- 172.21.15.153:0/3221483032 >> v2:172.21.15.177:3300/0 conn(0x7ff6a806dee0 0x7ff6a806e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a77fe700 1 -- 172.21.15.153:0/3221483032 >> v1:172.21.15.177:6789/0 conn(0x7ff6a806cdc0 legacy=0x7ff6a806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a77fe700 1 -- 172.21.15.153:0/3221483032 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a815d340 con 0x7ff6a8071040 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a77fe700 1 --2- 172.21.15.153:0/3221483032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a8071410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009840 tx=0x7ff69c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ab275c6c9dba008 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a67fc700 1 -- 172.21.15.153:0/3221483032 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011830 con 0x7ff6a8071040 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a67fc700 1 -- 172.21.15.153:0/3221483032 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c011990 con 0x7ff6a8071040 2023-11-28T13:07:18.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.406+0000 7ff6a67fc700 1 -- 172.21.15.153:0/3221483032 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011d20 con 0x7ff6a8071040 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 -- 172.21.15.153:0/3221483032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 msgr2=0x7ff6a8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 --2- 172.21.15.153:0/3221483032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a8071410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009840 tx=0x7ff69c009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 -- 172.21.15.153:0/3221483032 shutdown_connections 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 --2- 172.21.15.153:0/3221483032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a8071410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 --2- 172.21.15.153:0/3221483032 >> v2:172.21.15.177:3300/0 conn(0x7ff6a806dee0 0x7ff6a806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 -- 172.21.15.153:0/3221483032 >> 172.21.15.153:0/3221483032 conn(0x7ff6a80d0000 msgr2=0x7ff6a80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 -- 172.21.15.153:0/3221483032 shutdown_connections 2023-11-28T13:07:18.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.407+0000 7ff6addcd700 1 -- 172.21.15.153:0/3221483032 wait complete. 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.408+0000 7ff6addcd700 1 Processor -- start 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.408+0000 7ff6addcd700 1 -- start start 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6addcd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a806cdc0 0x7ff6a80b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6addcd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a806dee0 0x7ff6a8152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6addcd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a80c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6addcd700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b51f0 con 0x7ff6a806cdc0 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6addcd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b4530 con 0x7ff6a806dee0 2023-11-28T13:07:18.414 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6addcd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b3f70 con 0x7ff6a8071040 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a806cdc0 0x7ff6a80b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a806cdc0 0x7ff6a80b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43378/0 (socket says 172.21.15.153:43378) 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6a77fe700 1 -- 172.21.15.153:0/2733593512 learned_addr learned my addr 172.21.15.153:0/2733593512 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6a7fff700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a80c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.409+0000 7ff6a6ffd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a806dee0 0x7ff6a8152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6a77fe700 1 -- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 msgr2=0x7ff6a80c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6a77fe700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a80c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6a77fe700 1 -- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a806dee0 msgr2=0x7ff6a8152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6a77fe700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a806dee0 0x7ff6a8152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6a77fe700 1 -- 172.21.15.153:0/2733593512 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0094a0 con 0x7ff6a806cdc0 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6a77fe700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a806cdc0 0x7ff6a80b5680 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff69c014040 tx=0x7ff69c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.415 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff68ffff700 1 -- 172.21.15.153:0/2733593512 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c004400 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff68ffff700 1 -- 172.21.15.153:0/2733593512 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c01f070 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a815d900 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff68ffff700 1 -- 172.21.15.153:0/2733593512 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c0038c0 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.410+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a815d620 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.411+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a806e890 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.411+0000 7ff68ffff700 1 -- 172.21.15.153:0/2733593512 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff69c02a020 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.412+0000 7ff68ffff700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff6940682f0 0x7ff69406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.412+0000 7ff6a6ffd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff6940682f0 0x7ff69406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.412+0000 7ff68ffff700 1 -- 172.21.15.153:0/2733593512 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff69c087050 con 0x7ff6a806cdc0 2023-11-28T13:07:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.412+0000 7ff6a6ffd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff6940682f0 0x7ff69406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff6a815e7a0 tx=0x7ff698009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.416+0000 7ff68ffff700 1 -- 172.21.15.153:0/2733593512 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff69c053850 con 0x7ff6a806cdc0 2023-11-28T13:07:18.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.598+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6a8158a90 con 0x7ff6a806cdc0 2023-11-28T13:07:18.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.599+0000 7ff68ffff700 1 -- 172.21.15.153:0/2733593512 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff69c056e70 con 0x7ff6a806cdc0 2023-11-28T13:07:18.601 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff6940682f0 msgr2=0x7ff69406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff6940682f0 0x7ff69406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff6a815e7a0 tx=0x7ff698009210 comp rx=0 tx=0).stop 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a806cdc0 msgr2=0x7ff6a80b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a806cdc0 0x7ff6a80b5680 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff69c014040 tx=0x7ff69c004750 comp rx=0 tx=0).stop 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 shutdown_connections 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff6940682f0 0x7ff69406a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6a806cdc0 0x7ff6a80b5680 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6a8071040 0x7ff6a80c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 --2- 172.21.15.153:0/2733593512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6a806dee0 0x7ff6a8152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 >> 172.21.15.153:0/2733593512 conn(0x7ff6a80d0000 msgr2=0x7ff6a806abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:18.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 shutdown_connections 2023-11-28T13:07:18.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.601+0000 7ff6addcd700 1 -- 172.21.15.153:0/2733593512 wait complete. 2023-11-28T13:07:18.615 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:07:18.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.736+0000 7f6b8568c700 1 Processor -- start 2023-11-28T13:07:18.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.736+0000 7f6b8568c700 1 -- start start 2023-11-28T13:07:18.740 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b8568c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6b8015cd30 0x7f6b80161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b8568c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8006a980 0x7f6b800d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b8568c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b800d0840 con 0x7f6b800cfda0 2023-11-28T13:07:18.741 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b8568c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b800d0b20 con 0x7f6b8015cd30 2023-11-28T13:07:18.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b8568c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80160670 con 0x7f6b8006a980 2023-11-28T13:07:18.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7effd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6b8015cd30 0x7f6b80161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8006a980 0x7f6b800d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7e7fc700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f6b800cfda0 0x7f6b8006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:38164/0 (socket says 172.21.15.153:38164) 2023-11-28T13:07:18.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8006a980 0x7f6b800d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.153:32768/0 (socket says 172.21.15.153:32768) 2023-11-28T13:07:18.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7e7fc700 1 -- 172.21.15.153:0/3674005794 learned_addr learned my addr 172.21.15.153:0/3674005794 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:18.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7dffb700 1 -- 172.21.15.153:0/3674005794 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135456506 0 0) 0x7f6b800d0840 con 0x7f6b800cfda0 2023-11-28T13:07:18.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7dffb700 1 -- 172.21.15.153:0/3674005794 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b680035b0 con 0x7f6b800cfda0 2023-11-28T13:07:18.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7f7fe700 1 -- 172.21.15.153:0/3674005794 >> v2:172.21.15.177:3300/0 conn(0x7f6b8015cd30 msgr2=0x7f6b80161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:18.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7f7fe700 1 --2- 172.21.15.153:0/3674005794 >> v2:172.21.15.177:3300/0 conn(0x7f6b8015cd30 0x7f6b80161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7f7fe700 1 -- 172.21.15.153:0/3674005794 >> v1:172.21.15.177:6789/0 conn(0x7f6b800cfda0 legacy=0x7f6b8006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7f7fe700 1 -- 172.21.15.153:0/3674005794 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80160950 con 0x7f6b8006a980 2023-11-28T13:07:18.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.737+0000 7f6b7effd700 1 --2- 172.21.15.153:0/3674005794 >> v2:172.21.15.177:3300/0 conn(0x7f6b8015cd30 0x7f6b80161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:18.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b7f7fe700 1 --2- 172.21.15.153:0/3674005794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8006a980 0x7f6b800d0240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009dc0 tx=0x7f6b7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf2a46d3e47ee79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b7dffb700 1 -- 172.21.15.153:0/3674005794 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7800e070 con 0x7f6b8006a980 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b7dffb700 1 -- 172.21.15.153:0/3674005794 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7800bae0 con 0x7f6b8006a980 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b7dffb700 1 -- 172.21.15.153:0/3674005794 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7800be90 con 0x7f6b8006a980 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 -- 172.21.15.153:0/3674005794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8006a980 msgr2=0x7f6b800d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 --2- 172.21.15.153:0/3674005794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8006a980 0x7f6b800d0240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009dc0 tx=0x7f6b7800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 -- 172.21.15.153:0/3674005794 shutdown_connections 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 --2- 172.21.15.153:0/3674005794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8006a980 0x7f6b800d0240 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 --2- 172.21.15.153:0/3674005794 >> v2:172.21.15.177:3300/0 conn(0x7f6b8015cd30 0x7f6b80161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 -- 172.21.15.153:0/3674005794 >> 172.21.15.153:0/3674005794 conn(0x7f6b8006b380 msgr2=0x7f6b8006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:18.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 -- 172.21.15.153:0/3674005794 shutdown_connections 2023-11-28T13:07:18.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.738+0000 7f6b8568c700 1 -- 172.21.15.153:0/3674005794 wait complete. 2023-11-28T13:07:18.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 Processor -- start 2023-11-28T13:07:18.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 -- start start 2023-11-28T13:07:18.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b8006a980 0x7f6b800b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b800cfda0 0x7f6b800b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8015cd30 0x7f6b80151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800b5cb0 con 0x7f6b800cfda0 2023-11-28T13:07:18.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800b3d90 con 0x7f6b8006a980 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.740+0000 7f6b8568c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800b4070 con 0x7f6b8015cd30 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b800cfda0 0x7f6b800b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b800cfda0 0x7f6b800b5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43394/0 (socket says 172.21.15.153:43394) 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 -- 172.21.15.153:0/2980082481 learned_addr learned my addr 172.21.15.153:0/2980082481 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7effd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b8006a980 0x7f6b800b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7f7fe700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8015cd30 0x7f6b80151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 -- 172.21.15.153:0/2980082481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8015cd30 msgr2=0x7f6b80151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8015cd30 0x7f6b80151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 -- 172.21.15.153:0/2980082481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b8006a980 msgr2=0x7f6b800b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.752 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b8006a980 0x7f6b800b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 -- 172.21.15.153:0/2980082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b740094a0 con 0x7f6b800cfda0 2023-11-28T13:07:18.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b7e7fc700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b800cfda0 0x7f6b800b5770 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c700 tx=0x7f6b7000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b677fe700 1 -- 172.21.15.153:0/2980082481 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70005d70 con 0x7f6b800cfda0 2023-11-28T13:07:18.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b677fe700 1 -- 172.21.15.153:0/2980082481 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b700070d0 con 0x7f6b800cfda0 2023-11-28T13:07:18.753 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b78009a70 con 0x7f6b800cfda0 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b677fe700 1 -- 172.21.15.153:0/2980082481 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70007460 con 0x7f6b800cfda0 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.741+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b800cc280 con 0x7f6b800cfda0 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.742+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b80160c30 con 0x7f6b800cfda0 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.743+0000 7f6b677fe700 1 -- 172.21.15.153:0/2980082481 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6b70019030 con 0x7f6b800cfda0 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.743+0000 7f6b677fe700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6b6c0682f0 0x7f6b6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.743+0000 7f6b677fe700 1 -- 172.21.15.153:0/2980082481 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f6b700850c0 con 0x7f6b800cfda0 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.743+0000 7f6b7effd700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6b6c0682f0 0x7f6b6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:18.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.743+0000 7f6b7effd700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6b6c0682f0 0x7f6b6c06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009cd0 tx=0x7f6b74009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:18.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.747+0000 7f6b677fe700 1 -- 172.21.15.153:0/2980082481 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6b700517a0 con 0x7f6b800cfda0 2023-11-28T13:07:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.927+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b80070de0 con 0x7f6b800cfda0 2023-11-28T13:07:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.927+0000 7f6b677fe700 1 -- 172.21.15.153:0/2980082481 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b70017070 con 0x7f6b800cfda0 2023-11-28T13:07:18.929 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 64 2023-11-28T13:07:18.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6b6c0682f0 msgr2=0x7f6b6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6b6c0682f0 0x7f6b6c06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009cd0 tx=0x7f6b74009210 comp rx=0 tx=0).stop 2023-11-28T13:07:18.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b800cfda0 msgr2=0x7f6b800b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:18.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b800cfda0 0x7f6b800b5770 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c700 tx=0x7f6b7000ca10 comp rx=0 tx=0).stop 2023-11-28T13:07:18.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 shutdown_connections 2023-11-28T13:07:18.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b8015cd30 0x7f6b80151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b8006a980 0x7f6b800b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f6b6c0682f0 0x7f6b6c06a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 --2- 172.21.15.153:0/2980082481 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b800cfda0 0x7f6b800b5770 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:18.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 >> 172.21.15.153:0/2980082481 conn(0x7f6b8006b380 msgr2=0x7f6b8006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:18.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 shutdown_connections 2023-11-28T13:07:18.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:18.929+0000 7f6b8568c700 1 -- 172.21.15.153:0/2980082481 wait complete. 2023-11-28T13:07:18.943 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:19.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.063+0000 7f3d7434f700 1 Processor -- start 2023-11-28T13:07:19.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.063+0000 7f3d7434f700 1 -- start start 2023-11-28T13:07:19.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.064+0000 7f3d7434f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3d6c071040 0x7f3d6c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.064+0000 7f3d7434f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c160540 0x7f3d6c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.064+0000 7f3d7434f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c071950 con 0x7f3d6c06dee0 2023-11-28T13:07:19.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.064+0000 7f3d7434f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c15d060 con 0x7f3d6c071040 2023-11-28T13:07:19.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.064+0000 7f3d7434f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c15d340 con 0x7f3d6c160540 2023-11-28T13:07:19.068 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.064+0000 7f3d720eb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3d6c071040 0x7f3d6c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d720eb700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f3d6c071040 0x7f3d6c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43400/0 (socket says 172.21.15.153:43400) 2023-11-28T13:07:19.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d720eb700 1 -- 172.21.15.153:0/2055089629 learned_addr learned my addr 172.21.15.153:0/2055089629 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:19.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d728ec700 1 --2- 172.21.15.153:0/2055089629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c160540 0x7f3d6c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d720eb700 1 -- 172.21.15.153:0/2055089629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c160540 msgr2=0x7f3d6c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d720eb700 1 --2- 172.21.15.153:0/2055089629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c160540 0x7f3d6c1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d720eb700 1 -- 172.21.15.153:0/2055089629 >> v1:172.21.15.177:6789/0 conn(0x7f3d6c06dee0 legacy=0x7f3d6c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.069 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d720eb700 1 -- 172.21.15.153:0/2055089629 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c15d620 con 0x7f3d6c071040 2023-11-28T13:07:19.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d720eb700 1 --2- 172.21.15.153:0/2055089629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c071040 0x7f3d6c071410 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0098f0 tx=0x7f3d5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b40c8ac07714fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d710e9700 1 -- 172.21.15.153:0/2055089629 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c004230 con 0x7f3d6c071040 2023-11-28T13:07:19.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d710e9700 1 -- 172.21.15.153:0/2055089629 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c004390 con 0x7f3d6c071040 2023-11-28T13:07:19.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.065+0000 7f3d710e9700 1 -- 172.21.15.153:0/2055089629 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0036a0 con 0x7f3d6c071040 2023-11-28T13:07:19.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 -- 172.21.15.153:0/2055089629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c071040 msgr2=0x7f3d6c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 --2- 172.21.15.153:0/2055089629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c071040 0x7f3d6c071410 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0098f0 tx=0x7f3d5c009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:19.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 -- 172.21.15.153:0/2055089629 shutdown_connections 2023-11-28T13:07:19.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 --2- 172.21.15.153:0/2055089629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c160540 0x7f3d6c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 --2- 172.21.15.153:0/2055089629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c071040 0x7f3d6c071410 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 -- 172.21.15.153:0/2055089629 >> 172.21.15.153:0/2055089629 conn(0x7f3d6c0d0000 msgr2=0x7f3d6c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:19.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 -- 172.21.15.153:0/2055089629 shutdown_connections 2023-11-28T13:07:19.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.066+0000 7f3d7434f700 1 -- 172.21.15.153:0/2055089629 wait complete. 2023-11-28T13:07:19.073 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.067+0000 7f3d7434f700 1 Processor -- start 2023-11-28T13:07:19.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.067+0000 7f3d7434f700 1 -- start start 2023-11-28T13:07:19.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c06dee0 0x7f3d6c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c071040 0x7f3d6c158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c160540 0x7f3d6c15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c15b450 con 0x7f3d6c06dee0 2023-11-28T13:07:19.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c6510 con 0x7f3d6c160540 2023-11-28T13:07:19.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c3c40 con 0x7f3d6c071040 2023-11-28T13:07:19.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c06dee0 0x7f3d6c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c06dee0 0x7f3d6c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43404/0 (socket says 172.21.15.153:43404) 2023-11-28T13:07:19.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 -- 172.21.15.153:0/1464029348 learned_addr learned my addr 172.21.15.153:0/1464029348 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:19.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d718ea700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c071040 0x7f3d6c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d728ec700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c160540 0x7f3d6c15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 -- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c071040 msgr2=0x7f3d6c158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c071040 0x7f3d6c158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 -- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c160540 msgr2=0x7f3d6c15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c160540 0x7f3d6c15a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 -- 172.21.15.153:0/1464029348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c009550 con 0x7f3d6c06dee0 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d720eb700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c06dee0 0x7f3d6c0b2e30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009840 tx=0x7f3d5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d62ffd700 1 -- 172.21.15.153:0/1464029348 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00b860 con 0x7f3d6c06dee0 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d62ffd700 1 -- 172.21.15.153:0/1464029348 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c00ba00 con 0x7f3d6c06dee0 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c15ccd0 con 0x7f3d6c06dee0 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d62ffd700 1 -- 172.21.15.153:0/1464029348 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c00bdb0 con 0x7f3d6c06dee0 2023-11-28T13:07:19.076 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.068+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c15d900 con 0x7f3d6c06dee0 2023-11-28T13:07:19.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.069+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c06a270 con 0x7f3d6c06dee0 2023-11-28T13:07:19.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.070+0000 7f3d62ffd700 1 -- 172.21.15.153:0/1464029348 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d5c017020 con 0x7f3d6c06dee0 2023-11-28T13:07:19.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.070+0000 7f3d62ffd700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d500682f0 0x7f3d5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.070+0000 7f3d718ea700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d500682f0 0x7f3d5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.070+0000 7f3d62ffd700 1 -- 172.21.15.153:0/1464029348 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f3d5c087550 con 0x7f3d6c06dee0 2023-11-28T13:07:19.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.071+0000 7f3d718ea700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d500682f0 0x7f3d5006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009870 tx=0x7f3d68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.077 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.074+0000 7f3d62ffd700 1 -- 172.21.15.153:0/1464029348 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d5c053d20 con 0x7f3d6c06dee0 2023-11-28T13:07:19.303 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.302+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d6c0720a0 con 0x7f3d6c06dee0 2023-11-28T13:07:19.304 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.303+0000 7f3d62ffd700 1 -- 172.21.15.153:0/1464029348 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f3d5c057340 con 0x7f3d6c06dee0 2023-11-28T13:07:19.304 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:19.304 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:03:11.406430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:19.306 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d500682f0 msgr2=0x7f3d5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d500682f0 0x7f3d5006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009870 tx=0x7f3d68008040 comp rx=0 tx=0).stop 2023-11-28T13:07:19.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c06dee0 msgr2=0x7f3d6c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c06dee0 0x7f3d6c0b2e30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009840 tx=0x7f3d5c004750 comp rx=0 tx=0).stop 2023-11-28T13:07:19.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 shutdown_connections 2023-11-28T13:07:19.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f3d500682f0 0x7f3d5006a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d6c06dee0 0x7f3d6c0b2e30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.307 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c160540 0x7f3d6c15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 --2- 172.21.15.153:0/1464029348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c071040 0x7f3d6c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 >> 172.21.15.153:0/1464029348 conn(0x7f3d6c0d0000 msgr2=0x7f3d6c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:19.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 shutdown_connections 2023-11-28T13:07:19.308 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.305+0000 7f3d7434f700 1 -- 172.21.15.153:0/1464029348 wait complete. 2023-11-28T13:07:19.308 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2023-11-28T13:07:19.318 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:19.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.437+0000 7ffb3ecbe700 1 Processor -- start 2023-11-28T13:07:19.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ecbe700 1 -- start start 2023-11-28T13:07:19.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ecbe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb380cd8c0 0x7ffb38140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ecbe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffb380cdc90 0x7ffb380b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ecbe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ecbe700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3813ff00 con 0x7ffb380cd8c0 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ecbe700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb381404c0 con 0x7ffb380cdc90 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ecbe700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815aa80 con 0x7ffb380b2e70 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ca5a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb380cd8c0 0x7ffb38140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ca5a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffb380cd8c0 0x7ffb38140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43412/0 (socket says 172.21.15.153:43412) 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.438+0000 7ffb3ca5a700 1 -- 172.21.15.153:0/4216324406 learned_addr learned my addr 172.21.15.153:0/4216324406 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb37fff700 1 --2- 172.21.15.153:0/4216324406 >> v2:172.21.15.177:3300/0 conn(0x7ffb380cdc90 0x7ffb380b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb3d25b700 1 --2- 172.21.15.153:0/4216324406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb3ca5a700 1 -- 172.21.15.153:0/4216324406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 msgr2=0x7ffb3806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb3ca5a700 1 --2- 172.21.15.153:0/4216324406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3806f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb3ca5a700 1 -- 172.21.15.153:0/4216324406 >> v2:172.21.15.177:3300/0 conn(0x7ffb380cdc90 msgr2=0x7ffb380b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb3ca5a700 1 --2- 172.21.15.153:0/4216324406 >> v2:172.21.15.177:3300/0 conn(0x7ffb380cdc90 0x7ffb380b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb3ca5a700 1 -- 172.21.15.153:0/4216324406 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380b3310 con 0x7ffb380cd8c0 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb37fff700 1 --2- 172.21.15.153:0/4216324406 >> v2:172.21.15.177:3300/0 conn(0x7ffb380cdc90 0x7ffb380b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb3ca5a700 1 --2- 172.21.15.153:0/4216324406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cd8c0 0x7ffb38140950 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffb38071c40 tx=0x7ffb2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e796ee9d2dc1e24 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb377fe700 1 -- 172.21.15.153:0/4216324406 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c005e30 con 0x7ffb380cd8c0 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb377fe700 1 -- 172.21.15.153:0/4216324406 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c004030 con 0x7ffb380cd8c0 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.439+0000 7ffb377fe700 1 -- 172.21.15.153:0/4216324406 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0043e0 con 0x7ffb380cd8c0 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/4216324406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cd8c0 msgr2=0x7ffb38140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/4216324406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cd8c0 0x7ffb38140950 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffb38071c40 tx=0x7ffb2c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.443 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/4216324406 shutdown_connections 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/4216324406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/4216324406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cd8c0 0x7ffb38140950 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/4216324406 >> v2:172.21.15.177:3300/0 conn(0x7ffb380cdc90 0x7ffb380b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/4216324406 >> 172.21.15.153:0/4216324406 conn(0x7ffb3806aed0 msgr2=0x7ffb3806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/4216324406 shutdown_connections 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.440+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/4216324406 wait complete. 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.441+0000 7ffb3ecbe700 1 Processor -- start 2023-11-28T13:07:19.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.441+0000 7ffb3ecbe700 1 -- start start 2023-11-28T13:07:19.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3ecbe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3ecbe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb380cd8c0 0x7ffb3815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.447 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3ecbe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cdc90 0x7ffb380c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3ecbe700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815b580 con 0x7ffb380cdc90 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3ecbe700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb381573a0 con 0x7ffb380cd8c0 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3ecbe700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb380b3da0 con 0x7ffb380b2e70 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cdc90 0x7ffb380c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cdc90 0x7ffb380c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43422/0 (socket says 172.21.15.153:43422) 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 -- 172.21.15.153:0/2688629782 learned_addr learned my addr 172.21.15.153:0/2688629782 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3ca5a700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb37fff700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb380cd8c0 0x7ffb3815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 -- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 msgr2=0x7ffb3815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3815af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 -- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb380cd8c0 msgr2=0x7ffb3815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb380cd8c0 0x7ffb3815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 -- 172.21.15.153:0/2688629782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c009500 con 0x7ffb380cdc90 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb3d25b700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cdc90 0x7ffb380c5a30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000cc00 tx=0x7ffb3000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb357fa700 1 -- 172.21.15.153:0/2688629782 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000ab30 con 0x7ffb380cdc90 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.442+0000 7ffb357fa700 1 -- 172.21.15.153:0/2688629782 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3000ac90 con 0x7ffb380cdc90 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.443+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb38140150 con 0x7ffb380cdc90 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.443+0000 7ffb357fa700 1 -- 172.21.15.153:0/2688629782 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30003a20 con 0x7ffb380cdc90 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.443+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb380b35f0 con 0x7ffb380cdc90 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.443+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb380ce100 con 0x7ffb380cdc90 2023-11-28T13:07:19.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.444+0000 7ffb357fa700 1 -- 172.21.15.153:0/2688629782 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb3001f070 con 0x7ffb380cdc90 2023-11-28T13:07:19.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.444+0000 7ffb357fa700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb1c0682f0 0x7ffb1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.444+0000 7ffb357fa700 1 -- 172.21.15.153:0/2688629782 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ffb30085e60 con 0x7ffb380cdc90 2023-11-28T13:07:19.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.444+0000 7ffb3ca5a700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb1c0682f0 0x7ffb1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.445+0000 7ffb3ca5a700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb1c0682f0 0x7ffb1c06a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffb38071c40 tx=0x7ffb2c008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.450 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.449+0000 7ffb357fa700 1 -- 172.21.15.153:0/2688629782 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb30052540 con 0x7ffb380cdc90 2023-11-28T13:07:19.636 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.635+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb380d08c0 con 0x7ffb380cdc90 2023-11-28T13:07:19.638 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.637+0000 7ffb357fa700 1 -- 172.21.15.153:0/2688629782 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18209 (secure 0 0 0) 0x7ffb3001d070 con 0x7ffb380cdc90 2023-11-28T13:07:19.638 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:19.639 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":65,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:04:00.505453+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:52.747638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:53.909181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.3a","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:19.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb1c0682f0 msgr2=0x7ffb1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb1c0682f0 0x7ffb1c06a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffb38071c40 tx=0x7ffb2c008070 comp rx=0 tx=0).stop 2023-11-28T13:07:19.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cdc90 msgr2=0x7ffb380c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cdc90 0x7ffb380c5a30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000cc00 tx=0x7ffb3000a1f0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 shutdown_connections 2023-11-28T13:07:19.641 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffb1c0682f0 0x7ffb1c06a7a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb380cdc90 0x7ffb380c5a30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb380b2e70 0x7ffb3815af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 --2- 172.21.15.153:0/2688629782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb380cd8c0 0x7ffb3815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 >> 172.21.15.153:0/2688629782 conn(0x7ffb3806aed0 msgr2=0x7ffb380d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:19.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 shutdown_connections 2023-11-28T13:07:19.642 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.639+0000 7ffb3ecbe700 1 -- 172.21.15.153:0/2688629782 wait complete. 2023-11-28T13:07:19.652 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:07:19.652 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:19.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.772+0000 7f02bb1c1700 1 Processor -- start 2023-11-28T13:07:19.772 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.772+0000 7f02bb1c1700 1 -- start start 2023-11-28T13:07:19.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.772+0000 7f02bb1c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b4140950 0x7f02b40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.772+0000 7f02bb1c1700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f02b40b2e70 0x7f02b406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.772+0000 7f02bb1c1700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b413ff00 con 0x7f02b40cd8c0 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.772+0000 7f02bb1c1700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b41404c0 con 0x7f02b40b2e70 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.772+0000 7f02bb1c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4140dc0 con 0x7f02b4140950 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b975e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f02b40b2e70 0x7f02b406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b8f5d700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f02b40cd8c0 0x7f02b40cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33358/0 (socket says 172.21.15.153:33358) 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b8f5d700 1 -- 172.21.15.153:0/2577079286 learned_addr learned my addr 172.21.15.153:0/2577079286 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b3fff700 1 --2- 172.21.15.153:0/2577079286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b4140950 0x7f02b40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b975e700 1 -- 172.21.15.153:0/2577079286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b4140950 msgr2=0x7f02b40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b975e700 1 --2- 172.21.15.153:0/2577079286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b4140950 0x7f02b40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b975e700 1 -- 172.21.15.153:0/2577079286 >> v1:172.21.15.153:6789/0 conn(0x7f02b40cd8c0 legacy=0x7f02b40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b975e700 1 -- 172.21.15.153:0/2577079286 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b415aa80 con 0x7f02b40b2e70 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b975e700 1 --2- 172.21.15.153:0/2577079286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b40b2e70 0x7f02b406f850 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009d90 tx=0x7f02ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b550c0103ba2bd72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b37fe700 1 -- 172.21.15.153:0/2577079286 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00e070 con 0x7f02b40b2e70 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b37fe700 1 -- 172.21.15.153:0/2577079286 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac00bbd0 con 0x7f02b40b2e70 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.773+0000 7f02b37fe700 1 -- 172.21.15.153:0/2577079286 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac012430 con 0x7f02b40b2e70 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 -- 172.21.15.153:0/2577079286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b40b2e70 msgr2=0x7f02b406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/2577079286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b40b2e70 0x7f02b406f850 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009d90 tx=0x7f02ac00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 -- 172.21.15.153:0/2577079286 shutdown_connections 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/2577079286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b40b2e70 0x7f02b406f850 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/2577079286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b4140950 0x7f02b40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 -- 172.21.15.153:0/2577079286 >> 172.21.15.153:0/2577079286 conn(0x7f02b406aed0 msgr2=0x7f02b406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:19.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 -- 172.21.15.153:0/2577079286 shutdown_connections 2023-11-28T13:07:19.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.774+0000 7f02bb1c1700 1 -- 172.21.15.153:0/2577079286 wait complete. 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.775+0000 7f02bb1c1700 1 Processor -- start 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.775+0000 7f02bb1c1700 1 -- start start 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.776+0000 7f02bb1c1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b40b2e70 0x7f02b40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02bb1c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b40cd8c0 0x7f02b415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02bb1c1700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4140950 0x7f02b415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02bb1c1700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b415d010 con 0x7f02b4140950 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02bb1c1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41594b0 con 0x7f02b40b2e70 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02bb1c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40c6440 con 0x7f02b40cd8c0 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4140950 0x7f02b415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4140950 0x7f02b415c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43436/0 (socket says 172.21.15.153:43436) 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 -- 172.21.15.153:0/3621826202 learned_addr learned my addr 172.21.15.153:0/3621826202 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b3fff700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b40cd8c0 0x7f02b415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b8f5d700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b40b2e70 0x7f02b40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 -- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b40cd8c0 msgr2=0x7f02b415bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b40cd8c0 0x7f02b415bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 -- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b40b2e70 msgr2=0x7f02b40cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b40b2e70 0x7f02b40cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 -- 172.21.15.153:0/3621826202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac009a70 con 0x7f02b4140950 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b975e700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4140950 0x7f02b415c160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f02ac014040 tx=0x7f02ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b17fa700 1 -- 172.21.15.153:0/3621826202 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac003ea0 con 0x7f02b4140950 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b17fa700 1 -- 172.21.15.153:0/3621826202 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac020070 con 0x7f02b4140950 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b40b35f0 con 0x7f02b4140950 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02b17fa700 1 -- 172.21.15.153:0/3621826202 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0098f0 con 0x7f02b4140950 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.777+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b40b3310 con 0x7f02b4140950 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.778+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b4140150 con 0x7f02b4140950 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.779+0000 7f02b17fa700 1 -- 172.21.15.153:0/3621826202 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f02ac004040 con 0x7f02b4140950 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.779+0000 7f02b17fa700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f02980682f0 0x7f029806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.779+0000 7f02b17fa700 1 -- 172.21.15.153:0/3621826202 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f02ac022070 con 0x7f02b4140950 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.779+0000 7f02b8f5d700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f02980682f0 0x7f029806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.780+0000 7f02b8f5d700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f02980682f0 0x7f029806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02a8007fd0 tx=0x7f02a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:19.784+0000 7f02b17fa700 1 -- 172.21.15.153:0/3621826202 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f02ac054830 con 0x7f02b4140950 2023-11-28T13:07:20.008 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.008+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02b40b3ce0 con 0x7f02b4140950 2023-11-28T13:07:20.009 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.008+0000 7f02b17fa700 1 -- 172.21.15.153:0/3621826202 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f02ac057e50 con 0x7f02b4140950 2023-11-28T13:07:20.009 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:20.009 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:03:11.406430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:20.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.010+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f02980682f0 msgr2=0x7f029806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.011 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f02980682f0 0x7f029806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02a8007fd0 tx=0x7f02a8009040 comp rx=0 tx=0).stop 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4140950 msgr2=0x7f02b415c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4140950 0x7f02b415c160 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f02ac014040 tx=0x7f02ac004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 shutdown_connections 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f02980682f0 0x7f029806a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02b4140950 0x7f02b415c160 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b40b2e70 0x7f02b40cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 --2- 172.21.15.153:0/3621826202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b40cd8c0 0x7f02b415bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 >> 172.21.15.153:0/3621826202 conn(0x7f02b406aed0 msgr2=0x7f02b406bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 shutdown_connections 2023-11-28T13:07:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.011+0000 7f02bb1c1700 1 -- 172.21.15.153:0/3621826202 wait complete. 2023-11-28T13:07:20.013 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2023-11-28T13:07:20.024 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:20.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.143+0000 7f54a071e700 1 Processor -- start 2023-11-28T13:07:20.144 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.143+0000 7f54a071e700 1 -- start start 2023-11-28T13:07:20.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.144+0000 7f54a071e700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5498071040 0x7f5498071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.144+0000 7f54a071e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5498160540 0x7f5498160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.144+0000 7f54a071e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5498071950 con 0x7f549806dee0 2023-11-28T13:07:20.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.144+0000 7f54a071e700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549815d060 con 0x7f5498071040 2023-11-28T13:07:20.147 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.144+0000 7f54a071e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549815d340 con 0x7f5498160540 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.144+0000 7f549e4ba700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f5498071040 0x7f5498071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549dcb9700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f549806dee0 0x7f549806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33372/0 (socket says 172.21.15.153:33372) 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549dcb9700 1 -- 172.21.15.153:0/2685731563 learned_addr learned my addr 172.21.15.153:0/2685731563 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549e4ba700 1 --2- 172.21.15.153:0/2685731563 >> v2:172.21.15.153:3300/0 conn(0x7f5498071040 0x7f5498071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43440/0 (socket says 172.21.15.153:43440) 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549ecbb700 1 --2- 172.21.15.153:0/2685731563 >> v2:172.21.15.177:3300/0 conn(0x7f5498160540 0x7f5498160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549e4ba700 1 -- 172.21.15.153:0/2685731563 >> v2:172.21.15.177:3300/0 conn(0x7f5498160540 msgr2=0x7f5498160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549e4ba700 1 --2- 172.21.15.153:0/2685731563 >> v2:172.21.15.177:3300/0 conn(0x7f5498160540 0x7f5498160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549e4ba700 1 -- 172.21.15.153:0/2685731563 >> v1:172.21.15.153:6789/0 conn(0x7f549806dee0 legacy=0x7f549806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549e4ba700 1 -- 172.21.15.153:0/2685731563 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549815d620 con 0x7f5498071040 2023-11-28T13:07:20.148 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549d4b8700 1 -- 172.21.15.153:0/2685731563 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222028198 0 0) 0x7f5498071950 con 0x7f549806dee0 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549e4ba700 1 --2- 172.21.15.153:0/2685731563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5498071040 0x7f5498071410 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f54980c3a10 tx=0x7f5488009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a347e16ffb262f0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549d4b8700 1 -- 172.21.15.153:0/2685731563 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488011830 con 0x7f5498071040 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549d4b8700 1 -- 172.21.15.153:0/2685731563 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488011990 con 0x7f5498071040 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.145+0000 7f549d4b8700 1 -- 172.21.15.153:0/2685731563 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488011d20 con 0x7f5498071040 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 -- 172.21.15.153:0/2685731563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5498071040 msgr2=0x7f5498071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 --2- 172.21.15.153:0/2685731563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5498071040 0x7f5498071410 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f54980c3a10 tx=0x7f5488009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 -- 172.21.15.153:0/2685731563 shutdown_connections 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 --2- 172.21.15.153:0/2685731563 >> v2:172.21.15.177:3300/0 conn(0x7f5498160540 0x7f5498160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 --2- 172.21.15.153:0/2685731563 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5498071040 0x7f5498071410 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 -- 172.21.15.153:0/2685731563 >> 172.21.15.153:0/2685731563 conn(0x7f54980d0000 msgr2=0x7f54980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 -- 172.21.15.153:0/2685731563 shutdown_connections 2023-11-28T13:07:20.149 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.146+0000 7f54a071e700 1 -- 172.21.15.153:0/2685731563 wait complete. 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.147+0000 7f54a071e700 1 Processor -- start 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.147+0000 7f54a071e700 1 -- start start 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f549806dee0 0x7f54981f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5498071040 0x7f54980b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5498160540 0x7f54980cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980c68b0 con 0x7f549806dee0 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980c3b40 con 0x7f5498160540 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54981f60b0 con 0x7f5498071040 2023-11-28T13:07:20.153 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f549806dee0 0x7f54981f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f549806dee0 0x7f54981f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43446/0 (socket says 172.21.15.153:43446) 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 -- 172.21.15.153:0/3652328367 learned_addr learned my addr 172.21.15.153:0/3652328367 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549dcb9700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5498071040 0x7f54980b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549ecbb700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5498160540 0x7f54980cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 -- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5498071040 msgr2=0x7f54980b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5498071040 0x7f54980b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 -- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5498160540 msgr2=0x7f54980cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5498160540 0x7f54980cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 -- 172.21.15.153:0/3652328367 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880094a0 con 0x7f549806dee0 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549ecbb700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5498160540 0x7f54980cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:20.154 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f549e4ba700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f549806dee0 0x7f54981f5ad0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5488009790 tx=0x7f5488004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f548effd700 1 -- 172.21.15.153:0/3652328367 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880038a0 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f548effd700 1 -- 172.21.15.153:0/3652328367 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548801e070 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549815ccd0 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f548effd700 1 -- 172.21.15.153:0/3652328367 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488003c20 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.148+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549815d900 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.149+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549806a270 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.150+0000 7f548effd700 1 -- 172.21.15.153:0/3652328367 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f548800f050 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.150+0000 7f548effd700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f547c0682f0 0x7f547c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.150+0000 7f548effd700 1 -- 172.21.15.153:0/3652328367 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5488020070 con 0x7f549806dee0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.150+0000 7f549dcb9700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f547c0682f0 0x7f547c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.155 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.150+0000 7f549dcb9700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f547c0682f0 0x7f547c06a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f54980cc320 tx=0x7f5494007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.156 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.154+0000 7f548effd700 1 -- 172.21.15.153:0/3652328367 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5488053910 con 0x7f549806dee0 2023-11-28T13:07:20.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.377+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54980c3d10 con 0x7f549806dee0 2023-11-28T13:07:20.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.378+0000 7f548effd700 1 -- 172.21.15.153:0/3652328367 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4658 (secure 0 0 0) 0x7f5488056f30 con 0x7f549806dee0 2023-11-28T13:07:20.379 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:20.379 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:54.953387+0000","modified":"2023-11-28T13:03:11.406430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":7,"addr":"172.21.15.153:6837/3815439165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":3815439165},{"type":"v1","addr":"172.21.15.153:6837","nonce":3815439165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:20.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f547c0682f0 msgr2=0x7f547c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f547c0682f0 0x7f547c06a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f54980cc320 tx=0x7f5494007c90 comp rx=0 tx=0).stop 2023-11-28T13:07:20.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f549806dee0 msgr2=0x7f54981f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f549806dee0 0x7f54981f5ad0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5488009790 tx=0x7f5488004750 comp rx=0 tx=0).stop 2023-11-28T13:07:20.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 shutdown_connections 2023-11-28T13:07:20.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f547c0682f0 0x7f547c06a7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f549806dee0 0x7f54981f5ad0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5498160540 0x7f54980cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 --2- 172.21.15.153:0/3652328367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5498071040 0x7f54980b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 >> 172.21.15.153:0/3652328367 conn(0x7f54980d0000 msgr2=0x7f549815e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:20.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 shutdown_connections 2023-11-28T13:07:20.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.380+0000 7f54a071e700 1 -- 172.21.15.153:0/3652328367 wait complete. 2023-11-28T13:07:20.384 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2023-11-28T13:07:20.393 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:20.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.513+0000 7ff7b5b2f700 1 Processor -- start 2023-11-28T13:07:20.514 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.513+0000 7ff7b5b2f700 1 -- start start 2023-11-28T13:07:20.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7b5b2f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7b5b2f700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7b0160540 0x7ff7b01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7b5b2f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b015d060 con 0x7ff7b0071040 2023-11-28T13:07:20.516 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7b5b2f700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b015d340 con 0x7ff7b0160540 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7b5b2f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b015d620 con 0x7ff7b006dee0 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7affff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7b0160540 0x7ff7b01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7affff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ff7b0160540 0x7ff7b01609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43458/0 (socket says 172.21.15.153:43458) 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7affff700 1 -- 172.21.15.153:0/524784783 learned_addr learned my addr 172.21.15.153:0/524784783 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7aeffd700 1 --2- 172.21.15.153:0/524784783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7affff700 1 -- 172.21.15.153:0/524784783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 msgr2=0x7ff7b006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7affff700 1 --2- 172.21.15.153:0/524784783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7affff700 1 -- 172.21.15.153:0/524784783 >> v1:172.21.15.177:6789/0 conn(0x7ff7b0071040 legacy=0x7ff7b0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.514+0000 7ff7affff700 1 -- 172.21.15.153:0/524784783 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b015d900 con 0x7ff7b0160540 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7affff700 1 --2- 172.21.15.153:0/524784783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0160540 0x7ff7b01609c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009e50 tx=0x7ff7a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d7daa0fb46f32a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7ae7fc700 1 -- 172.21.15.153:0/524784783 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a800e070 con 0x7ff7b0160540 2023-11-28T13:07:20.517 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7ae7fc700 1 -- 172.21.15.153:0/524784783 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a800b9c0 con 0x7ff7b0160540 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7ae7fc700 1 -- 172.21.15.153:0/524784783 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a800bd70 con 0x7ff7b0160540 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/524784783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0160540 msgr2=0x7ff7b01609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/524784783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0160540 0x7ff7b01609c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009e50 tx=0x7ff7a800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/524784783 shutdown_connections 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/524784783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0160540 0x7ff7b01609c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/524784783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/524784783 >> 172.21.15.153:0/524784783 conn(0x7ff7b00d0000 msgr2=0x7ff7b00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/524784783 shutdown_connections 2023-11-28T13:07:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.515+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/524784783 wait complete. 2023-11-28T13:07:20.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.516+0000 7ff7b5b2f700 1 Processor -- start 2023-11-28T13:07:20.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.517+0000 7ff7b5b2f700 1 -- start start 2023-11-28T13:07:20.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.517+0000 7ff7b5b2f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b01f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.517+0000 7ff7b5b2f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0071040 0x7ff7b00c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.517+0000 7ff7b5b2f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b0160540 0x7ff7b00b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.522 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.517+0000 7ff7b5b2f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b01f3510 con 0x7ff7b0071040 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.517+0000 7ff7b5b2f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b00c6320 con 0x7ff7b0160540 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.517+0000 7ff7b5b2f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b00b3260 con 0x7ff7b006dee0 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0071040 0x7ff7b00c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0071040 0x7ff7b00c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43462/0 (socket says 172.21.15.153:43462) 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 -- 172.21.15.153:0/601187125 learned_addr learned my addr 172.21.15.153:0/601187125 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7affff700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b0160540 0x7ff7b00b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7af7fe700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b01f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 -- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 msgr2=0x7ff7b01f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b01f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 -- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b0160540 msgr2=0x7ff7b00b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b0160540 0x7ff7b00b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 -- 172.21.15.153:0/601187125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a8009b30 con 0x7ff7b0071040 2023-11-28T13:07:20.523 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7aeffd700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0071040 0x7ff7b00c5100 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000d800 tx=0x7ff7a000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff797fff700 1 -- 172.21.15.153:0/601187125 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00125f0 con 0x7ff7b0071040 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff797fff700 1 -- 172.21.15.153:0/601187125 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a0012750 con 0x7ff7b0071040 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b015ccd0 con 0x7ff7b0071040 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff797fff700 1 -- 172.21.15.153:0/601187125 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0012b00 con 0x7ff7b0071040 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.518+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b006e870 con 0x7ff7b0071040 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.519+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b006a270 con 0x7ff7b0071040 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.520+0000 7ff797fff700 1 -- 172.21.15.153:0/601187125 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff7a0012c60 con 0x7ff7b0071040 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.520+0000 7ff797fff700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff79c0682f0 0x7ff79c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.524 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.520+0000 7ff797fff700 1 -- 172.21.15.153:0/601187125 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff7a0085e70 con 0x7ff7b0071040 2023-11-28T13:07:20.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.520+0000 7ff7af7fe700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff79c0682f0 0x7ff79c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.520+0000 7ff7af7fe700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff79c0682f0 0x7ff79c06a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4007fd0 tx=0x7ff7a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.525 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.524+0000 7ff797fff700 1 -- 172.21.15.153:0/601187125 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7a0052640 con 0x7ff7b0071040 2023-11-28T13:07:20.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.714+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7b00720a0 con 0x7ff7b0071040 2023-11-28T13:07:20.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.715+0000 7ff797fff700 1 -- 172.21.15.153:0/601187125 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18209 (secure 0 0 0) 0x7ff7a0055c60 con 0x7ff7b0071040 2023-11-28T13:07:20.716 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:20.717 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":65,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:04:00.505453+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:52.747638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:53.909181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.4","mappings":[{"from":4,"to":3}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.3a","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:20.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff79c0682f0 msgr2=0x7ff79c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff79c0682f0 0x7ff79c06a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4007fd0 tx=0x7ff7a4009040 comp rx=0 tx=0).stop 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0071040 msgr2=0x7ff7b00c5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0071040 0x7ff7b00c5100 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000d800 tx=0x7ff7a000db10 comp rx=0 tx=0).stop 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 shutdown_connections 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b0160540 0x7ff7b00b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b006dee0 0x7ff7b01f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ff79c0682f0 0x7ff79c06a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 --2- 172.21.15.153:0/601187125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff7b0071040 0x7ff7b00c5100 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.717+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 >> 172.21.15.153:0/601187125 conn(0x7ff7b00d0000 msgr2=0x7ff7b015e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.718+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 shutdown_connections 2023-11-28T13:07:20.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.718+0000 7ff7b5b2f700 1 -- 172.21.15.153:0/601187125 wait complete. 2023-11-28T13:07:20.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:07:20.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.850+0000 7fc6d74ec700 1 Processor -- start 2023-11-28T13:07:20.851 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.850+0000 7fc6d74ec700 1 -- start start 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6d74ec700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 0x7fc6d0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6d74ec700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d0071950 con 0x7fc6d0160540 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6d74ec700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d015d060 con 0x7fc6d006dee0 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6d74ec700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d015d340 con 0x7fc6d0071040 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6d5a89700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fc6d0160540 0x7fc6d01609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33404/0 (socket says 172.21.15.153:33404) 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6d5a89700 1 -- 172.21.15.153:0/1581027106 learned_addr learned my addr 172.21.15.153:0/1581027106 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6d5288700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 0x7fc6d0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541433942 0 0) 0x7fc6d0071950 con 0x7fc6d0160540 2023-11-28T13:07:20.862 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.851+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6bc0035b0 con 0x7fc6d0160540 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812815701 0 0) 0x7fc6d015d060 con 0x7fc6d006dee0 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d0071950 con 0x7fc6d006dee0 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2135598630 0 0) 0x7fc6bc0035b0 con 0x7fc6d0160540 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 msgr2=0x7fc6d0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 0x7fc6d0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 >> v1:172.21.15.177:6789/0 conn(0x7fc6d006dee0 legacy=0x7fc6d006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d015d620 con 0x7fc6d0160540 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fc6c80019f0 con 0x7fc6d0160540 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 >> v1:172.21.15.153:6789/0 conn(0x7fc6d0160540 legacy=0x7fc6d01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 0x7fc6bc003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6bc004120 0x7fc6bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc004d90 0x7fc6bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d015d340 con 0x7fc6bc004d90 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc00bc00 con 0x7fc6bc004120 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc00bdc0 con 0x7fc6d0071040 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5a89700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc004d90 0x7fc6bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5288700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 0x7fc6bc003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d4a87700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6bc004120 0x7fc6bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5a89700 1 -- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 msgr2=0x7fc6bc003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5a89700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d0071040 0x7fc6bc003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5a89700 1 -- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6bc004120 msgr2=0x7fc6bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5a89700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6bc004120 0x7fc6bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5a89700 1 -- 172.21.15.153:0/1581027106 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0035b0 con 0x7fc6bc004d90 2023-11-28T13:07:20.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6d5a89700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc004d90 0x7fc6bc005200 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8002230 tx=0x7fc6bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92ba78bdd5ddd99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8012600 con 0x7fc6bc004d90 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.852+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c8012780 con 0x7fc6bc004d90 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6c7fff700 1 -- 172.21.15.153:0/1581027106 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8012b30 con 0x7fc6bc004d90 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1581027106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc004d90 msgr2=0x7fc6bc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6d74ec700 1 --2- 172.21.15.153:0/1581027106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc004d90 0x7fc6bc005200 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8002230 tx=0x7fc6bc005880 comp rx=0 tx=0).stop 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6d5a89700 1 -- 172.21.15.153:0/1581027106 reap_dead start 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1581027106 shutdown_connections 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1581027106 >> 172.21.15.153:0/1581027106 conn(0x7fc6d00d0000 msgr2=0x7fc6d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1581027106 shutdown_connections 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.853+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1581027106 wait complete. 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.854+0000 7fc6d74ec700 1 Processor -- start 2023-11-28T13:07:20.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.854+0000 7fc6d74ec700 1 -- start start 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d74ec700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d014b430 0x7fc6d00ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d74ec700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6d00ceee0 0x7fc6d00c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d74ec700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6d00c3fa0 0x7fc6d00b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d74ec700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d00cf350 con 0x7fc6d00ceee0 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d74ec700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d00c5830 con 0x7fc6d00c3fa0 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d74ec700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d00c4790 con 0x7fc6d014b430 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6d00ceee0 0x7fc6d00c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6d00ceee0 0x7fc6d00c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43482/0 (socket says 172.21.15.153:43482) 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 -- 172.21.15.153:0/1735180043 learned_addr learned my addr 172.21.15.153:0/1735180043 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d5288700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d014b430 0x7fc6d00ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 -- 172.21.15.153:0/1735180043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d014b430 msgr2=0x7fc6d00ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d014b430 0x7fc6d00ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 -- 172.21.15.153:0/1735180043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6d00c3fa0 msgr2=0x7fc6d00b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6d00c3fa0 0x7fc6d00b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.855+0000 7fc6d4a87700 1 -- 172.21.15.153:0/1735180043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d015ccd0 con 0x7fc6d00ceee0 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.856+0000 7fc6d4a87700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6d00ceee0 0x7fc6d00c3a60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000f770 tx=0x7fc6c000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.856+0000 7fc6c5ffb700 1 -- 172.21.15.153:0/1735180043 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00052e0 con 0x7fc6d00ceee0 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.856+0000 7fc6c5ffb700 1 -- 172.21.15.153:0/1735180043 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c0005460 con 0x7fc6d00ceee0 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.856+0000 7fc6c5ffb700 1 -- 172.21.15.153:0/1735180043 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00038c0 con 0x7fc6d00ceee0 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.856+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d015d900 con 0x7fc6d00ceee0 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.856+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d006a270 con 0x7fc6d00ceee0 2023-11-28T13:07:20.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.857+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d00720a0 con 0x7fc6d00ceee0 2023-11-28T13:07:20.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.857+0000 7fc6c5ffb700 1 -- 172.21.15.153:0/1735180043 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6c0003a20 con 0x7fc6d00ceee0 2023-11-28T13:07:20.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.858+0000 7fc6c5ffb700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6b40682f0 0x7fc6b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:20.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.858+0000 7fc6c5ffb700 1 -- 172.21.15.153:0/1735180043 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc6c0085db0 con 0x7fc6d00ceee0 2023-11-28T13:07:20.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.858+0000 7fc6d5288700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6b40682f0 0x7fc6b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:20.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.858+0000 7fc6d5288700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6b40682f0 0x7fc6b406a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009aa0 tx=0x7fc6cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:20.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:20.862+0000 7fc6c5ffb700 1 -- 172.21.15.153:0/1735180043 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6c008a070 con 0x7fc6d00ceee0 2023-11-28T13:07:21.088 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.088+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc6d009de50 con 0x7fc6d00ceee0 2023-11-28T13:07:21.641 INFO:tasks.ceph.mon.a.smithi153.stderr:2023-11-28T13:07:21.640+0000 7feb34d4f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:07:21.647 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.647+0000 7fc6c5ffb700 1 -- 172.21.15.153:0/1735180043 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc6c0052490 con 0x7fc6d00ceee0 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6b40682f0 msgr2=0x7fc6b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6b40682f0 0x7fc6b406a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009aa0 tx=0x7fc6cc008040 comp rx=0 tx=0).stop 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6d00ceee0 msgr2=0x7fc6d00c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6d00ceee0 0x7fc6d00c3a60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000f770 tx=0x7fc6c000fb30 comp rx=0 tx=0).stop 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 shutdown_connections 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6d00c3fa0 0x7fc6d00b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6d014b430 0x7fc6d00ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc6b40682f0 0x7fc6b406a7a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 --2- 172.21.15.153:0/1735180043 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6d00ceee0 0x7fc6d00c3a60 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:21.651 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 >> 172.21.15.153:0/1735180043 conn(0x7fc6d00d0000 msgr2=0x7fc6d015f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:21.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.649+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 shutdown_connections 2023-11-28T13:07:21.652 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.650+0000 7fc6d74ec700 1 -- 172.21.15.153:0/1735180043 wait complete. 2023-11-28T13:07:21.652 INFO:teuthology.orchestra.run.smithi153.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:07:21.664 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:07:21.686 INFO:tasks.ceph.mds.c.smithi153.stdout:starting mds.c at 2023-11-28T13:07:21.790 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.789+0000 7fd3193ad700 1 Processor -- start 2023-11-28T13:07:21.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.790+0000 7fd3193ad700 1 -- start start 2023-11-28T13:07:21.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3193ad700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd31415b9d0 0x7fd31406acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:21.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3193ad700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3140aee40 con 0x7fd3140ae9d0 2023-11-28T13:07:21.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3193ad700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31406c930 con 0x7fd31406e500 2023-11-28T13:07:21.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3193ad700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd31406cc10 con 0x7fd31415b9d0 2023-11-28T13:07:21.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3137fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd31415b9d0 0x7fd31406acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:21.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3127fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fd3140ae9d0 0x7fd31406a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33434/0 (socket says 172.21.15.153:33434) 2023-11-28T13:07:21.798 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3127fc700 1 -- 172.21.15.153:0/1160877720 learned_addr learned my addr 172.21.15.153:0/1160877720 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3137fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd31415b9d0 0x7fd31406acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43504/0 (socket says 172.21.15.153:43504) 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3137fe700 1 -- 172.21.15.153:0/1160877720 >> v1:172.21.15.177:6789/0 conn(0x7fd31406e500 legacy=0x7fd31406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3137fe700 1 -- 172.21.15.153:0/1160877720 >> v1:172.21.15.153:6789/0 conn(0x7fd3140ae9d0 legacy=0x7fd31406a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3137fe700 1 -- 172.21.15.153:0/1160877720 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3140664c0 con 0x7fd31415b9d0 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd3137fe700 1 --2- 172.21.15.153:0/1160877720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406acf0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009d90 tx=0x7fd30c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5728dd6b1b1f21ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd311ffb700 1 -- 172.21.15.153:0/1160877720 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c00e070 con 0x7fd31415b9d0 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd311ffb700 1 -- 172.21.15.153:0/1160877720 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c00bb90 con 0x7fd31415b9d0 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.791+0000 7fd311ffb700 1 -- 172.21.15.153:0/1160877720 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c003a20 con 0x7fd31415b9d0 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.792+0000 7fd3193ad700 1 -- 172.21.15.153:0/1160877720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 msgr2=0x7fd31406acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.792+0000 7fd3193ad700 1 --2- 172.21.15.153:0/1160877720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406acf0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009d90 tx=0x7fd30c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.792+0000 7fd3193ad700 1 -- 172.21.15.153:0/1160877720 shutdown_connections 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.792+0000 7fd3193ad700 1 --2- 172.21.15.153:0/1160877720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406acf0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.792+0000 7fd3193ad700 1 -- 172.21.15.153:0/1160877720 >> 172.21.15.153:0/1160877720 conn(0x7fd3140b12b0 msgr2=0x7fd3140b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.792+0000 7fd3193ad700 1 -- 172.21.15.153:0/1160877720 shutdown_connections 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.792+0000 7fd3193ad700 1 -- 172.21.15.153:0/1160877720 wait complete. 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.793+0000 7fd3193ad700 1 Processor -- start 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.793+0000 7fd3193ad700 1 -- start start 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3193ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd31406e500 0x7fd31414da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3193ad700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3140ae9d0 0x7fd31414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3193ad700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3193ad700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31415d660 con 0x7fd31415b9d0 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3193ad700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31414fdf0 con 0x7fd3140ae9d0 2023-11-28T13:07:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3193ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31415ca20 con 0x7fd31406e500 2023-11-28T13:07:21.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:21.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43510/0 (socket says 172.21.15.153:43510) 2023-11-28T13:07:21.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 -- 172.21.15.153:0/3567258707 learned_addr learned my addr 172.21.15.153:0/3567258707 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:21.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3127fc700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3140ae9d0 0x7fd31414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:21.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd312ffd700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd31406e500 0x7fd31414da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:21.801 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 -- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd31406e500 msgr2=0x7fd31414da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd31406e500 0x7fd31414da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 -- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3140ae9d0 msgr2=0x7fd31414f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3140ae9d0 0x7fd31414f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd3137fe700 1 -- 172.21.15.153:0/3567258707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c009a70 con 0x7fd31415b9d0 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.794+0000 7fd312ffd700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd31406e500 0x7fd31414da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.795+0000 7fd3137fe700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406b090 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00b730 tx=0x7fd30c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.795+0000 7fd2fb7fe700 1 -- 172.21.15.153:0/3567258707 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c012630 con 0x7fd31415b9d0 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.795+0000 7fd2fb7fe700 1 -- 172.21.15.153:0/3567258707 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c0127b0 con 0x7fd31415b9d0 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.795+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31415bdd0 con 0x7fd31415b9d0 2023-11-28T13:07:21.802 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.795+0000 7fd2fb7fe700 1 -- 172.21.15.153:0/3567258707 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c012b60 con 0x7fd31415b9d0 2023-11-28T13:07:21.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.795+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31406cef0 con 0x7fd31415b9d0 2023-11-28T13:07:21.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.796+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31415ac80 con 0x7fd31415b9d0 2023-11-28T13:07:21.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.796+0000 7fd2fb7fe700 1 -- 172.21.15.153:0/3567258707 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd30c012cc0 con 0x7fd31415b9d0 2023-11-28T13:07:21.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.797+0000 7fd2fb7fe700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3000682f0 0x7fd30006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:21.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.797+0000 7fd312ffd700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3000682f0 0x7fd30006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:21.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.797+0000 7fd2fb7fe700 1 -- 172.21.15.153:0/3567258707 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd30c088700 con 0x7fd31415b9d0 2023-11-28T13:07:21.803 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.797+0000 7fd312ffd700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3000682f0 0x7fd30006a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd304009910 tx=0x7fd304008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:21.804 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:21.801+0000 7fd2fb7fe700 1 -- 172.21.15.153:0/3567258707 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd30c054f30 con 0x7fd31415b9d0 2023-11-28T13:07:22.027 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.026+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd31415d7a0 con 0x7fd31415b9d0 2023-11-28T13:07:22.654 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.653+0000 7fd2fb7fe700 1 -- 172.21.15.153:0/3567258707 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd31415d7a0 con 0x7fd31415b9d0 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3000682f0 msgr2=0x7fd30006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3000682f0 0x7fd30006a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd304009910 tx=0x7fd304008040 comp rx=0 tx=0).stop 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 msgr2=0x7fd31406b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406b090 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00b730 tx=0x7fd30c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 shutdown_connections 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd3000682f0 0x7fd30006a7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd31415b9d0 0x7fd31406b090 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.657 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd31406e500 0x7fd31414da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 --2- 172.21.15.153:0/3567258707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3140ae9d0 0x7fd31414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 >> 172.21.15.153:0/3567258707 conn(0x7fd3140b12b0 msgr2=0x7fd3140af060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:22.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 shutdown_connections 2023-11-28T13:07:22.658 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.656+0000 7fd3193ad700 1 -- 172.21.15.153:0/3567258707 wait complete. 2023-11-28T13:07:22.671 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:07:22.837 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.836+0000 7f069bead700 1 Processor -- start 2023-11-28T13:07:22.838 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.837+0000 7f069bead700 1 -- start start 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.837+0000 7f069bead700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f06940b4b80 0x7f069406c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.837+0000 7f069bead700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f069415ac50 0x7f069406cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.837+0000 7f069bead700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069406a270 con 0x7f069415a880 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.837+0000 7f069bead700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06940b5040 con 0x7f06940b4b80 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.837+0000 7f069bead700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06940b5320 con 0x7f069415ac50 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699448700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f06940b4b80 0x7f069406c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699c49700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f069415a880 0x7f06940b4470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33448/0 (socket says 172.21.15.153:33448) 2023-11-28T13:07:22.844 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699c49700 1 -- 172.21.15.153:0/1997068643 learned_addr learned my addr 172.21.15.153:0/1997068643 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699448700 1 -- 172.21.15.153:0/1997068643 >> v2:172.21.15.177:3300/0 conn(0x7f069415ac50 msgr2=0x7f069406cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699448700 1 --2- 172.21.15.153:0/1997068643 >> v2:172.21.15.177:3300/0 conn(0x7f069415ac50 0x7f069406cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699448700 1 -- 172.21.15.153:0/1997068643 >> v1:172.21.15.153:6789/0 conn(0x7f069415a880 legacy=0x7f06940b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0698c47700 1 -- 172.21.15.153:0/1997068643 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322369787 0 0) 0x7f069406a270 con 0x7f069415a880 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699448700 1 -- 172.21.15.153:0/1997068643 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06940d0020 con 0x7f06940b4b80 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0699448700 1 --2- 172.21.15.153:0/1997068643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06940b4b80 0x7f069406c890 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f06940ae510 tx=0x7f0690009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5f28da596bfc3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0698c47700 1 -- 172.21.15.153:0/1997068643 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690011830 con 0x7f06940b4b80 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0698c47700 1 -- 172.21.15.153:0/1997068643 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690011990 con 0x7f06940b4b80 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.838+0000 7f0698c47700 1 -- 172.21.15.153:0/1997068643 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690011d20 con 0x7f06940b4b80 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 -- 172.21.15.153:0/1997068643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06940b4b80 msgr2=0x7f069406c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 --2- 172.21.15.153:0/1997068643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06940b4b80 0x7f069406c890 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f06940ae510 tx=0x7f0690009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 -- 172.21.15.153:0/1997068643 shutdown_connections 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 --2- 172.21.15.153:0/1997068643 >> v2:172.21.15.177:3300/0 conn(0x7f069415ac50 0x7f069406cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 --2- 172.21.15.153:0/1997068643 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06940b4b80 0x7f069406c890 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 -- 172.21.15.153:0/1997068643 >> 172.21.15.153:0/1997068643 conn(0x7f06940b3080 msgr2=0x7f06940cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 -- 172.21.15.153:0/1997068643 shutdown_connections 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.839+0000 7f069bead700 1 -- 172.21.15.153:0/1997068643 wait complete. 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.840+0000 7f069bead700 1 Processor -- start 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.840+0000 7f069bead700 1 -- start start 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069bead700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06940b4b80 0x7f0694098a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069bead700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069415a880 0x7f06941530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:22.846 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069bead700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069415ac50 0x7f06940c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069bead700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06941579c0 con 0x7f069415a880 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069bead700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0694098f40 con 0x7f069415ac50 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069bead700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06941535f0 con 0x7f06940b4b80 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069415a880 0x7f06941530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069415a880 0x7f06941530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43524/0 (socket says 172.21.15.153:43524) 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 -- 172.21.15.153:0/2422263305 learned_addr learned my addr 172.21.15.153:0/2422263305 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699c49700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06940b4b80 0x7f0694098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069a44a700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069415ac50 0x7f06940c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 -- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06940b4b80 msgr2=0x7f0694098a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06940b4b80 0x7f0694098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 -- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069415ac50 msgr2=0x7f06940c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069415ac50 0x7f06940c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:22.847 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 -- 172.21.15.153:0/2422263305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900094a0 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f0699448700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069415a880 0x7f06941530b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0690013000 tx=0x7f0690004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f068a7fc700 1 -- 172.21.15.153:0/2422263305 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900038a0 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f068a7fc700 1 -- 172.21.15.153:0/2422263305 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069001f070 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06940aec90 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.841+0000 7f068a7fc700 1 -- 172.21.15.153:0/2422263305 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690003c40 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.842+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06940d0300 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.842+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0694065ef0 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.843+0000 7f068a7fc700 1 -- 172.21.15.153:0/2422263305 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f069000f050 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.843+0000 7f068a7fc700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f06780682f0 0x7f067806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.843+0000 7f068a7fc700 1 -- 172.21.15.153:0/2422263305 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f0690021070 con 0x7f069415a880 2023-11-28T13:07:22.848 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.843+0000 7f0699c49700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f06780682f0 0x7f067806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:22.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.844+0000 7f0699c49700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f06780682f0 0x7f067806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0684000f60 tx=0x7f0684009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:22.849 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:22.848+0000 7f068a7fc700 1 -- 172.21.15.153:0/2422263305 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0690054000 con 0x7f069415a880 2023-11-28T13:07:23.032 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.032+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06940ccc10 con 0x7f069415a880 2023-11-28T13:07:23.710 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.710+0000 7f068a7fc700 1 -- 172.21.15.153:0/2422263305 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0690057620 con 0x7f069415a880 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.713+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f06780682f0 msgr2=0x7f067806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.713+0000 7f069bead700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f06780682f0 0x7f067806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0684000f60 tx=0x7f0684009040 comp rx=0 tx=0).stop 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.713+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069415a880 msgr2=0x7f06941530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.713+0000 7f069bead700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069415a880 0x7f06941530b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0690013000 tx=0x7f0690004750 comp rx=0 tx=0).stop 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 shutdown_connections 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069415ac50 0x7f06940c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06940b4b80 0x7f0694098a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f06780682f0 0x7f067806a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 --2- 172.21.15.153:0/2422263305 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069415a880 0x7f06941530b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 >> 172.21.15.153:0/2422263305 conn(0x7f06940b3080 msgr2=0x7f06940ac0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 shutdown_connections 2023-11-28T13:07:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.714+0000 7f069bead700 1 -- 172.21.15.153:0/2422263305 wait complete. 2023-11-28T13:07:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:07:23.730 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:07:23.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.859+0000 7f121881b700 1 Processor -- start 2023-11-28T13:07:23.860 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.859+0000 7f121881b700 1 -- start start 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.860+0000 7f121881b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f12140af060 0x7f1214070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.860+0000 7f121881b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f121406a980 0x7f121406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.860+0000 7f121881b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12140713c0 con 0x7f12140af430 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.860+0000 7f121881b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121406dee0 con 0x7f121406a980 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.860+0000 7f121881b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12140716a0 con 0x7f12140af060 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f121406a980 0x7f121406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f121406a980 0x7f121406d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43534/0 (socket says 172.21.15.153:43534) 2023-11-28T13:07:23.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 -- 172.21.15.153:0/3445046825 learned_addr learned my addr 172.21.15.153:0/3445046825 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f121259c700 1 --2- 172.21.15.153:0/3445046825 >> v2:172.21.15.177:3300/0 conn(0x7f12140af060 0x7f1214070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 -- 172.21.15.153:0/3445046825 >> v2:172.21.15.177:3300/0 conn(0x7f12140af060 msgr2=0x7f1214070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 --2- 172.21.15.153:0/3445046825 >> v2:172.21.15.177:3300/0 conn(0x7f12140af060 0x7f1214070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 -- 172.21.15.153:0/3445046825 >> v1:172.21.15.177:6789/0 conn(0x7f12140af430 legacy=0x7f121406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 -- 172.21.15.153:0/3445046825 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214071980 con 0x7f121406a980 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f1212d9d700 1 --2- 172.21.15.153:0/3445046825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121406a980 0x7f121406d9a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f12140725d0 tx=0x7f120c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7740daa628a42a33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f121159a700 1 -- 172.21.15.153:0/3445046825 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00e070 con 0x7f121406a980 2023-11-28T13:07:23.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f121159a700 1 -- 172.21.15.153:0/3445046825 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c00b910 con 0x7f121406a980 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.861+0000 7f121159a700 1 -- 172.21.15.153:0/3445046825 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00bcc0 con 0x7f121406a980 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 -- 172.21.15.153:0/3445046825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121406a980 msgr2=0x7f121406d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 --2- 172.21.15.153:0/3445046825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121406a980 0x7f121406d9a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f12140725d0 tx=0x7f120c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 -- 172.21.15.153:0/3445046825 shutdown_connections 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 --2- 172.21.15.153:0/3445046825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f121406a980 0x7f121406d9a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 --2- 172.21.15.153:0/3445046825 >> v2:172.21.15.177:3300/0 conn(0x7f12140af060 0x7f1214070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 -- 172.21.15.153:0/3445046825 >> 172.21.15.153:0/3445046825 conn(0x7f121406b380 msgr2=0x7f121406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 -- 172.21.15.153:0/3445046825 shutdown_connections 2023-11-28T13:07:23.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.862+0000 7f121881b700 1 -- 172.21.15.153:0/3445046825 wait complete. 2023-11-28T13:07:23.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.863+0000 7f121881b700 1 Processor -- start 2023-11-28T13:07:23.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.863+0000 7f121881b700 1 -- start start 2023-11-28T13:07:23.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f121881b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f121406a980 0x7f12140b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:23.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f121881b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12140af060 0x7f12140ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:23.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f121881b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12140af430 0x7f121415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:23.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f121881b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12140b1880 con 0x7f12140af060 2023-11-28T13:07:23.868 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f121881b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121415c520 con 0x7f121406a980 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f121881b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140b0880 con 0x7f12140af430 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12140af060 0x7f12140ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12140af060 0x7f12140ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43538/0 (socket says 172.21.15.153:43538) 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 -- 172.21.15.153:0/340762631 learned_addr learned my addr 172.21.15.153:0/340762631 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 -- 172.21.15.153:0/340762631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12140af430 msgr2=0x7f121415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12140af430 0x7f121415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 -- 172.21.15.153:0/340762631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f121406a980 msgr2=0x7f12140b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f121406a980 0x7f12140b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 -- 172.21.15.153:0/340762631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c009b30 con 0x7f12140af060 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.864+0000 7f1211d9b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12140af060 0x7f12140ad990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f120400c4b0 tx=0x7f120400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.865+0000 7f1202ffd700 1 -- 172.21.15.153:0/340762631 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040052d0 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.865+0000 7f1202ffd700 1 -- 172.21.15.153:0/340762631 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1204005430 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.865+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1214071f40 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.865+0000 7f1202ffd700 1 -- 172.21.15.153:0/340762631 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040038c0 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.865+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1214071c60 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.866+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12140ca660 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.866+0000 7f1202ffd700 1 -- 172.21.15.153:0/340762631 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1204019030 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.866+0000 7f1202ffd700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f11f40682f0 0x7f11f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.867+0000 7f1202ffd700 1 -- 172.21.15.153:0/340762631 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5826+0+0 (secure 0 0 0) 0x7f1204084f50 con 0x7f12140af060 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.867+0000 7f121259c700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f11f40682f0 0x7f11f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:23.870 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.867+0000 7f121259c700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f11f40682f0 0x7f11f406a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1208006b20 tx=0x7f1208006ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:23.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:23.871+0000 7f1202ffd700 1 -- 172.21.15.153:0/340762631 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1204051890 con 0x7f12140af060 2023-11-28T13:07:24.054 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.052+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1214068690 con 0x7f12140af060 2023-11-28T13:07:24.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.716+0000 7f1202ffd700 1 -- 172.21.15.153:0/340762631 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1204054eb0 con 0x7f12140af060 2023-11-28T13:07:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f11f40682f0 msgr2=0x7f11f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f11f40682f0 0x7f11f406a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1208006b20 tx=0x7f1208006ab0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12140af060 msgr2=0x7f12140ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12140af060 0x7f12140ad990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f120400c4b0 tx=0x7f120400c870 comp rx=0 tx=0).stop 2023-11-28T13:07:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 shutdown_connections 2023-11-28T13:07:24.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12140af430 0x7f121415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f121406a980 0x7f12140b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f11f40682f0 0x7f11f406a7a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 --2- 172.21.15.153:0/340762631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12140af060 0x7f12140ad990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 >> 172.21.15.153:0/340762631 conn(0x7f121406b380 msgr2=0x7f12141e6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:24.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 shutdown_connections 2023-11-28T13:07:24.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.722+0000 7f121881b700 1 -- 172.21.15.153:0/340762631 wait complete. 2023-11-28T13:07:24.725 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'cephfs_data' removed 2023-11-28T13:07:24.736 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:07:24.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.860+0000 7f69f8209700 1 Processor -- start 2023-11-28T13:07:24.861 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.861+0000 7f69f8209700 1 -- start start 2023-11-28T13:07:24.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.861+0000 7f69f8209700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f69f0071040 0x7f69f0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:24.863 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.861+0000 7f69f8209700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.861+0000 7f69f8209700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f0071950 con 0x7f69f006dee0 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.861+0000 7f69f8209700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f015d060 con 0x7f69f0071040 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.861+0000 7f69f8209700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015d340 con 0x7f69f0160540 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f67a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f5fa5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f69f0071040 0x7f69f0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f57a4700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f69f006dee0 0x7f69f006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:38322/0 (socket says 172.21.15.153:38322) 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f57a4700 1 -- 172.21.15.153:0/586108804 learned_addr learned my addr 172.21.15.153:0/586108804 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f4fa3700 1 -- 172.21.15.153:0/586108804 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761289527 0 0) 0x7f69f0071950 con 0x7f69f006dee0 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f4fa3700 1 -- 172.21.15.153:0/586108804 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69dc0035b0 con 0x7f69f006dee0 2023-11-28T13:07:24.864 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f67a6700 1 -- 172.21.15.153:0/586108804 >> v2:172.21.15.177:3300/0 conn(0x7f69f0071040 msgr2=0x7f69f0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f67a6700 1 --2- 172.21.15.153:0/586108804 >> v2:172.21.15.177:3300/0 conn(0x7f69f0071040 0x7f69f0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f67a6700 1 -- 172.21.15.153:0/586108804 >> v1:172.21.15.177:6789/0 conn(0x7f69f006dee0 legacy=0x7f69f006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f67a6700 1 -- 172.21.15.153:0/586108804 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f015d620 con 0x7f69f0160540 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f5fa5700 1 --2- 172.21.15.153:0/586108804 >> v2:172.21.15.177:3300/0 conn(0x7f69f0071040 0x7f69f0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.862+0000 7f69f67a6700 1 --2- 172.21.15.153:0/586108804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f01609c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009d90 tx=0x7f69e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de06b8279e4421d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f4fa3700 1 -- 172.21.15.153:0/586108804 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800e070 con 0x7f69f0160540 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f4fa3700 1 -- 172.21.15.153:0/586108804 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e800bb20 con 0x7f69f0160540 2023-11-28T13:07:24.865 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f4fa3700 1 -- 172.21.15.153:0/586108804 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800be90 con 0x7f69f0160540 2023-11-28T13:07:24.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 -- 172.21.15.153:0/586108804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 msgr2=0x7f69f01609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:24.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 --2- 172.21.15.153:0/586108804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f01609c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009d90 tx=0x7f69e800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:24.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 -- 172.21.15.153:0/586108804 shutdown_connections 2023-11-28T13:07:24.866 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 --2- 172.21.15.153:0/586108804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f01609c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 --2- 172.21.15.153:0/586108804 >> v2:172.21.15.177:3300/0 conn(0x7f69f0071040 0x7f69f0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 -- 172.21.15.153:0/586108804 >> 172.21.15.153:0/586108804 conn(0x7f69f00d0000 msgr2=0x7f69f00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:24.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 -- 172.21.15.153:0/586108804 shutdown_connections 2023-11-28T13:07:24.867 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.863+0000 7f69f8209700 1 -- 172.21.15.153:0/586108804 wait complete. 2023-11-28T13:07:24.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.865+0000 7f69f8209700 1 Processor -- start 2023-11-28T13:07:24.871 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.866+0000 7f69f8209700 1 -- start start 2023-11-28T13:07:24.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.868+0000 7f69f8209700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f006dee0 0x7f69f00ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:24.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.868+0000 7f69f8209700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f0071040 0x7f69f0157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:24.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.868+0000 7f69f57a4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f0071040 0x7f69f0157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:24.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f8209700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f0157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:24.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f8209700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00c3cc0 con 0x7f69f0071040 2023-11-28T13:07:24.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f8209700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00ade90 con 0x7f69f006dee0 2023-11-28T13:07:24.872 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f8209700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00ae170 con 0x7f69f0160540 2023-11-28T13:07:24.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f0071040 0x7f69f0157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43552/0 (socket says 172.21.15.153:43552) 2023-11-28T13:07:24.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 -- 172.21.15.153:0/4004834506 learned_addr learned my addr 172.21.15.153:0/4004834506 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:24.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f67a6700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f0157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:24.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 -- 172.21.15.153:0/4004834506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 msgr2=0x7f69f0157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:24.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f0157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 -- 172.21.15.153:0/4004834506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f006dee0 msgr2=0x7f69f00ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:24.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f006dee0 0x7f69f00ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:24.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 -- 172.21.15.153:0/4004834506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00094a0 con 0x7f69f0071040 2023-11-28T13:07:24.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.870+0000 7f69f57a4700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f0071040 0x7f69f0157150 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f69ec001b70 tx=0x7f69ec014330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:24.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.871+0000 7f69e67fc700 1 -- 172.21.15.153:0/4004834506 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec017070 con 0x7f69f0071040 2023-11-28T13:07:24.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.871+0000 7f69e67fc700 1 -- 172.21.15.153:0/4004834506 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec001d10 con 0x7f69f0071040 2023-11-28T13:07:24.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.871+0000 7f69e67fc700 1 -- 172.21.15.153:0/4004834506 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec014df0 con 0x7f69f0071040 2023-11-28T13:07:24.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.871+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e8009a70 con 0x7f69f0071040 2023-11-28T13:07:24.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.871+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f015ccd0 con 0x7f69f0071040 2023-11-28T13:07:24.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.872+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f015d900 con 0x7f69f0071040 2023-11-28T13:07:24.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.873+0000 7f69e67fc700 1 -- 172.21.15.153:0/4004834506 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f69ec022020 con 0x7f69f0071040 2023-11-28T13:07:24.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.874+0000 7f69e67fc700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f69d40682f0 0x7f69d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:24.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.874+0000 7f69e67fc700 1 -- 172.21.15.153:0/4004834506 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f69ec08d3b0 con 0x7f69f0071040 2023-11-28T13:07:24.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.874+0000 7f69f5fa5700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f69d40682f0 0x7f69d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:24.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.874+0000 7f69f5fa5700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f69d40682f0 0x7f69d406a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f69e0006f10 tx=0x7f69e0006e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:24.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:24.878+0000 7f69e67fc700 1 -- 172.21.15.153:0/4004834506 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f69ec059e40 con 0x7f69f0071040 2023-11-28T13:07:25.064 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.063+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f69f006a270 con 0x7f69f0071040 2023-11-28T13:07:25.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.067+0000 7f69e67fc700 1 -- 172.21.15.153:0/4004834506 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f69ec05d460 con 0x7f69f0071040 2023-11-28T13:07:25.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f69d40682f0 msgr2=0x7f69d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f69d40682f0 0x7f69d406a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f69e0006f10 tx=0x7f69e0006e60 comp rx=0 tx=0).stop 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f0071040 msgr2=0x7f69f0157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f0071040 0x7f69f0157150 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f69ec001b70 tx=0x7f69ec014330 comp rx=0 tx=0).stop 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 shutdown_connections 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69f0160540 0x7f69f0157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69f006dee0 0x7f69f00ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f69d40682f0 0x7f69d406a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 --2- 172.21.15.153:0/4004834506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69f0071040 0x7f69f0157150 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 >> 172.21.15.153:0/4004834506 conn(0x7f69f00d0000 msgr2=0x7f69f00cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 shutdown_connections 2023-11-28T13:07:25.071 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.069+0000 7f69f8209700 1 -- 172.21.15.153:0/4004834506 wait complete. 2023-11-28T13:07:25.082 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T13:07:25.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.203+0000 7f9106389700 1 Processor -- start 2023-11-28T13:07:25.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.203+0000 7f9106389700 1 -- start start 2023-11-28T13:07:25.208 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.204+0000 7f9106389700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9100071040 0x7f9100071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.204+0000 7f9106389700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 0x7f91001609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.204+0000 7f9106389700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9100071950 con 0x7f910006dee0 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.204+0000 7f9106389700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910015d060 con 0x7f9100071040 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.204+0000 7f9106389700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910015d340 con 0x7f9100160540 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.204+0000 7f90ff7fe700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f910006dee0 0x7f910006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33492/0 (socket says 172.21.15.153:33492) 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.204+0000 7f90ff7fe700 1 -- 172.21.15.153:0/2599537491 learned_addr learned my addr 172.21.15.153:0/2599537491 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 --2- 172.21.15.153:0/2599537491 >> v2:172.21.15.177:3300/0 conn(0x7f9100071040 0x7f9100071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f9104926700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 0x7f91001609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833390974 0 0) 0x7f9100071950 con 0x7f910006dee0 2023-11-28T13:07:25.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90ec0035b0 con 0x7f910006dee0 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4135544501 0 0) 0x7f90ec0035b0 con 0x7f910006dee0 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 msgr2=0x7f91001609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 0x7f91001609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 >> v2:172.21.15.177:3300/0 conn(0x7f9100071040 msgr2=0x7f9100071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 --2- 172.21.15.153:0/2599537491 >> v2:172.21.15.177:3300/0 conn(0x7f9100071040 0x7f9100071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910015d620 con 0x7f910006dee0 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f90e8001670 con 0x7f910006dee0 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 >> v1:172.21.15.153:6789/0 conn(0x7f910006dee0 legacy=0x7f910006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f90ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 0x7f90ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90ec004d90 0x7f90ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910015d060 con 0x7f90ec003a00 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910015d340 con 0x7f90ec004d90 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec00ba90 con 0x7f9100160540 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f90ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90ff7fe700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 0x7f90ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f9104926700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90ec004d90 0x7f90ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 -- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 msgr2=0x7f90ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100160540 0x7f90ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 -- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90ec004d90 msgr2=0x7f90ec005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90ec004d90 0x7f90ec005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 -- 172.21.15.153:0/2599537491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec0035b0 con 0x7f90ec003a00 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.205+0000 7f90fffff700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f90ec003e10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009b60 tx=0x7f90ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c37d9475c59f53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000e6c0 con 0x7f90ec003a00 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f000e820 con 0x7f90ec003a00 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f90feffd700 1 -- 172.21.15.153:0/2599537491 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000eb90 con 0x7f90ec003a00 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f9106389700 1 -- 172.21.15.153:0/2599537491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 msgr2=0x7f90ec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f9106389700 1 --2- 172.21.15.153:0/2599537491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f90ec003e10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009b60 tx=0x7f90ec005880 comp rx=0 tx=0).stop 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f9104926700 1 -- 172.21.15.153:0/2599537491 reap_dead start 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f9106389700 1 -- 172.21.15.153:0/2599537491 shutdown_connections 2023-11-28T13:07:25.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f9106389700 1 -- 172.21.15.153:0/2599537491 >> 172.21.15.153:0/2599537491 conn(0x7f91000d0000 msgr2=0x7f91000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:25.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f9106389700 1 -- 172.21.15.153:0/2599537491 shutdown_connections 2023-11-28T13:07:25.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.206+0000 7f9106389700 1 -- 172.21.15.153:0/2599537491 wait complete. 2023-11-28T13:07:25.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.208+0000 7f9106389700 1 Processor -- start 2023-11-28T13:07:25.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.208+0000 7f9106389700 1 -- start start 2023-11-28T13:07:25.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9106389700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f9100147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9106389700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910014b760 0x7f91001f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9106389700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91001f64d0 0x7f91001f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9106389700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001f6040 con 0x7f90ec003a00 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9106389700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100148bd0 con 0x7f910014b760 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9106389700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9100148330 con 0x7f91001f64d0 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f9100147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f9100147ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43570/0 (socket says 172.21.15.153:43570) 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 -- 172.21.15.153:0/2434097017 learned_addr learned my addr 172.21.15.153:0/2434097017 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9104926700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91001f64d0 0x7f91001f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90ff7fe700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910014b760 0x7f91001f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 -- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91001f64d0 msgr2=0x7f91001f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91001f64d0 0x7f91001f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 -- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910014b760 msgr2=0x7f91001f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910014b760 0x7f91001f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 -- 172.21.15.153:0/2434097017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910015ccd0 con 0x7f90ec003a00 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f9104926700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91001f64d0 0x7f91001f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fffff700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f9100147ca0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e4b0 tx=0x7f90f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.209+0000 7f90fcff9700 1 -- 172.21.15.153:0/2434097017 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f000e6c0 con 0x7f90ec003a00 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.210+0000 7f90fcff9700 1 -- 172.21.15.153:0/2434097017 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f0003ad0 con 0x7f90ec003a00 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.210+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910015d900 con 0x7f90ec003a00 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.210+0000 7f90fcff9700 1 -- 172.21.15.153:0/2434097017 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0003e80 con 0x7f90ec003a00 2023-11-28T13:07:25.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.210+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910006a270 con 0x7f90ec003a00 2023-11-28T13:07:25.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.211+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91001f6320 con 0x7f90ec003a00 2023-11-28T13:07:25.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.211+0000 7f90fcff9700 1 -- 172.21.15.153:0/2434097017 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90f0019030 con 0x7f90ec003a00 2023-11-28T13:07:25.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.211+0000 7f90fcff9700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00682f0 0x7f90e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.211+0000 7f90fcff9700 1 -- 172.21.15.153:0/2434097017 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f90f00870b0 con 0x7f90ec003a00 2023-11-28T13:07:25.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.211+0000 7f90ff7fe700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00682f0 0x7f90e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.212+0000 7f90ff7fe700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00682f0 0x7f90e006a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9100157c70 tx=0x7f90e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:25.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.216+0000 7f90fcff9700 1 -- 172.21.15.153:0/2434097017 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90f0053c10 con 0x7f90ec003a00 2023-11-28T13:07:25.472 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.471+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f91000720a0 con 0x7f90ec003a00 2023-11-28T13:07:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.714+0000 7f90fcff9700 1 -- 172.21.15.153:0/2434097017 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v323) v1 ==== 169+0+0 (secure 0 0 0) 0x7f90f0057230 con 0x7f90ec003a00 2023-11-28T13:07:25.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00682f0 msgr2=0x7f90e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.717 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00682f0 0x7f90e006a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9100157c70 tx=0x7f90e8009040 comp rx=0 tx=0).stop 2023-11-28T13:07:25.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 msgr2=0x7f9100147ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f9100147ca0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e4b0 tx=0x7f90f00047c0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 shutdown_connections 2023-11-28T13:07:25.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f90e00682f0 0x7f90e006a7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90ec003a00 0x7f9100147ca0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91001f64d0 0x7f91001f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 --2- 172.21.15.153:0/2434097017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f910014b760 0x7f91001f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 >> 172.21.15.153:0/2434097017 conn(0x7f91000d0000 msgr2=0x7f91000cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:25.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 shutdown_connections 2023-11-28T13:07:25.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:25.716+0000 7f9106389700 1 -- 172.21.15.153:0/2434097017 wait complete. 2023-11-28T13:07:25.730 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T13:07:25.730 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:25.730 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=200) 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 200s and 0 retries 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:25.731 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner:Ran 3 tests in 397.975s 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=200) 2023-11-28T13:07:25.732 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T13:07:25.733 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T13:07:25.733 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 200s and 0 retries 2023-11-28T13:07:25.733 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:25.733 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:25.833 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ddd754f5f1d143ffb59c580558af7612 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:25.834 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T13:07:25.844 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T13:07:25.865 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T13:07:25.865 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:07:25.866 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:25.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:07:25.886 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2023-11-28T13:07:25.886 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:25.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20795 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:07:25.944 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:07:25.944 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:25.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:25.966 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T13:07:25.966 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:26.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18768 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:26.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.026 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:07:26.043 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:07:26.044 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:07:26.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.045 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:07:26.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.123 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:07:26.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.206 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-11-28T13:07:26.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.351 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2023-11-28T13:07:26.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.368 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2023-11-28T13:07:26.438 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2023-11-28T13:07:26.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:07:26.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.440 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T13:07:26.460 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T13:07:26.460 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T13:07:26.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.461 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T13:07:26.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.544 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:07:26.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.632 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T13:07:26.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.715 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T13:07:26.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:26.733 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T13:07:26.802 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-11-28T13:07:26.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:07:26.805 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T13:07:26.815 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T13:07:26.815 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:26.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.937+0000 7fa6fd632700 1 Processor -- start 2023-11-28T13:07:26.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.937+0000 7fa6fd632700 1 -- start start 2023-11-28T13:07:26.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6fd632700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f815ca50 0x7fa6f806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6fd632700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6f80cce10 0x7fa6f80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6fd632700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80cf5e0 con 0x7fa6f806b8c0 2023-11-28T13:07:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6fd632700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80cd290 con 0x7fa6f80cce10 2023-11-28T13:07:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6fd632700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80cd470 con 0x7fa6f815ca50 2023-11-28T13:07:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6f80cce10 0x7fa6f80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6f80cce10 0x7fa6f80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43582/0 (socket says 172.21.15.153:43582) 2023-11-28T13:07:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 -- 172.21.15.153:0/3105596654 learned_addr learned my addr 172.21.15.153:0/3105596654 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:26.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f6ffd700 1 --2- 172.21.15.153:0/3105596654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f815ca50 0x7fa6f806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:26.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 -- 172.21.15.153:0/3105596654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f815ca50 msgr2=0x7fa6f806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:26.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 --2- 172.21.15.153:0/3105596654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f815ca50 0x7fa6f806b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:26.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 -- 172.21.15.153:0/3105596654 >> v1:172.21.15.177:6789/0 conn(0x7fa6f806b8c0 legacy=0x7fa6f80cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:26.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 -- 172.21.15.153:0/3105596654 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80cecb0 con 0x7fa6f80cce10 2023-11-28T13:07:26.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f6ffd700 1 --2- 172.21.15.153:0/3105596654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f815ca50 0x7fa6f806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:26.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f77fe700 1 --2- 172.21.15.153:0/3105596654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f80cce10 0x7fa6f80cfa70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009e80 tx=0x7fa6f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5732a20620fa4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f5ffb700 1 -- 172.21.15.153:0/3105596654 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000e070 con 0x7fa6f80cce10 2023-11-28T13:07:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f5ffb700 1 -- 172.21.15.153:0/3105596654 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000b980 con 0x7fa6f80cce10 2023-11-28T13:07:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.938+0000 7fa6f5ffb700 1 -- 172.21.15.153:0/3105596654 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000bd30 con 0x7fa6f80cce10 2023-11-28T13:07:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 -- 172.21.15.153:0/3105596654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f80cce10 msgr2=0x7fa6f80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 --2- 172.21.15.153:0/3105596654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f80cce10 0x7fa6f80cfa70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009e80 tx=0x7fa6f000b330 comp rx=0 tx=0).stop 2023-11-28T13:07:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 -- 172.21.15.153:0/3105596654 shutdown_connections 2023-11-28T13:07:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 --2- 172.21.15.153:0/3105596654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f80cce10 0x7fa6f80cfa70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:26.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 --2- 172.21.15.153:0/3105596654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f815ca50 0x7fa6f806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:26.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 -- 172.21.15.153:0/3105596654 >> 172.21.15.153:0/3105596654 conn(0x7fa6f809a230 msgr2=0x7fa6f808afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:26.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 -- 172.21.15.153:0/3105596654 shutdown_connections 2023-11-28T13:07:26.944 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.939+0000 7fa6fd632700 1 -- 172.21.15.153:0/3105596654 wait complete. 2023-11-28T13:07:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.941+0000 7fa6fd632700 1 Processor -- start 2023-11-28T13:07:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.941+0000 7fa6fd632700 1 -- start start 2023-11-28T13:07:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6fd632700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f806b8c0 0x7fa6f81574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6fd632700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f80cce10 0x7fa6f80b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6fd632700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f815ca50 0x7fa6f815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6fd632700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8160230 con 0x7fa6f806b8c0 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6fd632700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80ae520 con 0x7fa6f815ca50 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6fd632700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80adc30 con 0x7fa6f80cce10 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f806b8c0 0x7fa6f81574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f806b8c0 0x7fa6f81574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43584/0 (socket says 172.21.15.153:43584) 2023-11-28T13:07:26.948 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 -- 172.21.15.153:0/41676857 learned_addr learned my addr 172.21.15.153:0/41676857 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:26.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f77fe700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f815ca50 0x7fa6f815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:26.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f67fc700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f80cce10 0x7fa6f80b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:26.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 -- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f80cce10 msgr2=0x7fa6f80b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:26.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f80cce10 0x7fa6f80b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:26.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 -- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f815ca50 msgr2=0x7fa6f815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:26.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f815ca50 0x7fa6f815d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:26.949 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 -- 172.21.15.153:0/41676857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0009b30 con 0x7fa6f806b8c0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6f6ffd700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f806b8c0 0x7fa6f81574d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800e790 tx=0x7fa6e800eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6df7fe700 1 -- 172.21.15.153:0/41676857 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800c070 con 0x7fa6f806b8c0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6df7fe700 1 -- 172.21.15.153:0/41676857 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8013610 con 0x7fa6f806b8c0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.942+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f80cf270 con 0x7fa6f806b8c0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.943+0000 7fa6df7fe700 1 -- 172.21.15.153:0/41676857 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80139c0 con 0x7fa6f806b8c0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.943+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f80cef90 con 0x7fa6f806b8c0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.943+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f815ce20 con 0x7fa6f806b8c0 2023-11-28T13:07:26.950 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.944+0000 7fa6df7fe700 1 -- 172.21.15.153:0/41676857 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6e8023070 con 0x7fa6f806b8c0 2023-11-28T13:07:26.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.944+0000 7fa6df7fe700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6e40682f0 0x7fa6e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:26.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.944+0000 7fa6df7fe700 1 -- 172.21.15.153:0/41676857 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa6e8085580 con 0x7fa6f806b8c0 2023-11-28T13:07:26.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.944+0000 7fa6f67fc700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6e40682f0 0x7fa6e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:26.951 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.945+0000 7fa6f67fc700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6e40682f0 0x7fa6e406a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6f81f5660 tx=0x7fa6ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:26.952 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:26.948+0000 7fa6df7fe700 1 -- 172.21.15.153:0/41676857 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6e80520e0 con 0x7fa6f806b8c0 2023-11-28T13:07:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.125+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6f80ae660 con 0x7fa6e40682f0 2023-11-28T13:07:27.128 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.127+0000 7fa6df7fe700 1 -- 172.21.15.153:0/41676857 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73230 (secure 0 0 0) 0x7fa6ec001560 con 0x7fa6e40682f0 2023-11-28T13:07:27.129 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:27.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.129+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6e40682f0 msgr2=0x7fa6e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.129+0000 7fa6fd632700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6e40682f0 0x7fa6e406a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6f81f5660 tx=0x7fa6ec009040 comp rx=0 tx=0).stop 2023-11-28T13:07:27.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.129+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f806b8c0 msgr2=0x7fa6f81574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.131 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.129+0000 7fa6fd632700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f806b8c0 0x7fa6f81574d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800e790 tx=0x7fa6e800eaa0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 shutdown_connections 2023-11-28T13:07:27.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6e40682f0 0x7fa6e406a7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f806b8c0 0x7fa6f81574d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f815ca50 0x7fa6f815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.132 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 --2- 172.21.15.153:0/41676857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f80cce10 0x7fa6f80b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 >> 172.21.15.153:0/41676857 conn(0x7fa6f809a230 msgr2=0x7fa6f80cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 shutdown_connections 2023-11-28T13:07:27.133 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.130+0000 7fa6fd632700 1 -- 172.21.15.153:0/41676857 wait complete. 2023-11-28T13:07:27.134 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:07:27.142 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-11-28T13:07:26.253716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84004,"kb_used_data":7396,"kb_used_omap":0,"kb_used_meta":76544,"kb_avail":754890716,"statfs":{"total":773094113280,"available":773008093184,"internally_reserved":0,"allocated":7573504,"data_stored":3995712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78381056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001403"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:29.570232+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:06:29.570232+0000","last_peered":"2023-11-28T13:06:29.570232+0000","last_clean":"2023-11-28T13:06:29.570232+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:06:29.570232+0000","last_undegraded":"2023-11-28T13:06:29.570232+0000","last_fullsized":"2023-11-28T13:06:29.570232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.382636+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:06:28.382636+0000","last_peered":"2023-11-28T13:06:28.382636+0000","last_clean":"2023-11-28T13:06:28.382636+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:06:28.382636+0000","last_undegraded":"2023-11-28T13:06:28.382636+0000","last_fullsized":"2023-11-28T13:06:28.382636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:29.569990+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:06:29.569990+0000","last_peered":"2023-11-28T13:06:29.569990+0000","last_clean":"2023-11-28T13:06:29.569990+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:06:29.569990+0000","last_undegraded":"2023-11-28T13:06:29.569990+0000","last_fullsized":"2023-11-28T13:06:29.569990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.382665+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:06:28.382665+0000","last_peered":"2023-11-28T13:06:28.382665+0000","last_clean":"2023-11-28T13:06:28.382665+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:06:28.382665+0000","last_undegraded":"2023-11-28T13:06:28.382665+0000","last_fullsized":"2023-11-28T13:06:28.382665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:08.372261+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:06:08.372261+0000","last_peered":"2023-11-28T13:06:08.372261+0000","last_clean":"2023-11-28T13:06:08.372261+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:06:08.372261+0000","last_undegraded":"2023-11-28T13:06:08.372261+0000","last_fullsized":"2023-11-28T13:06:08.372261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:13.372839+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:06:13.372839+0000","last_peered":"2023-11-28T13:06:13.372839+0000","last_clean":"2023-11-28T13:06:13.372839+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:06:13.372839+0000","last_undegraded":"2023-11-28T13:06:13.372839+0000","last_fullsized":"2023-11-28T13:06:13.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.374619+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:06:28.374619+0000","last_peered":"2023-11-28T13:06:28.374619+0000","last_clean":"2023-11-28T13:06:28.374619+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:06:28.374619+0000","last_undegraded":"2023-11-28T13:06:28.374619+0000","last_fullsized":"2023-11-28T13:06:28.374619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:44.571926+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:06:44.571926+0000","last_peered":"2023-11-28T13:06:44.571926+0000","last_clean":"2023-11-28T13:06:44.571926+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:06:44.571926+0000","last_undegraded":"2023-11-28T13:06:44.571926+0000","last_fullsized":"2023-11-28T13:06:44.571926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:43.376175+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:06:43.376175+0000","last_peered":"2023-11-28T13:06:43.376175+0000","last_clean":"2023-11-28T13:06:43.376175+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:06:43.376175+0000","last_undegraded":"2023-11-28T13:06:43.376175+0000","last_fullsized":"2023-11-28T13:06:43.376175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":802816,"data_stored":354605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.372,"15min":0.371},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":0.497,"5min":0.57699999999999996,"15min":1.2669999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.57399999999999995},"min":{"1min":0.222,"5min":0.222,"15min":0.216},"max":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":8.218},"last":0.22}]},{"osd":1,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.26300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.623,"5min":0.63600000000000001,"15min":1.8280000000000001},"last":0.25},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.41199999999999998},"min":{"1min":0.29699999999999999,"5min":0.218,"15min":0.214},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":1.355},"last":0.23699999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.76400000000000001},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":13.384},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.26500000000000001},"max":{"1min":0.60999999999999999,"5min":0.65200000000000002,"15min":5.226},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.35999999999999999,"15min":0.35699999999999998},"max":{"1min":0.81799999999999995,"5min":0.94499999999999995,"15min":8.1890000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.73399999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78900000000000003,"5min":0.93999999999999995,"15min":5.3609999999999998},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":1.3029999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.80400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.878,"5min":0.96899999999999997,"15min":8.1579999999999995},"last":0.53300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":5.3209999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.3320000000000001},"last":0.50800000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.83099999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":8.1029999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.76000000000000001},"min":{"1min":0.50800000000000001,"5min":0.40300000000000002,"15min":0.38900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":5.3849999999999998},"last":0.52000000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9544,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362296,"statfs":{"total":96636764160,"available":96626991104,"internally_reserved":0,"allocated":786432,"data_stored":348000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35199999999999998,"15min":0.33000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":0.51700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.187},"max":{"1min":0.48599999999999999,"5min":0.61599999999999999,"15min":1.0169999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.224,"15min":0.20499999999999999},"max":{"1min":0.52600000000000002,"5min":0.62,"15min":0.95999999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.221},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":1.1459999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.39200000000000002,"15min":0.36599999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.63400000000000001,"15min":1.1659999999999999},"last":0.375},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.57599999999999996,"5min":0.70299999999999996,"15min":1.1919999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.641},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.91100000000000003,"15min":1.056},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.89800000000000002,"15min":1.6120000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.34499999999999997},"max":{"1min":0.72599999999999998,"5min":0.82899999999999996,"15min":1.774},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.626,"15min":0.58199999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84599999999999997,"15min":1.242},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.92900000000000005,"15min":1.6539999999999999},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.876,"15min":1.6919999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.89300000000000002,"15min":1.2130000000000001},"last":0.65900000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":806912,"data_stored":359788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58099999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35999999999999999},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.56299999999999994},"min":{"1min":0.47199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.753,"15min":1.2270000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.52300000000000002},"min":{"1min":0.45600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73199999999999998,"5min":0.86899999999999999,"15min":1.2110000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.76900000000000002,"15min":0.81200000000000006},"last":0.376},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51000000000000001,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.52900000000000003},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.247},"max":{"1min":0.69099999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":0.83599999999999997,"15min":0.88700000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14856,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356984,"statfs":{"total":96636764160,"available":96621551616,"internally_reserved":0,"allocated":1376256,"data_stored":937599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.33800000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.214},"max":{"1min":0.52800000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.74199999999999999},"min":{"1min":0.34000000000000002,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":0.56200000000000006,"5min":0.61599999999999999,"15min":14.957000000000001},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.747},"min":{"1min":0.36499999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53200000000000003,"5min":0.60499999999999998,"15min":15.032999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.754},"min":{"1min":0.40699999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51900000000000002,"5min":0.59499999999999997,"15min":14.997999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41099999999999998,"15min":0.84699999999999998},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.21199999999999999},"max":{"1min":0.49399999999999999,"5min":0.56999999999999995,"15min":19.045999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.41099999999999998},"min":{"1min":0.40300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60699999999999998,"15min":0.67200000000000004},"min":{"1min":0.53700000000000003,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.1799999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":1.073},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.35299999999999998},"max":{"1min":0.754,"5min":0.89700000000000002,"15min":18.995000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61299999999999999,"15min":1.0600000000000001},"min":{"1min":0.55200000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.93700000000000006,"15min":19.02},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.628,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":4.2119999999999997},"last":0.68600000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63600000000000001,"15min":0.70199999999999996},"min":{"1min":0.57399999999999995,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":4.2359999999999998},"last":0.753},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63500000000000001,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.92500000000000004,"15min":4.1150000000000002},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.70199999999999996},"min":{"1min":0.57999999999999996,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.876,"15min":4.1539999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64500000000000002,"15min":1.085},"min":{"1min":0.63300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":18.957999999999998},"last":0.77800000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10972,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360868,"statfs":{"total":96636764160,"available":96625528832,"internally_reserved":0,"allocated":1396736,"data_stored":939128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.78400000000000003,"15min":1.1699999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.81599999999999995,"15min":1.415},"last":0.624}]},{"osd":1,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.80600000000000005,"15min":1.867},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57299999999999995,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":0.70399999999999996,"5min":0.79700000000000004,"15min":4.7830000000000004},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.84499999999999997,"15min":1.9550000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34000000000000002},"max":{"1min":0.68400000000000005,"5min":0.81499999999999995,"15min":1.9510000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.88600000000000001,"15min":1.6859999999999999},"last":0.503},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73799999999999999,"5min":0.89900000000000002,"15min":4.8440000000000003},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.33300000000000002,"15min":0.28499999999999998},"max":{"1min":0.69999999999999996,"5min":0.873,"15min":2.6030000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":1.528},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.311},"max":{"1min":0.64700000000000002,"5min":0.94699999999999995,"15min":1.25},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71599999999999997,"5min":0.93600000000000005,"15min":2.5329999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":0.65700000000000003,"5min":0.96599999999999997,"15min":1.972},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.98899999999999999,"15min":2.508},"last":0.64500000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9676,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362164,"statfs":{"total":96636764160,"available":96626855936,"internally_reserved":0,"allocated":790528,"data_stored":348022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68999999999999995,"5min":0.82799999999999996,"15min":1.419},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.68100000000000005,"5min":0.78500000000000003,"15min":0.80000000000000004},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.343,"15min":1.343},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70599999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.40799999999999997,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":3.6160000000000001},"last":0.56899999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.81100000000000005,"15min":1.4490000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.74399999999999999,"5min":1.228,"15min":1.228},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46100000000000002,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.624},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":0.85299999999999998,"15min":0.89600000000000002},"last":0.67200000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542236,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9484,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362356,"statfs":{"total":96636764160,"available":96627052544,"internally_reserved":0,"allocated":790528,"data_stored":348034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47799999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.75,"15min":0.75},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":1.0249999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.54800000000000004},"min":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.38},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69499999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.49099999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.76500000000000001,"5min":0.77700000000000002,"15min":0.79500000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78000000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.33100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.83399999999999996},"last":0.80500000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.40000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73199999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9708,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362132,"statfs":{"total":96636764160,"available":96626823168,"internally_reserved":0,"allocated":823296,"data_stored":360536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.46300000000000002,"5min":0.60999999999999999,"15min":0.97099999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39300000000000002,"15min":0.35899999999999999},"min":{"1min":0.25800000000000001,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":0.55400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.25800000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.375,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52900000000000003,"5min":0.57499999999999996,"15min":0.72899999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.41099999999999998,"15min":0.378},"min":{"1min":0.27500000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54500000000000004,"5min":0.60099999999999998,"15min":0.64300000000000002},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26500000000000001},"max":{"1min":0.67000000000000004,"5min":0.69199999999999995,"15min":0.752},"last":0.378},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42699999999999999,"15min":0.40400000000000003},"min":{"1min":0.29399999999999998,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.875},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.58799999999999997},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.36299999999999999},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.499},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":1.044},"last":0.59099999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.47699999999999998,"5min":0.433,"15min":0.28999999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.36799999999999999},"max":{"1min":0.82199999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.46500000000000002,"15min":0.31900000000000001},"max":{"1min":0.83299999999999996,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.38900000000000001},"max":{"1min":0.90000000000000002,"5min":0.93899999999999995,"15min":1.089},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66700000000000004,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":1.202},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.63600000000000001},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":0.95799999999999996,"15min":1.02},"last":0.52700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:27.145 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:27.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.264+0000 7f0c64e0b700 1 Processor -- start 2023-11-28T13:07:27.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.264+0000 7f0c64e0b700 1 -- start start 2023-11-28T13:07:27.267 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c64e0b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0c6006dee0 0x7f0c6006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c64e0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c60160540 0x7f0c601609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c64e0b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6015d060 con 0x7f0c60071040 2023-11-28T13:07:27.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c64e0b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6015d340 con 0x7f0c6006dee0 2023-11-28T13:07:27.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c64e0b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6015d620 con 0x7f0c60160540 2023-11-28T13:07:27.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0c6006dee0 0x7f0c6006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5dd9b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f0c6006dee0 0x7f0c6006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43594/0 (socket says 172.21.15.153:43594) 2023-11-28T13:07:27.268 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/3798958197 learned_addr learned my addr 172.21.15.153:0/3798958197 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:27.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5ed9d700 1 --2- 172.21.15.153:0/3798958197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c60160540 0x7f0c601609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/3798958197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c60160540 msgr2=0x7f0c601609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:27.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5dd9b700 1 --2- 172.21.15.153:0/3798958197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c60160540 0x7f0c601609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/3798958197 >> v1:172.21.15.177:6789/0 conn(0x7f0c60071040 legacy=0x7f0c60071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.265+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/3798958197 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6015d900 con 0x7f0c6006dee0 2023-11-28T13:07:27.269 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c5dd9b700 1 --2- 172.21.15.153:0/3798958197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c6006dee0 0x7f0c6006e330 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0c500098f0 tx=0x7f0c50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec369bd6398b524a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c5d59a700 1 -- 172.21.15.153:0/3798958197 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50011830 con 0x7f0c6006dee0 2023-11-28T13:07:27.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c5d59a700 1 -- 172.21.15.153:0/3798958197 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50011990 con 0x7f0c6006dee0 2023-11-28T13:07:27.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c5d59a700 1 -- 172.21.15.153:0/3798958197 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50011d20 con 0x7f0c6006dee0 2023-11-28T13:07:27.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 -- 172.21.15.153:0/3798958197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c6006dee0 msgr2=0x7f0c6006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/3798958197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c6006dee0 0x7f0c6006e330 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0c500098f0 tx=0x7f0c50009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:27.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 -- 172.21.15.153:0/3798958197 shutdown_connections 2023-11-28T13:07:27.270 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/3798958197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c60160540 0x7f0c601609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/3798958197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c6006dee0 0x7f0c6006e330 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 -- 172.21.15.153:0/3798958197 >> 172.21.15.153:0/3798958197 conn(0x7f0c600d0000 msgr2=0x7f0c600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:27.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 -- 172.21.15.153:0/3798958197 shutdown_connections 2023-11-28T13:07:27.271 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.266+0000 7f0c64e0b700 1 -- 172.21.15.153:0/3798958197 wait complete. 2023-11-28T13:07:27.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 Processor -- start 2023-11-28T13:07:27.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 -- start start 2023-11-28T13:07:27.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6006dee0 0x7f0c60152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c60071040 0x7f0c600b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c60160540 0x7f0c600b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.274 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600c6020 con 0x7f0c60071040 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601f6cb0 con 0x7f0c60160540 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.268+0000 7f0c64e0b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c601f63c0 con 0x7f0c6006dee0 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c60071040 0x7f0c600b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c60071040 0x7f0c600b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43600/0 (socket says 172.21.15.153:43600) 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/1879585931 learned_addr learned my addr 172.21.15.153:0/1879585931 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5e59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6006dee0 0x7f0c60152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5ed9d700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c60160540 0x7f0c600b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/1879585931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6006dee0 msgr2=0x7f0c60152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6006dee0 0x7f0c60152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/1879585931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c60160540 msgr2=0x7f0c600b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c60160540 0x7f0c600b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 -- 172.21.15.153:0/1879585931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50009550 con 0x7f0c60071040 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c5dd9b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c60071040 0x7f0c600b0a50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009c40 tx=0x7f0c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c4effd700 1 -- 172.21.15.153:0/1879585931 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50004510 con 0x7f0c60071040 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c4effd700 1 -- 172.21.15.153:0/1879585931 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5001e070 con 0x7f0c60071040 2023-11-28T13:07:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6015ccd0 con 0x7f0c60071040 2023-11-28T13:07:27.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c4effd700 1 -- 172.21.15.153:0/1879585931 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500038e0 con 0x7f0c60071040 2023-11-28T13:07:27.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.269+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6006e870 con 0x7f0c60071040 2023-11-28T13:07:27.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.270+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6006a270 con 0x7f0c60071040 2023-11-28T13:07:27.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.271+0000 7f0c4effd700 1 -- 172.21.15.153:0/1879585931 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c5000f050 con 0x7f0c60071040 2023-11-28T13:07:27.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.271+0000 7f0c4effd700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0c40068320 0x7f0c4006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.271+0000 7f0c4effd700 1 -- 172.21.15.153:0/1879585931 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c50086790 con 0x7f0c60071040 2023-11-28T13:07:27.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.271+0000 7f0c5e59c700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0c40068320 0x7f0c4006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.271+0000 7f0c5e59c700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0c40068320 0x7f0c4006a7d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c54007fd0 tx=0x7f0c54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.275+0000 7f0c4effd700 1 -- 172.21.15.153:0/1879585931 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c500532f0 con 0x7f0c60071040 2023-11-28T13:07:27.455 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.455+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c601f6e20 con 0x7f0c40068320 2023-11-28T13:07:27.458 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.457+0000 7f0c4effd700 1 -- 172.21.15.153:0/1879585931 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73230 (secure 0 0 0) 0x7f0c601f6e20 con 0x7f0c40068320 2023-11-28T13:07:27.459 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:27.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0c40068320 msgr2=0x7f0c4006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.460 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0c40068320 0x7f0c4006a7d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c54007fd0 tx=0x7f0c54009040 comp rx=0 tx=0).stop 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c60071040 msgr2=0x7f0c600b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c60071040 0x7f0c600b0a50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009c40 tx=0x7f0c50004750 comp rx=0 tx=0).stop 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 shutdown_connections 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c60160540 0x7f0c600b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6006dee0 0x7f0c60152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0c40068320 0x7f0c4006a7d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 --2- 172.21.15.153:0/1879585931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c60071040 0x7f0c600b0a50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.461 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.459+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 >> 172.21.15.153:0/1879585931 conn(0x7f0c600d0000 msgr2=0x7f0c60071410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:27.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.460+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 shutdown_connections 2023-11-28T13:07:27.462 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.460+0000 7f0c64e0b700 1 -- 172.21.15.153:0/1879585931 wait complete. 2023-11-28T13:07:27.462 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:07:27.473 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-11-28T13:07:26.253716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84004,"kb_used_data":7396,"kb_used_omap":0,"kb_used_meta":76544,"kb_avail":754890716,"statfs":{"total":773094113280,"available":773008093184,"internally_reserved":0,"allocated":7573504,"data_stored":3995712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78381056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001403"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:29.570232+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:06:29.570232+0000","last_peered":"2023-11-28T13:06:29.570232+0000","last_clean":"2023-11-28T13:06:29.570232+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:06:29.570232+0000","last_undegraded":"2023-11-28T13:06:29.570232+0000","last_fullsized":"2023-11-28T13:06:29.570232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.382636+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:06:28.382636+0000","last_peered":"2023-11-28T13:06:28.382636+0000","last_clean":"2023-11-28T13:06:28.382636+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:06:28.382636+0000","last_undegraded":"2023-11-28T13:06:28.382636+0000","last_fullsized":"2023-11-28T13:06:28.382636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:29.569990+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:06:29.569990+0000","last_peered":"2023-11-28T13:06:29.569990+0000","last_clean":"2023-11-28T13:06:29.569990+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:06:29.569990+0000","last_undegraded":"2023-11-28T13:06:29.569990+0000","last_fullsized":"2023-11-28T13:06:29.569990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.382665+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:06:28.382665+0000","last_peered":"2023-11-28T13:06:28.382665+0000","last_clean":"2023-11-28T13:06:28.382665+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:06:28.382665+0000","last_undegraded":"2023-11-28T13:06:28.382665+0000","last_fullsized":"2023-11-28T13:06:28.382665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:08.372261+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:06:08.372261+0000","last_peered":"2023-11-28T13:06:08.372261+0000","last_clean":"2023-11-28T13:06:08.372261+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:06:08.372261+0000","last_undegraded":"2023-11-28T13:06:08.372261+0000","last_fullsized":"2023-11-28T13:06:08.372261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:13.372839+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:06:13.372839+0000","last_peered":"2023-11-28T13:06:13.372839+0000","last_clean":"2023-11-28T13:06:13.372839+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:06:13.372839+0000","last_undegraded":"2023-11-28T13:06:13.372839+0000","last_fullsized":"2023-11-28T13:06:13.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.374619+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:06:28.374619+0000","last_peered":"2023-11-28T13:06:28.374619+0000","last_clean":"2023-11-28T13:06:28.374619+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:06:28.374619+0000","last_undegraded":"2023-11-28T13:06:28.374619+0000","last_fullsized":"2023-11-28T13:06:28.374619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:44.571926+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:06:44.571926+0000","last_peered":"2023-11-28T13:06:44.571926+0000","last_clean":"2023-11-28T13:06:44.571926+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:06:44.571926+0000","last_undegraded":"2023-11-28T13:06:44.571926+0000","last_fullsized":"2023-11-28T13:06:44.571926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:43.376175+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:06:43.376175+0000","last_peered":"2023-11-28T13:06:43.376175+0000","last_clean":"2023-11-28T13:06:43.376175+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:06:43.376175+0000","last_undegraded":"2023-11-28T13:06:43.376175+0000","last_fullsized":"2023-11-28T13:06:43.376175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":802816,"data_stored":354605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.372,"15min":0.371},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":0.497,"5min":0.57699999999999996,"15min":1.2669999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.57399999999999995},"min":{"1min":0.222,"5min":0.222,"15min":0.216},"max":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":8.218},"last":0.22}]},{"osd":1,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.26300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.623,"5min":0.63600000000000001,"15min":1.8280000000000001},"last":0.25},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.41199999999999998},"min":{"1min":0.29699999999999999,"5min":0.218,"15min":0.214},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":1.355},"last":0.23699999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.76400000000000001},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":13.384},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.26500000000000001},"max":{"1min":0.60999999999999999,"5min":0.65200000000000002,"15min":5.226},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.35999999999999999,"15min":0.35699999999999998},"max":{"1min":0.81799999999999995,"5min":0.94499999999999995,"15min":8.1890000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.73399999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78900000000000003,"5min":0.93999999999999995,"15min":5.3609999999999998},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":1.3029999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.80400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.878,"5min":0.96899999999999997,"15min":8.1579999999999995},"last":0.53300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":5.3209999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.3320000000000001},"last":0.50800000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.83099999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":8.1029999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.76000000000000001},"min":{"1min":0.50800000000000001,"5min":0.40300000000000002,"15min":0.38900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":5.3849999999999998},"last":0.52000000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9544,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362296,"statfs":{"total":96636764160,"available":96626991104,"internally_reserved":0,"allocated":786432,"data_stored":348000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35199999999999998,"15min":0.33000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":0.51700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.187},"max":{"1min":0.48599999999999999,"5min":0.61599999999999999,"15min":1.0169999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.224,"15min":0.20499999999999999},"max":{"1min":0.52600000000000002,"5min":0.62,"15min":0.95999999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.221},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":1.1459999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.39200000000000002,"15min":0.36599999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.63400000000000001,"15min":1.1659999999999999},"last":0.375},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.57599999999999996,"5min":0.70299999999999996,"15min":1.1919999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.641},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.91100000000000003,"15min":1.056},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.89800000000000002,"15min":1.6120000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.34499999999999997},"max":{"1min":0.72599999999999998,"5min":0.82899999999999996,"15min":1.774},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.626,"15min":0.58199999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84599999999999997,"15min":1.242},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.92900000000000005,"15min":1.6539999999999999},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.876,"15min":1.6919999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.89300000000000002,"15min":1.2130000000000001},"last":0.65900000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":806912,"data_stored":359788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58099999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35999999999999999},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.56299999999999994},"min":{"1min":0.47199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.753,"15min":1.2270000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.52300000000000002},"min":{"1min":0.45600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73199999999999998,"5min":0.86899999999999999,"15min":1.2110000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.76900000000000002,"15min":0.81200000000000006},"last":0.376},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51000000000000001,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.52900000000000003},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.247},"max":{"1min":0.69099999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":0.83599999999999997,"15min":0.88700000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14856,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356984,"statfs":{"total":96636764160,"available":96621551616,"internally_reserved":0,"allocated":1376256,"data_stored":937599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.33800000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.214},"max":{"1min":0.52800000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.74199999999999999},"min":{"1min":0.34000000000000002,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":0.56200000000000006,"5min":0.61599999999999999,"15min":14.957000000000001},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.747},"min":{"1min":0.36499999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53200000000000003,"5min":0.60499999999999998,"15min":15.032999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.754},"min":{"1min":0.40699999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51900000000000002,"5min":0.59499999999999997,"15min":14.997999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41099999999999998,"15min":0.84699999999999998},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.21199999999999999},"max":{"1min":0.49399999999999999,"5min":0.56999999999999995,"15min":19.045999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.41099999999999998},"min":{"1min":0.40300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60699999999999998,"15min":0.67200000000000004},"min":{"1min":0.53700000000000003,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.1799999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":1.073},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.35299999999999998},"max":{"1min":0.754,"5min":0.89700000000000002,"15min":18.995000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61299999999999999,"15min":1.0600000000000001},"min":{"1min":0.55200000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.93700000000000006,"15min":19.02},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.628,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":4.2119999999999997},"last":0.68600000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63600000000000001,"15min":0.70199999999999996},"min":{"1min":0.57399999999999995,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":4.2359999999999998},"last":0.753},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63500000000000001,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.92500000000000004,"15min":4.1150000000000002},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.70199999999999996},"min":{"1min":0.57999999999999996,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.876,"15min":4.1539999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64500000000000002,"15min":1.085},"min":{"1min":0.63300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":18.957999999999998},"last":0.77800000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10972,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360868,"statfs":{"total":96636764160,"available":96625528832,"internally_reserved":0,"allocated":1396736,"data_stored":939128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.78400000000000003,"15min":1.1699999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.81599999999999995,"15min":1.415},"last":0.624}]},{"osd":1,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.80600000000000005,"15min":1.867},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57299999999999995,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":0.70399999999999996,"5min":0.79700000000000004,"15min":4.7830000000000004},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.84499999999999997,"15min":1.9550000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34000000000000002},"max":{"1min":0.68400000000000005,"5min":0.81499999999999995,"15min":1.9510000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.88600000000000001,"15min":1.6859999999999999},"last":0.503},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73799999999999999,"5min":0.89900000000000002,"15min":4.8440000000000003},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.33300000000000002,"15min":0.28499999999999998},"max":{"1min":0.69999999999999996,"5min":0.873,"15min":2.6030000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":1.528},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.311},"max":{"1min":0.64700000000000002,"5min":0.94699999999999995,"15min":1.25},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71599999999999997,"5min":0.93600000000000005,"15min":2.5329999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":0.65700000000000003,"5min":0.96599999999999997,"15min":1.972},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.98899999999999999,"15min":2.508},"last":0.64500000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9676,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362164,"statfs":{"total":96636764160,"available":96626855936,"internally_reserved":0,"allocated":790528,"data_stored":348022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68999999999999995,"5min":0.82799999999999996,"15min":1.419},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.68100000000000005,"5min":0.78500000000000003,"15min":0.80000000000000004},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.343,"15min":1.343},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70599999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.40799999999999997,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":3.6160000000000001},"last":0.56899999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.81100000000000005,"15min":1.4490000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.74399999999999999,"5min":1.228,"15min":1.228},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46100000000000002,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.624},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":0.85299999999999998,"15min":0.89600000000000002},"last":0.67200000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542236,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9484,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362356,"statfs":{"total":96636764160,"available":96627052544,"internally_reserved":0,"allocated":790528,"data_stored":348034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47799999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.75,"15min":0.75},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":1.0249999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.54800000000000004},"min":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.38},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69499999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.49099999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.76500000000000001,"5min":0.77700000000000002,"15min":0.79500000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78000000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.33100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.83399999999999996},"last":0.80500000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.40000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73199999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9708,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362132,"statfs":{"total":96636764160,"available":96626823168,"internally_reserved":0,"allocated":823296,"data_stored":360536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.46300000000000002,"5min":0.60999999999999999,"15min":0.97099999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39300000000000002,"15min":0.35899999999999999},"min":{"1min":0.25800000000000001,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":0.55400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.25800000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.375,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52900000000000003,"5min":0.57499999999999996,"15min":0.72899999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.41099999999999998,"15min":0.378},"min":{"1min":0.27500000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54500000000000004,"5min":0.60099999999999998,"15min":0.64300000000000002},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26500000000000001},"max":{"1min":0.67000000000000004,"5min":0.69199999999999995,"15min":0.752},"last":0.378},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42699999999999999,"15min":0.40400000000000003},"min":{"1min":0.29399999999999998,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.875},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.58799999999999997},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.36299999999999999},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.499},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":1.044},"last":0.59099999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.47699999999999998,"5min":0.433,"15min":0.28999999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.36799999999999999},"max":{"1min":0.82199999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.46500000000000002,"15min":0.31900000000000001},"max":{"1min":0.83299999999999996,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.38900000000000001},"max":{"1min":0.90000000000000002,"5min":0.93899999999999995,"15min":1.089},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66700000000000004,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":1.202},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.63600000000000001},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":0.95799999999999996,"15min":1.02},"last":0.52700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:27.475 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T13:07:27.475 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:27.597 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.596+0000 7fa6f6355700 1 Processor -- start 2023-11-28T13:07:27.598 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.597+0000 7fa6f6355700 1 -- start start 2023-11-28T13:07:27.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.597+0000 7fa6f6355700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6f00d0a00 0x7fa6f015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.597+0000 7fa6f6355700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.597+0000 7fa6f6355700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f006b420 con 0x7fa6f00ce040 2023-11-28T13:07:27.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.597+0000 7fa6f6355700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f00b32f0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.597+0000 7fa6f6355700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f006ff60 con 0x7fa6f00b2930 2023-11-28T13:07:27.600 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.597+0000 7fa6ef7fe700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fa6f00d0a00 0x7fa6f015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6effff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa6f00ce040 0x7fa6f006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33538/0 (socket says 172.21.15.153:33538) 2023-11-28T13:07:27.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6effff700 1 -- 172.21.15.153:0/2866207788 learned_addr learned my addr 172.21.15.153:0/2866207788 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:27.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6f48f2700 1 --2- 172.21.15.153:0/2866207788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6ef7fe700 1 -- 172.21.15.153:0/2866207788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 msgr2=0x7fa6f00b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:27.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6ef7fe700 1 --2- 172.21.15.153:0/2866207788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6ef7fe700 1 -- 172.21.15.153:0/2866207788 >> v1:172.21.15.153:6789/0 conn(0x7fa6f00ce040 legacy=0x7fa6f006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.601 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6ef7fe700 1 -- 172.21.15.153:0/2866207788 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f006b700 con 0x7fa6f00d0a00 2023-11-28T13:07:27.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6eeffd700 1 -- 172.21.15.153:0/2866207788 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275367992 0 0) 0x7fa6f006b420 con 0x7fa6f00ce040 2023-11-28T13:07:27.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6ef7fe700 1 --2- 172.21.15.153:0/2866207788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f015df10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009840 tx=0x7fa6d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff33afa0b7d3ff73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6eeffd700 1 -- 172.21.15.153:0/2866207788 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011830 con 0x7fa6f00d0a00 2023-11-28T13:07:27.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6eeffd700 1 -- 172.21.15.153:0/2866207788 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d8011990 con 0x7fa6f00d0a00 2023-11-28T13:07:27.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.598+0000 7fa6eeffd700 1 -- 172.21.15.153:0/2866207788 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011d20 con 0x7fa6f00d0a00 2023-11-28T13:07:27.602 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 -- 172.21.15.153:0/2866207788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 msgr2=0x7fa6f015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 --2- 172.21.15.153:0/2866207788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f015df10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009840 tx=0x7fa6d8009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:27.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 -- 172.21.15.153:0/2866207788 shutdown_connections 2023-11-28T13:07:27.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 --2- 172.21.15.153:0/2866207788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 --2- 172.21.15.153:0/2866207788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f015df10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 -- 172.21.15.153:0/2866207788 >> 172.21.15.153:0/2866207788 conn(0x7fa6f006aed0 msgr2=0x7fa6f006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:27.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 -- 172.21.15.153:0/2866207788 shutdown_connections 2023-11-28T13:07:27.603 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.599+0000 7fa6f6355700 1 -- 172.21.15.153:0/2866207788 wait complete. 2023-11-28T13:07:27.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.600+0000 7fa6f6355700 1 Processor -- start 2023-11-28T13:07:27.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.600+0000 7fa6f6355700 1 -- start start 2023-11-28T13:07:27.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f6355700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f6355700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f00ce040 0x7fa6f01f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f6355700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f00ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f6355700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f00b43e0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f6355700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f01f2260 con 0x7fa6f00ce040 2023-11-28T13:07:27.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f6355700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f00c7070 con 0x7fa6f00b2930 2023-11-28T13:07:27.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f00ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f00ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43616/0 (socket says 172.21.15.153:43616) 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 -- 172.21.15.153:0/1295497567 learned_addr learned my addr 172.21.15.153:0/1295497567 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6effff700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6ef7fe700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f00ce040 0x7fa6f01f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 -- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 msgr2=0x7fa6f00c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 -- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f00ce040 msgr2=0x7fa6f01f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f00ce040 0x7fa6f01f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 -- 172.21.15.153:0/1295497567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d80094a0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6ef7fe700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f00ce040 0x7fa6f01f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:27.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6effff700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:27.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6f48f2700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f00ccb70 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800cc30 tx=0x7fa6e800a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.601+0000 7fa6ecff9700 1 -- 172.21.15.153:0/1295497567 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800ab70 con 0x7fa6f00d0a00 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.602+0000 7fa6ecff9700 1 -- 172.21.15.153:0/1295497567 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e800acd0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.602+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f006bcc0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.602+0000 7fa6ecff9700 1 -- 172.21.15.153:0/1295497567 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8003c40 con 0x7fa6f00d0a00 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.602+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f006b9e0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.602+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f006bfa0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.603+0000 7fa6ecff9700 1 -- 172.21.15.153:0/1295497567 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6e801f070 con 0x7fa6f00d0a00 2023-11-28T13:07:27.611 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.603+0000 7fa6ecff9700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6d00682f0 0x7fa6d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.603+0000 7fa6ecff9700 1 -- 172.21.15.153:0/1295497567 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa6e8085b00 con 0x7fa6f00d0a00 2023-11-28T13:07:27.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.603+0000 7fa6effff700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6d00682f0 0x7fa6d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.612 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.604+0000 7fa6effff700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6d00682f0 0x7fa6d006a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0001730 tx=0x7fa6e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.613 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.608+0000 7fa6ecff9700 1 -- 172.21.15.153:0/1295497567 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6e80525d0 con 0x7fa6f00d0a00 2023-11-28T13:07:27.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.786+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6f0068690 con 0x7fa6d00682f0 2023-11-28T13:07:27.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.788+0000 7fa6ecff9700 1 -- 172.21.15.153:0/1295497567 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73230 (secure 0 0 0) 0x7fa6f0068690 con 0x7fa6d00682f0 2023-11-28T13:07:27.790 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:27.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6d00682f0 msgr2=0x7fa6d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.791 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6d00682f0 0x7fa6d006a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0001730 tx=0x7fa6e0009040 comp rx=0 tx=0).stop 2023-11-28T13:07:27.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 msgr2=0x7fa6f00ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f00ccb70 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800cc30 tx=0x7fa6e800a1f0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 shutdown_connections 2023-11-28T13:07:27.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa6d00682f0 0x7fa6d006a7a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f00d0a00 0x7fa6f00ccb70 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.792 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6f00b2930 0x7fa6f00c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 --2- 172.21.15.153:0/1295497567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6f00ce040 0x7fa6f01f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.790+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 >> 172.21.15.153:0/1295497567 conn(0x7fa6f006aed0 msgr2=0x7fa6f015df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:27.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.791+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 shutdown_connections 2023-11-28T13:07:27.793 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.791+0000 7fa6f6355700 1 -- 172.21.15.153:0/1295497567 wait complete. 2023-11-28T13:07:27.793 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:07:27.803 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-11-28T13:07:26.253716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84004,"kb_used_data":7396,"kb_used_omap":0,"kb_used_meta":76544,"kb_avail":754890716,"statfs":{"total":773094113280,"available":773008093184,"internally_reserved":0,"allocated":7573504,"data_stored":3995712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78381056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-75,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001403"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:29.570232+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:06:29.570232+0000","last_peered":"2023-11-28T13:06:29.570232+0000","last_clean":"2023-11-28T13:06:29.570232+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:06:29.570232+0000","last_undegraded":"2023-11-28T13:06:29.570232+0000","last_fullsized":"2023-11-28T13:06:29.570232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.382636+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:06:28.382636+0000","last_peered":"2023-11-28T13:06:28.382636+0000","last_clean":"2023-11-28T13:06:28.382636+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:06:28.382636+0000","last_undegraded":"2023-11-28T13:06:28.382636+0000","last_fullsized":"2023-11-28T13:06:28.382636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:29.569990+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:06:29.569990+0000","last_peered":"2023-11-28T13:06:29.569990+0000","last_clean":"2023-11-28T13:06:29.569990+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:06:29.569990+0000","last_undegraded":"2023-11-28T13:06:29.569990+0000","last_fullsized":"2023-11-28T13:06:29.569990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.382665+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:06:28.382665+0000","last_peered":"2023-11-28T13:06:28.382665+0000","last_clean":"2023-11-28T13:06:28.382665+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:06:28.382665+0000","last_undegraded":"2023-11-28T13:06:28.382665+0000","last_fullsized":"2023-11-28T13:06:28.382665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:08.372261+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:06:08.372261+0000","last_peered":"2023-11-28T13:06:08.372261+0000","last_clean":"2023-11-28T13:06:08.372261+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:06:08.372261+0000","last_undegraded":"2023-11-28T13:06:08.372261+0000","last_fullsized":"2023-11-28T13:06:08.372261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:13.372839+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:06:13.372839+0000","last_peered":"2023-11-28T13:06:13.372839+0000","last_clean":"2023-11-28T13:06:13.372839+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:06:13.372839+0000","last_undegraded":"2023-11-28T13:06:13.372839+0000","last_fullsized":"2023-11-28T13:06:13.372839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":50,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:28.374619+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:06:28.374619+0000","last_peered":"2023-11-28T13:06:28.374619+0000","last_clean":"2023-11-28T13:06:28.374619+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:06:28.374619+0000","last_undegraded":"2023-11-28T13:06:28.374619+0000","last_fullsized":"2023-11-28T13:06:28.374619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:44.571926+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:06:44.571926+0000","last_peered":"2023-11-28T13:06:44.571926+0000","last_clean":"2023-11-28T13:06:44.571926+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:06:44.571926+0000","last_undegraded":"2023-11-28T13:06:44.571926+0000","last_fullsized":"2023-11-28T13:06:44.571926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:06:43.376175+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:06:43.376175+0000","last_peered":"2023-11-28T13:06:43.376175+0000","last_clean":"2023-11-28T13:06:43.376175+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:06:43.376175+0000","last_undegraded":"2023-11-28T13:06:43.376175+0000","last_fullsized":"2023-11-28T13:06:43.376175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":802816,"data_stored":354605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.372,"15min":0.371},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":0.497,"5min":0.57699999999999996,"15min":1.2669999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.57399999999999995},"min":{"1min":0.222,"5min":0.222,"15min":0.216},"max":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":8.218},"last":0.22}]},{"osd":1,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.26300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.623,"5min":0.63600000000000001,"15min":1.8280000000000001},"last":0.25},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.41199999999999998},"min":{"1min":0.29699999999999999,"5min":0.218,"15min":0.214},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":1.355},"last":0.23699999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.76400000000000001},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":13.384},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.26500000000000001},"max":{"1min":0.60999999999999999,"5min":0.65200000000000002,"15min":5.226},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.35999999999999999,"15min":0.35699999999999998},"max":{"1min":0.81799999999999995,"5min":0.94499999999999995,"15min":8.1890000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.73399999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78900000000000003,"5min":0.93999999999999995,"15min":5.3609999999999998},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":1.3029999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.80400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.878,"5min":0.96899999999999997,"15min":8.1579999999999995},"last":0.53300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":5.3209999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.3320000000000001},"last":0.50800000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.83099999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":8.1029999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.76000000000000001},"min":{"1min":0.50800000000000001,"5min":0.40300000000000002,"15min":0.38900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":5.3849999999999998},"last":0.52000000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9544,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362296,"statfs":{"total":96636764160,"available":96626991104,"internally_reserved":0,"allocated":786432,"data_stored":348000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35199999999999998,"15min":0.33000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":0.51700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.187},"max":{"1min":0.48599999999999999,"5min":0.61599999999999999,"15min":1.0169999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.224,"15min":0.20499999999999999},"max":{"1min":0.52600000000000002,"5min":0.62,"15min":0.95999999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.221},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":1.1459999999999999},"last":0.52300000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.39200000000000002,"15min":0.36599999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.63400000000000001,"15min":1.1659999999999999},"last":0.375},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.57599999999999996,"5min":0.70299999999999996,"15min":1.1919999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.641},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.91100000000000003,"15min":1.056},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.89800000000000002,"15min":1.6120000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.34499999999999997},"max":{"1min":0.72599999999999998,"5min":0.82899999999999996,"15min":1.774},"last":0.55700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.626,"15min":0.58199999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84599999999999997,"15min":1.242},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.92900000000000005,"15min":1.6539999999999999},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.876,"15min":1.6919999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.89300000000000002,"15min":1.2130000000000001},"last":0.65900000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":806912,"data_stored":359788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58099999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35999999999999999},"max":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58799999999999997,"15min":0.56299999999999994},"min":{"1min":0.47199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.61199999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.44800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.753,"15min":1.2270000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.52300000000000002},"min":{"1min":0.45600000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72699999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.46500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73199999999999998,"5min":0.86899999999999999,"15min":1.2110000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.45600000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53300000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.76900000000000002,"15min":0.81200000000000006},"last":0.376},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51000000000000001,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.52900000000000003},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.247},"max":{"1min":0.69099999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":0.83599999999999997,"15min":0.88700000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14856,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356984,"statfs":{"total":96636764160,"available":96621551616,"internally_reserved":0,"allocated":1376256,"data_stored":937599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.33800000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.214},"max":{"1min":0.52800000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.74199999999999999},"min":{"1min":0.34000000000000002,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":0.56200000000000006,"5min":0.61599999999999999,"15min":14.957000000000001},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.747},"min":{"1min":0.36499999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53200000000000003,"5min":0.60499999999999998,"15min":15.032999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.754},"min":{"1min":0.40699999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51900000000000002,"5min":0.59499999999999997,"15min":14.997999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41099999999999998,"15min":0.84699999999999998},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.21199999999999999},"max":{"1min":0.49399999999999999,"5min":0.56999999999999995,"15min":19.045999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.41099999999999998},"min":{"1min":0.40300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60699999999999998,"15min":0.67200000000000004},"min":{"1min":0.53700000000000003,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.1799999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":1.073},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.35299999999999998},"max":{"1min":0.754,"5min":0.89700000000000002,"15min":18.995000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61299999999999999,"15min":1.0600000000000001},"min":{"1min":0.55200000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.93700000000000006,"15min":19.02},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.628,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":4.2119999999999997},"last":0.68600000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63600000000000001,"15min":0.70199999999999996},"min":{"1min":0.57399999999999995,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":4.2359999999999998},"last":0.753},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63500000000000001,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.92500000000000004,"15min":4.1150000000000002},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.70199999999999996},"min":{"1min":0.57999999999999996,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.876,"15min":4.1539999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64500000000000002,"15min":1.085},"min":{"1min":0.63300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":18.957999999999998},"last":0.77800000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10972,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360868,"statfs":{"total":96636764160,"available":96625528832,"internally_reserved":0,"allocated":1396736,"data_stored":939128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.78400000000000003,"15min":1.1699999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.81599999999999995,"15min":1.415},"last":0.624}]},{"osd":1,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.80600000000000005,"15min":1.867},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57299999999999995,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":0.70399999999999996,"5min":0.79700000000000004,"15min":4.7830000000000004},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.84499999999999997,"15min":1.9550000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34000000000000002},"max":{"1min":0.68400000000000005,"5min":0.81499999999999995,"15min":1.9510000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.88600000000000001,"15min":1.6859999999999999},"last":0.503},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73799999999999999,"5min":0.89900000000000002,"15min":4.8440000000000003},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.33300000000000002,"15min":0.28499999999999998},"max":{"1min":0.69999999999999996,"5min":0.873,"15min":2.6030000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":1.528},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.311},"max":{"1min":0.64700000000000002,"5min":0.94699999999999995,"15min":1.25},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71599999999999997,"5min":0.93600000000000005,"15min":2.5329999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":0.65700000000000003,"5min":0.96599999999999997,"15min":1.972},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.98899999999999999,"15min":2.508},"last":0.64500000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9676,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362164,"statfs":{"total":96636764160,"available":96626855936,"internally_reserved":0,"allocated":790528,"data_stored":348022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68999999999999995,"5min":0.82799999999999996,"15min":1.419},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.68100000000000005,"5min":0.78500000000000003,"15min":0.80000000000000004},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.343,"15min":1.343},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70599999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.40799999999999997,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":3.6160000000000001},"last":0.56899999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.81100000000000005,"15min":1.4490000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.74399999999999999,"5min":1.228,"15min":1.228},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46100000000000002,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.624},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":0.85299999999999998,"15min":0.89600000000000002},"last":0.67200000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542236,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9484,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362356,"statfs":{"total":96636764160,"available":96627052544,"internally_reserved":0,"allocated":790528,"data_stored":348034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47799999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.75,"15min":0.75},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":1.0249999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.54800000000000004},"min":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.38},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69499999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.49099999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.76500000000000001,"5min":0.77700000000000002,"15min":0.79500000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78000000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.33100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.83399999999999996},"last":0.80500000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.40000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73199999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9708,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362132,"statfs":{"total":96636764160,"available":96626823168,"internally_reserved":0,"allocated":823296,"data_stored":360536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.46300000000000002,"5min":0.60999999999999999,"15min":0.97099999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39300000000000002,"15min":0.35899999999999999},"min":{"1min":0.25800000000000001,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":0.55400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.25800000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.375,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52900000000000003,"5min":0.57499999999999996,"15min":0.72899999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.41099999999999998,"15min":0.378},"min":{"1min":0.27500000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54500000000000004,"5min":0.60099999999999998,"15min":0.64300000000000002},"last":0.29699999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26500000000000001},"max":{"1min":0.67000000000000004,"5min":0.69199999999999995,"15min":0.752},"last":0.378},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42699999999999999,"15min":0.40400000000000003},"min":{"1min":0.29399999999999998,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.875},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.58799999999999997},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.36299999999999999},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.499},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":1.044},"last":0.59099999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.47699999999999998,"5min":0.433,"15min":0.28999999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.36799999999999999},"max":{"1min":0.82199999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.46500000000000002,"15min":0.31900000000000001},"max":{"1min":0.83299999999999996,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.38900000000000001},"max":{"1min":0.90000000000000002,"5min":0.93899999999999995,"15min":1.089},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66700000000000004,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":1.202},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.63600000000000001},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":0.95799999999999996,"15min":1.02},"last":0.52700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:27.805 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.926+0000 7f46d79a6700 1 Processor -- start 2023-11-28T13:07:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.926+0000 7f46d79a6700 1 -- start start 2023-11-28T13:07:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d79a6700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d79a6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d79a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d79a6700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d0071950 con 0x7f46d0071040 2023-11-28T13:07:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d79a6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d015d060 con 0x7f46d006dee0 2023-11-28T13:07:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d79a6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d015d340 con 0x7f46d006cdc0 2023-11-28T13:07:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43620/0 (socket says 172.21.15.153:43620) 2023-11-28T13:07:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 -- 172.21.15.153:0/3491130439 learned_addr learned my addr 172.21.15.153:0/3491130439 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d4f41700 1 --2- 172.21.15.153:0/3491130439 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5f43700 1 --2- 172.21.15.153:0/3491130439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 -- 172.21.15.153:0/3491130439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 msgr2=0x7f46d006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:27.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 --2- 172.21.15.153:0/3491130439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 -- 172.21.15.153:0/3491130439 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 msgr2=0x7f46d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 --2- 172.21.15.153:0/3491130439 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 -- 172.21.15.153:0/3491130439 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d015d620 con 0x7f46d0071040 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46d5742700 1 --2- 172.21.15.153:0/3491130439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d0071410 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f46d006fb00 tx=0x7f46cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38b7eb8e82e63e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.927+0000 7f46c7fff700 1 -- 172.21.15.153:0/3491130439 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc011830 con 0x7f46d0071040 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46c7fff700 1 -- 172.21.15.153:0/3491130439 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc011990 con 0x7f46d0071040 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46c7fff700 1 -- 172.21.15.153:0/3491130439 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc011d20 con 0x7f46d0071040 2023-11-28T13:07:27.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 -- 172.21.15.153:0/3491130439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 msgr2=0x7f46d0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 --2- 172.21.15.153:0/3491130439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d0071410 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f46d006fb00 tx=0x7f46cc009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 -- 172.21.15.153:0/3491130439 shutdown_connections 2023-11-28T13:07:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 --2- 172.21.15.153:0/3491130439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d006cdc0 0x7f46d006d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 --2- 172.21.15.153:0/3491130439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d0071040 0x7f46d0071410 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 --2- 172.21.15.153:0/3491130439 >> v2:172.21.15.177:3300/0 conn(0x7f46d006dee0 0x7f46d006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 -- 172.21.15.153:0/3491130439 >> 172.21.15.153:0/3491130439 conn(0x7f46d00d0000 msgr2=0x7f46d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:27.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 -- 172.21.15.153:0/3491130439 shutdown_connections 2023-11-28T13:07:27.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.928+0000 7f46d79a6700 1 -- 172.21.15.153:0/3491130439 wait complete. 2023-11-28T13:07:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.929+0000 7f46d79a6700 1 Processor -- start 2023-11-28T13:07:27.936 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.929+0000 7f46d79a6700 1 -- start start 2023-11-28T13:07:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d79a6700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d006cdc0 0x7f46d00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d79a6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006dee0 0x7f46d0153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d79a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d0071040 0x7f46d01462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d79a6700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c5d10 con 0x7f46d006cdc0 2023-11-28T13:07:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d79a6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c5090 con 0x7f46d006dee0 2023-11-28T13:07:27.937 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d79a6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d00c5370 con 0x7f46d0071040 2023-11-28T13:07:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5742700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d006cdc0 0x7f46d00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5742700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d006cdc0 0x7f46d00b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43626/0 (socket says 172.21.15.153:43626) 2023-11-28T13:07:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5742700 1 -- 172.21.15.153:0/4024006938 learned_addr learned my addr 172.21.15.153:0/4024006938 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5f43700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d0071040 0x7f46d01462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d4f41700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006dee0 0x7f46d0153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5742700 1 -- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d0071040 msgr2=0x7f46d01462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.938 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5742700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d0071040 0x7f46d01462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5742700 1 -- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006dee0 msgr2=0x7f46d0153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.930+0000 7f46d5742700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006dee0 0x7f46d0153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.931+0000 7f46d5742700 1 -- 172.21.15.153:0/4024006938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc009550 con 0x7f46d006cdc0 2023-11-28T13:07:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.931+0000 7f46d5742700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d006cdc0 0x7f46d00b2e30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f46cc014040 tx=0x7f46cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.939 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.931+0000 7f46c5ffb700 1 -- 172.21.15.153:0/4024006938 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc004510 con 0x7f46d006cdc0 2023-11-28T13:07:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.931+0000 7f46c5ffb700 1 -- 172.21.15.153:0/4024006938 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc01e070 con 0x7f46d006cdc0 2023-11-28T13:07:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.931+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d006e870 con 0x7f46d006cdc0 2023-11-28T13:07:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.931+0000 7f46c5ffb700 1 -- 172.21.15.153:0/4024006938 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0038a0 con 0x7f46d006cdc0 2023-11-28T13:07:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.931+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d015d900 con 0x7f46d006cdc0 2023-11-28T13:07:27.940 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.932+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d015ccd0 con 0x7f46d006cdc0 2023-11-28T13:07:27.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.932+0000 7f46c5ffb700 1 -- 172.21.15.153:0/4024006938 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46cc00f050 con 0x7f46d006cdc0 2023-11-28T13:07:27.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.933+0000 7f46c5ffb700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:27.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.933+0000 7f46c5ffb700 1 -- 172.21.15.153:0/4024006938 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f46cc020070 con 0x7f46d006cdc0 2023-11-28T13:07:27.941 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.933+0000 7f46d4f41700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:27.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.933+0000 7f46d4f41700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009ad0 tx=0x7f46c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:27.942 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:27.937+0000 7f46c5ffb700 1 -- 172.21.15.153:0/4024006938 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46cc053700 con 0x7f46d006cdc0 2023-11-28T13:07:28.120 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.120+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46d015fe70 con 0x7f46d006cdc0 2023-11-28T13:07:28.121 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.120+0000 7f46c5ffb700 1 -- 172.21.15.153:0/4024006938 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+14447 (secure 0 0 0) 0x7f46cc02a020 con 0x7f46d006cdc0 2023-11-28T13:07:28.121 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:28.122 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":68,"fsid":"47bfaa00-a151-4537-aa4c-32b0866eb248","created":"2023-11-28T12:59:52.561869+0000","modified":"2023-11-28T13:07:24.706505+0000","last_up_change":"2023-11-28T13:00:03.842722+0000","last_in_change":"2023-11-28T12:59:55.084798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T13:00:04.175316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T13:00:10.995932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63fec646-657d-4957-bcc2-76a5db288946","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":103561},{"type":"v1","addr":"172.21.15.153:6801","nonce":103561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":103561},{"type":"v1","addr":"172.21.15.153:6803","nonce":103561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":103561},{"type":"v1","addr":"172.21.15.153:6807","nonce":103561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":103561},{"type":"v1","addr":"172.21.15.153:6805","nonce":103561}]},"public_addr":"172.21.15.153:6801/103561","cluster_addr":"172.21.15.153:6803/103561","heartbeat_back_addr":"172.21.15.153:6807/103561","heartbeat_front_addr":"172.21.15.153:6805/103561","state":["exists","up"]},{"osd":1,"uuid":"c9553e4f-e498-4be9-bda7-52af39c42b26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":103564},{"type":"v1","addr":"172.21.15.153:6817","nonce":103564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":103564},{"type":"v1","addr":"172.21.15.153:6819","nonce":103564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6823","nonce":103564},{"type":"v1","addr":"172.21.15.153:6825","nonce":103564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":103564},{"type":"v1","addr":"172.21.15.153:6821","nonce":103564}]},"public_addr":"172.21.15.153:6817/103564","cluster_addr":"172.21.15.153:6819/103564","heartbeat_back_addr":"172.21.15.153:6825/103564","heartbeat_front_addr":"172.21.15.153:6821/103564","state":["exists","up"]},{"osd":2,"uuid":"66975359-a324-4e64-a069-173dbe9e0aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":103562},{"type":"v1","addr":"172.21.15.153:6809","nonce":103562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":103562},{"type":"v1","addr":"172.21.15.153:6811","nonce":103562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":103562},{"type":"v1","addr":"172.21.15.153:6815","nonce":103562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":103562},{"type":"v1","addr":"172.21.15.153:6813","nonce":103562}]},"public_addr":"172.21.15.153:6809/103562","cluster_addr":"172.21.15.153:6811/103562","heartbeat_back_addr":"172.21.15.153:6815/103562","heartbeat_front_addr":"172.21.15.153:6813/103562","state":["exists","up"]},{"osd":3,"uuid":"bd9de07f-3315-48c1-9d94-69f8f7a164fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":103565},{"type":"v1","addr":"172.21.15.153:6824","nonce":103565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":103565},{"type":"v1","addr":"172.21.15.153:6827","nonce":103565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":103565},{"type":"v1","addr":"172.21.15.153:6831","nonce":103565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":103565},{"type":"v1","addr":"172.21.15.153:6829","nonce":103565}]},"public_addr":"172.21.15.153:6824/103565","cluster_addr":"172.21.15.153:6827/103565","heartbeat_back_addr":"172.21.15.153:6831/103565","heartbeat_front_addr":"172.21.15.153:6829/103565","state":["exists","up"]},{"osd":4,"uuid":"b595bb37-b598-404f-a1a7-80b1f1c28b11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6809","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6813","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103772},{"type":"v1","addr":"172.21.15.177:6823","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103772},{"type":"v1","addr":"172.21.15.177:6819","nonce":103772}]},"public_addr":"172.21.15.177:6811/103772","cluster_addr":"172.21.15.177:6815/103772","heartbeat_back_addr":"172.21.15.177:6823/103772","heartbeat_front_addr":"172.21.15.177:6819/103772","state":["exists","up"]},{"osd":5,"uuid":"7eaefb1e-4587-46f9-9e26-614f22833df4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103770},{"type":"v1","addr":"172.21.15.177:6801","nonce":103770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103770},{"type":"v1","addr":"172.21.15.177:6803","nonce":103770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103770},{"type":"v1","addr":"172.21.15.177:6807","nonce":103770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103770},{"type":"v1","addr":"172.21.15.177:6805","nonce":103770}]},"public_addr":"172.21.15.177:6801/103770","cluster_addr":"172.21.15.177:6803/103770","heartbeat_back_addr":"172.21.15.177:6807/103770","heartbeat_front_addr":"172.21.15.177:6805/103770","state":["exists","up"]},{"osd":6,"uuid":"891d9b75-4a1a-49b0-ac1f-393a9271171f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103787},{"type":"v1","addr":"172.21.15.177:6825","nonce":103787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":103787},{"type":"v1","addr":"172.21.15.177:6827","nonce":103787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":103787},{"type":"v1","addr":"172.21.15.177:6831","nonce":103787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103787},{"type":"v1","addr":"172.21.15.177:6829","nonce":103787}]},"public_addr":"172.21.15.177:6825/103787","cluster_addr":"172.21.15.177:6827/103787","heartbeat_back_addr":"172.21.15.177:6831/103787","heartbeat_front_addr":"172.21.15.177:6829/103787","state":["exists","up"]},{"osd":7,"uuid":"b34ca128-3e2e-4ff0-8d55-8372fd8df439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103771},{"type":"v1","addr":"172.21.15.177:6810","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103771},{"type":"v1","addr":"172.21.15.177:6814","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103771},{"type":"v1","addr":"172.21.15.177:6822","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103771},{"type":"v1","addr":"172.21.15.177:6818","nonce":103771}]},"public_addr":"172.21.15.177:6810/103771","cluster_addr":"172.21.15.177:6814/103771","heartbeat_back_addr":"172.21.15.177:6822/103771","heartbeat_front_addr":"172.21.15.177:6818/103771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.013900+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.320801+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.358778+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.307989+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.253381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.219225+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.315915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:00.319828+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:6837/3815439165":"2023-11-29T13:07:21.088933+0000","172.21.15.153:6835/3815439165":"2023-11-29T13:07:21.088933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:28.123 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.122+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 msgr2=0x7f46b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:28.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.122+0000 7f46d79a6700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009ad0 tx=0x7f46c0008040 comp rx=0 tx=0).stop 2023-11-28T13:07:28.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.122+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d006cdc0 msgr2=0x7f46d00b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:28.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.122+0000 7f46d79a6700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d006cdc0 0x7f46d00b2e30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f46cc014040 tx=0x7f46cc004750 comp rx=0 tx=0).stop 2023-11-28T13:07:28.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 shutdown_connections 2023-11-28T13:07:28.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f46b40682f0 0x7f46b406a7a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:28.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46d006cdc0 0x7f46d00b2e30 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:28.124 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46d0071040 0x7f46d01462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:28.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 --2- 172.21.15.153:0/4024006938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46d006dee0 0x7f46d0153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:28.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 >> 172.21.15.153:0/4024006938 conn(0x7f46d00d0000 msgr2=0x7f46d015f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:28.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 shutdown_connections 2023-11-28T13:07:28.125 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:28.123+0000 7f46d79a6700 1 -- 172.21.15.153:0/4024006938 wait complete. 2023-11-28T13:07:29.136 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T13:07:29.136 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:29.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.274+0000 7fd0d83f4700 1 Processor -- start 2023-11-28T13:07:29.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.274+0000 7fd0d83f4700 1 -- start start 2023-11-28T13:07:29.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d83f4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd0d01596b0 0x7fd0d0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d83f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 0x7fd0d00cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.277 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d83f4700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd0d00d02a0 0x7fd0d015d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d83f4700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d00d0720 con 0x7fd0d00d02a0 2023-11-28T13:07:29.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d83f4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d00cc7b0 con 0x7fd0d01596b0 2023-11-28T13:07:29.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d83f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00cd610 con 0x7fd0d00cc260 2023-11-28T13:07:29.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd0d00d02a0 0x7fd0d015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fd0d00d02a0 0x7fd0d015d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43638/0 (socket says 172.21.15.153:43638) 2023-11-28T13:07:29.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 -- 172.21.15.153:0/1406477638 learned_addr learned my addr 172.21.15.153:0/1406477638 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:29.278 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d598f700 1 --2- 172.21.15.153:0/1406477638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 0x7fd0d00cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6190700 1 --2- 172.21.15.153:0/1406477638 >> v2:172.21.15.177:3300/0 conn(0x7fd0d01596b0 0x7fd0d0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 -- 172.21.15.153:0/1406477638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 msgr2=0x7fd0d00cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 --2- 172.21.15.153:0/1406477638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 0x7fd0d00cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 -- 172.21.15.153:0/1406477638 >> v2:172.21.15.177:3300/0 conn(0x7fd0d01596b0 msgr2=0x7fd0d0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 --2- 172.21.15.153:0/1406477638 >> v2:172.21.15.177:3300/0 conn(0x7fd0d01596b0 0x7fd0d0071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 -- 172.21.15.153:0/1406477638 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0159a80 con 0x7fd0d00d02a0 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.275+0000 7fd0d6991700 1 --2- 172.21.15.153:0/1406477638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d00d02a0 0x7fd0d015d170 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00cfd30 tx=0x7fd0c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f9f8b85089bbfb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.279 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d518e700 1 -- 172.21.15.153:0/1406477638 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800e070 con 0x7fd0d00d02a0 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d518e700 1 -- 172.21.15.153:0/1406477638 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c800b9c0 con 0x7fd0d00d02a0 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d518e700 1 -- 172.21.15.153:0/1406477638 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800bd70 con 0x7fd0d00d02a0 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 -- 172.21.15.153:0/1406477638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d00d02a0 msgr2=0x7fd0d015d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 --2- 172.21.15.153:0/1406477638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d00d02a0 0x7fd0d015d170 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00cfd30 tx=0x7fd0c800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 -- 172.21.15.153:0/1406477638 shutdown_connections 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 --2- 172.21.15.153:0/1406477638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d00d02a0 0x7fd0d015d170 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 --2- 172.21.15.153:0/1406477638 >> v2:172.21.15.177:3300/0 conn(0x7fd0d01596b0 0x7fd0d0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.280 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 --2- 172.21.15.153:0/1406477638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 0x7fd0d00cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 -- 172.21.15.153:0/1406477638 >> 172.21.15.153:0/1406477638 conn(0x7fd0d006b380 msgr2=0x7fd0d006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:29.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 -- 172.21.15.153:0/1406477638 shutdown_connections 2023-11-28T13:07:29.281 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.276+0000 7fd0d83f4700 1 -- 172.21.15.153:0/1406477638 wait complete. 2023-11-28T13:07:29.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 Processor -- start 2023-11-28T13:07:29.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 -- start start 2023-11-28T13:07:29.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 0x7fd0d0155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d00d02a0 0x7fd0d0152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.284 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d01596b0 0x7fd0d00b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00c8ff0 con 0x7fd0d01596b0 2023-11-28T13:07:29.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00c4950 con 0x7fd0d00d02a0 2023-11-28T13:07:29.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d83f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00c4c30 con 0x7fd0d00cc260 2023-11-28T13:07:29.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d6991700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d01596b0 0x7fd0d00b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d6991700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d01596b0 0x7fd0d00b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43644/0 (socket says 172.21.15.153:43644) 2023-11-28T13:07:29.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.278+0000 7fd0d6991700 1 -- 172.21.15.153:0/521032799 learned_addr learned my addr 172.21.15.153:0/521032799 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:29.285 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d598f700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d00d02a0 0x7fd0d0152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d6190700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 0x7fd0d0155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d6991700 1 -- 172.21.15.153:0/521032799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 msgr2=0x7fd0d0155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d6991700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0d00cc260 0x7fd0d0155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d6991700 1 -- 172.21.15.153:0/521032799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d00d02a0 msgr2=0x7fd0d0152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d6991700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0d00d02a0 0x7fd0d0152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d6991700 1 -- 172.21.15.153:0/521032799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8009b30 con 0x7fd0d01596b0 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d6991700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d01596b0 0x7fd0d00b1db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800b370 tx=0x7fd0c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.286 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8004250 con 0x7fd0d01596b0 2023-11-28T13:07:29.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c80043f0 con 0x7fd0d01596b0 2023-11-28T13:07:29.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d00ccce0 con 0x7fd0d01596b0 2023-11-28T13:07:29.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80047a0 con 0x7fd0d01596b0 2023-11-28T13:07:29.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.279+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d00cca00 con 0x7fd0d01596b0 2023-11-28T13:07:29.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.280+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0d00ccfc0 con 0x7fd0d01596b0 2023-11-28T13:07:29.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd0c802c070 con 0x7fd0d01596b0 2023-11-28T13:07:29.287 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0c6ffd700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd0b40682f0 0x7fd0b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd0c8086d80 con 0x7fd0d01596b0 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0d6190700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd0b40682f0 0x7fd0b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0c6ffd700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7fd0b406e760 0x7fd0b4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 --> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0b4071240 con 0x7fd0b406e760 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0c8089430 con 0x7fd0d01596b0 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0d598f700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7fd0b406e760 0x7fd0b4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.281+0000 7fd0d6190700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd0b40682f0 0x7fd0b406a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000ab60 tx=0x7fd0c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.288 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.282+0000 7fd0d598f700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7fd0b406e760 0x7fd0b4070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.289 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.282+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== osd.0 v2:172.21.15.153:6800/103561 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd0b4071240 con 0x7fd0b406e760 2023-11-28T13:07:29.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.296+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 --> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd0d0068480 con 0x7fd0b406e760 2023-11-28T13:07:29.297 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0c6ffd700 1 -- 172.21.15.153:0/521032799 <== osd.0 v2:172.21.15.153:6800/103561 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd0d0068480 con 0x7fd0b406e760 2023-11-28T13:07:29.298 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:29.298 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:29.298 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:29.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7fd0b406e760 msgr2=0x7fd0b4070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6800/103561,v1:172.21.15.153:6801/103561] conn(0x7fd0b406e760 0x7fd0b4070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd0b40682f0 msgr2=0x7fd0b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fd0b40682f0 0x7fd0b406a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000ab60 tx=0x7fd0c0008040 comp rx=0 tx=0).stop 2023-11-28T13:07:29.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d01596b0 msgr2=0x7fd0d00b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.299 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 --2- 172.21.15.153:0/521032799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd0d01596b0 0x7fd0d00b1db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800b370 tx=0x7fd0c8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d6991700 1 -- 172.21.15.153:0/521032799 reap_dead start 2023-11-28T13:07:29.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 shutdown_connections 2023-11-28T13:07:29.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 >> 172.21.15.153:0/521032799 conn(0x7fd0d006b380 msgr2=0x7fd0d00c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:29.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 shutdown_connections 2023-11-28T13:07:29.300 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.297+0000 7fd0d83f4700 1 -- 172.21.15.153:0/521032799 wait complete. 2023-11-28T13:07:29.309 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T13:07:29.430 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.430+0000 7f0fe280a700 1 Processor -- start 2023-11-28T13:07:29.431 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.430+0000 7f0fe280a700 1 -- start start 2023-11-28T13:07:29.432 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.430+0000 7f0fe280a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0fdc15cef0 0x7f0fdc0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.430+0000 7f0fe280a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc06ba90 0x7f0fdc0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.430+0000 7f0fe280a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fdc0cc5f0 con 0x7f0fdc0cbdf0 2023-11-28T13:07:29.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.430+0000 7f0fe280a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fdc15d470 con 0x7f0fdc15cef0 2023-11-28T13:07:29.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.430+0000 7f0fe280a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc15d750 con 0x7f0fdc06ba90 2023-11-28T13:07:29.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fdbfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0fdc15cef0 0x7f0fdc0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fe0da7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc06ba90 0x7f0fdc0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.433 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fdb7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0fdc0cbdf0 0x7f0fdc06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.153:38430/0 (socket says 172.21.15.153:38430) 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fdb7fe700 1 -- 172.21.15.153:0/3626630282 learned_addr learned my addr 172.21.15.153:0/3626630282 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fe0da7700 1 -- 172.21.15.153:0/3626630282 >> v2:172.21.15.177:3300/0 conn(0x7f0fdc15cef0 msgr2=0x7f0fdc0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fe0da7700 1 --2- 172.21.15.153:0/3626630282 >> v2:172.21.15.177:3300/0 conn(0x7f0fdc15cef0 0x7f0fdc0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fe0da7700 1 -- 172.21.15.153:0/3626630282 >> v1:172.21.15.177:6789/0 conn(0x7f0fdc0cbdf0 legacy=0x7f0fdc06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fe0da7700 1 -- 172.21.15.153:0/3626630282 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc15da30 con 0x7f0fdc06ba90 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fe0da7700 1 --2- 172.21.15.153:0/3626630282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc06ba90 0x7f0fdc0ca650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009dc0 tx=0x7f0fd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cfdc5cd3d9a603 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fdaffd700 1 -- 172.21.15.153:0/3626630282 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd400e070 con 0x7f0fdc06ba90 2023-11-28T13:07:29.434 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fdaffd700 1 -- 172.21.15.153:0/3626630282 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd400bae0 con 0x7f0fdc06ba90 2023-11-28T13:07:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.431+0000 7f0fdaffd700 1 -- 172.21.15.153:0/3626630282 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd400be90 con 0x7f0fdc06ba90 2023-11-28T13:07:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 -- 172.21.15.153:0/3626630282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc06ba90 msgr2=0x7f0fdc0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 --2- 172.21.15.153:0/3626630282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc06ba90 0x7f0fdc0ca650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009dc0 tx=0x7f0fd400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 -- 172.21.15.153:0/3626630282 shutdown_connections 2023-11-28T13:07:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 --2- 172.21.15.153:0/3626630282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc06ba90 0x7f0fdc0ca650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 --2- 172.21.15.153:0/3626630282 >> v2:172.21.15.177:3300/0 conn(0x7f0fdc15cef0 0x7f0fdc0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 -- 172.21.15.153:0/3626630282 >> 172.21.15.153:0/3626630282 conn(0x7f0fdc0d0000 msgr2=0x7f0fdc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:29.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 -- 172.21.15.153:0/3626630282 shutdown_connections 2023-11-28T13:07:29.436 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.432+0000 7f0fe280a700 1 -- 172.21.15.153:0/3626630282 wait complete. 2023-11-28T13:07:29.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.433+0000 7f0fe280a700 1 Processor -- start 2023-11-28T13:07:29.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.433+0000 7f0fe280a700 1 -- start start 2023-11-28T13:07:29.438 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe280a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fdc06ba90 0x7f0fdc1f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe280a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc0cbdf0 0x7f0fdc06a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe280a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc15cef0 0x7f0fdc06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe280a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc1e36e0 con 0x7f0fdc15cef0 2023-11-28T13:07:29.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe280a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0c9ef0 con 0x7f0fdc06ba90 2023-11-28T13:07:29.439 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe280a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0c9930 con 0x7f0fdc0cbdf0 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe0da7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc15cef0 0x7f0fdc06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe0da7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc15cef0 0x7f0fdc06a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43660/0 (socket says 172.21.15.153:43660) 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fe0da7700 1 -- 172.21.15.153:0/1482736371 learned_addr learned my addr 172.21.15.153:0/1482736371 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fdb7fe700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc0cbdf0 0x7f0fdc06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.434+0000 7f0fdbfff700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fdc06ba90 0x7f0fdc1f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe0da7700 1 -- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc0cbdf0 msgr2=0x7f0fdc06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe0da7700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc0cbdf0 0x7f0fdc06a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.440 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe0da7700 1 -- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fdc06ba90 msgr2=0x7f0fdc1f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe0da7700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fdc06ba90 0x7f0fdc1f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe0da7700 1 -- 172.21.15.153:0/1482736371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4009a70 con 0x7f0fdc15cef0 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe0da7700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc15cef0 0x7f0fdc06a7b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4014040 tx=0x7f0fd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fd8ff9700 1 -- 172.21.15.153:0/1482736371 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4004550 con 0x7f0fdc15cef0 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fd8ff9700 1 -- 172.21.15.153:0/1482736371 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd4020070 con 0x7f0fdc15cef0 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc15dff0 con 0x7f0fdc15cef0 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fd8ff9700 1 -- 172.21.15.153:0/1482736371 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4004910 con 0x7f0fdc15cef0 2023-11-28T13:07:29.441 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.435+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc15dd10 con 0x7f0fdc15cef0 2023-11-28T13:07:29.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.436+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc0cab90 con 0x7f0fdc15cef0 2023-11-28T13:07:29.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.436+0000 7f0fd8ff9700 1 -- 172.21.15.153:0/1482736371 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0fd4012430 con 0x7f0fdc15cef0 2023-11-28T13:07:29.442 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.436+0000 7f0fd8ff9700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fc8068320 0x7f0fc806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.437+0000 7f0fd8ff9700 1 -- 172.21.15.153:0/1482736371 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0fd4022070 con 0x7f0fdc15cef0 2023-11-28T13:07:29.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.437+0000 7f0fdbfff700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fc8068320 0x7f0fc806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.437+0000 7f0fdbfff700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fc8068320 0x7f0fc806a7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009b40 tx=0x7f0fcc009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.444 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.441+0000 7f0fd8ff9700 1 -- 172.21.15.153:0/1482736371 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0fd4054aa0 con 0x7f0fdc15cef0 2023-11-28T13:07:29.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.624+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fdc1e3820 con 0x7f0fc8068320 2023-11-28T13:07:29.625 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.624+0000 7f0fd8ff9700 1 -- 172.21.15.153:0/1482736371 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0fdc1e3820 con 0x7f0fc8068320 2023-11-28T13:07:29.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fc8068320 msgr2=0x7f0fc806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fc8068320 0x7f0fc806a7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009b40 tx=0x7f0fcc009210 comp rx=0 tx=0).stop 2023-11-28T13:07:29.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc15cef0 msgr2=0x7f0fdc06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.627 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc15cef0 0x7f0fdc06a7b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4014040 tx=0x7f0fd4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 shutdown_connections 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0fc8068320 0x7f0fc806a7d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fdc15cef0 0x7f0fdc06a7b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fdc06ba90 0x7f0fdc1f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 --2- 172.21.15.153:0/1482736371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fdc0cbdf0 0x7f0fdc06a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 >> 172.21.15.153:0/1482736371 conn(0x7f0fdc0d0000 msgr2=0x7f0fdc071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 shutdown_connections 2023-11-28T13:07:29.628 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.626+0000 7f0fe280a700 1 -- 172.21.15.153:0/1482736371 wait complete. 2023-11-28T13:07:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T13:07:29.639 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T13:07:29.639 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:29.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.763+0000 7f8e70312700 1 Processor -- start 2023-11-28T13:07:29.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.763+0000 7f8e70312700 1 -- start start 2023-11-28T13:07:29.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e70312700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e680af060 0x7f8e68070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e70312700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8e680af430 0x7f8e6806d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e70312700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e680713c0 con 0x7f8e6806a270 2023-11-28T13:07:29.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e70312700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6806dee0 con 0x7f8e680af060 2023-11-28T13:07:29.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e70312700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680716a0 con 0x7f8e680af430 2023-11-28T13:07:29.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e6e0ae700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e680af060 0x7f8e68070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e6e0ae700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8e680af060 0x7f8e68070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43664/0 (socket says 172.21.15.153:43664) 2023-11-28T13:07:29.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.764+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/3270527927 learned_addr learned my addr 172.21.15.153:0/3270527927 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:29.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/3270527927 >> v2:172.21.15.177:3300/0 conn(0x7f8e680af430 msgr2=0x7f8e6806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:29.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6e0ae700 1 --2- 172.21.15.153:0/3270527927 >> v2:172.21.15.177:3300/0 conn(0x7f8e680af430 0x7f8e6806d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/3270527927 >> v1:172.21.15.177:6789/0 conn(0x7f8e6806a270 legacy=0x7f8e6806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:29.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/3270527927 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e68071980 con 0x7f8e680af060 2023-11-28T13:07:29.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6e0ae700 1 --2- 172.21.15.153:0/3270527927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680af060 0x7f8e68070d50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8e680af800 tx=0x7f8e58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4ac42dae025c2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.768 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6d0ac700 1 -- 172.21.15.153:0/3270527927 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58004230 con 0x7f8e680af060 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6d0ac700 1 -- 172.21.15.153:0/3270527927 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e58004390 con 0x7f8e680af060 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.765+0000 7f8e6d0ac700 1 -- 172.21.15.153:0/3270527927 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e580036a0 con 0x7f8e680af060 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 -- 172.21.15.153:0/3270527927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680af060 msgr2=0x7f8e68070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 --2- 172.21.15.153:0/3270527927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680af060 0x7f8e68070d50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8e680af800 tx=0x7f8e58009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 -- 172.21.15.153:0/3270527927 shutdown_connections 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 --2- 172.21.15.153:0/3270527927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e680af060 0x7f8e68070d50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 --2- 172.21.15.153:0/3270527927 >> v2:172.21.15.177:3300/0 conn(0x7f8e680af430 0x7f8e6806d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.769 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 -- 172.21.15.153:0/3270527927 >> 172.21.15.153:0/3270527927 conn(0x7f8e6806b380 msgr2=0x7f8e6806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:29.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 -- 172.21.15.153:0/3270527927 shutdown_connections 2023-11-28T13:07:29.770 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.766+0000 7f8e70312700 1 -- 172.21.15.153:0/3270527927 wait complete. 2023-11-28T13:07:29.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.767+0000 7f8e70312700 1 Processor -- start 2023-11-28T13:07:29.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.767+0000 7f8e70312700 1 -- start start 2023-11-28T13:07:29.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e6806a270 0x7f8e68157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.773 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e680af060 0x7f8e680c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e680af430 0x7f8e680b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6815f470 con 0x7f8e6806a270 2023-11-28T13:07:29.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680af9f0 con 0x7f8e680af060 2023-11-28T13:07:29.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6815ebd0 con 0x7f8e680af430 2023-11-28T13:07:29.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e6806a270 0x7f8e68157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e6806a270 0x7f8e68157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43670/0 (socket says 172.21.15.153:43670) 2023-11-28T13:07:29.774 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/1491662070 learned_addr learned my addr 172.21.15.153:0/1491662070 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:29.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6d8ad700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e680af060 0x7f8e680c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e8af700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e680af430 0x7f8e680b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/1491662070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e680af430 msgr2=0x7f8e680b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e680af430 0x7f8e680b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/1491662070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e680af060 msgr2=0x7f8e680c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.775 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e680af060 0x7f8e680c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 -- 172.21.15.153:0/1491662070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58009550 con 0x7f8e6806a270 2023-11-28T13:07:29.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6d8ad700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e680af060 0x7f8e680c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:29.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e6e0ae700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e6806a270 0x7f8e68157500 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8e6806c720 tx=0x7f8e58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58003dc0 con 0x7f8e6806a270 2023-11-28T13:07:29.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5801f070 con 0x7f8e6806a270 2023-11-28T13:07:29.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e68071f40 con 0x7f8e6806a270 2023-11-28T13:07:29.776 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58011870 con 0x7f8e6806a270 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.768+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e68071c60 con 0x7f8e6806a270 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.769+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e680ca660 con 0x7f8e6806a270 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8e58017020 con 0x7f8e6806a270 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e5effd700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e4c0682f0 0x7f8e4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8e58087230 con 0x7f8e6806a270 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e6d8ad700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e4c0682f0 0x7f8e4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e5effd700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f8e4c06e760 0x7f8e4c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.777 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 --> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e4c071240 con 0x7f8e4c06e760 2023-11-28T13:07:29.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e580898e0 con 0x7f8e6806a270 2023-11-28T13:07:29.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e6e8af700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f8e4c06e760 0x7f8e4c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.770+0000 7f8e6d8ad700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e4c0682f0 0x7f8e4c06a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e680c8fc0 tx=0x7f8e64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.771+0000 7f8e6e8af700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f8e4c06e760 0x7f8e4c070b70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.778 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.771+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== osd.1 v2:172.21.15.153:6816/103564 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8e4c071240 con 0x7f8e4c06e760 2023-11-28T13:07:29.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.785+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 --> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8e680af830 con 0x7f8e4c06e760 2023-11-28T13:07:29.786 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.785+0000 7f8e5effd700 1 -- 172.21.15.153:0/1491662070 <== osd.1 v2:172.21.15.153:6816/103564 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8e680af830 con 0x7f8e4c06e760 2023-11-28T13:07:29.786 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:29.786 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:29.787 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:29.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f8e4c06e760 msgr2=0x7f8e4c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6816/103564,v1:172.21.15.153:6817/103564] conn(0x7f8e4c06e760 0x7f8e4c070b70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e4c0682f0 msgr2=0x7f8e4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8e4c0682f0 0x7f8e4c06a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e680c8fc0 tx=0x7f8e64008040 comp rx=0 tx=0).stop 2023-11-28T13:07:29.788 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e6806a270 msgr2=0x7f8e68157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 --2- 172.21.15.153:0/1491662070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8e6806a270 0x7f8e68157500 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8e6806c720 tx=0x7f8e58004750 comp rx=0 tx=0).stop 2023-11-28T13:07:29.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e6e8af700 1 -- 172.21.15.153:0/1491662070 reap_dead start 2023-11-28T13:07:29.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 shutdown_connections 2023-11-28T13:07:29.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 >> 172.21.15.153:0/1491662070 conn(0x7f8e6806b380 msgr2=0x7f8e680cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:29.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 shutdown_connections 2023-11-28T13:07:29.789 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.786+0000 7f8e70312700 1 -- 172.21.15.153:0/1491662070 wait complete. 2023-11-28T13:07:29.798 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T13:07:29.919 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.919+0000 7ffa8eb1b700 1 Processor -- start 2023-11-28T13:07:29.920 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.919+0000 7ffa8eb1b700 1 -- start start 2023-11-28T13:07:29.922 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.919+0000 7ffa8eb1b700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffa8806fb30 0x7ffa88158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.919+0000 7ffa8eb1b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffa8815d1f0 0x7ffa880b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.919+0000 7ffa8eb1b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa880ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.919+0000 7ffa8eb1b700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8806ff00 con 0x7ffa8806fb30 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8eb1b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8806bf10 con 0x7ffa8815d1f0 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8eb1b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880ce6f0 con 0x7ffa880cdda0 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffa8806fb30 0x7ffa88158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7ffa8806fb30 0x7ffa88158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43680/0 (socket says 172.21.15.153:43680) 2023-11-28T13:07:29.923 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 -- 172.21.15.153:0/1602479623 learned_addr learned my addr 172.21.15.153:0/1602479623 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:29.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa87fff700 1 --2- 172.21.15.153:0/1602479623 >> v2:172.21.15.177:3300/0 conn(0x7ffa8815d1f0 0x7ffa880b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8d0b8700 1 --2- 172.21.15.153:0/1602479623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa880ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 -- 172.21.15.153:0/1602479623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 msgr2=0x7ffa880ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:29.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 --2- 172.21.15.153:0/1602479623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa880ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 -- 172.21.15.153:0/1602479623 >> v2:172.21.15.177:3300/0 conn(0x7ffa8815d1f0 msgr2=0x7ffa880b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:29.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 --2- 172.21.15.153:0/1602479623 >> v2:172.21.15.177:3300/0 conn(0x7ffa8815d1f0 0x7ffa880b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.924 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 -- 172.21.15.153:0/1602479623 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa880cb0d0 con 0x7ffa8806fb30 2023-11-28T13:07:29.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa87fff700 1 --2- 172.21.15.153:0/1602479623 >> v2:172.21.15.177:3300/0 conn(0x7ffa8815d1f0 0x7ffa880b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:29.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8d0b8700 1 --2- 172.21.15.153:0/1602479623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa880ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:29.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa8c8b7700 1 --2- 172.21.15.153:0/1602479623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8806fb30 0x7ffa88158ce0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffa8806da60 tx=0x7ffa7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8de575416f8fb48 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa877fe700 1 -- 172.21.15.153:0/1602479623 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00b870 con 0x7ffa8806fb30 2023-11-28T13:07:29.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa877fe700 1 -- 172.21.15.153:0/1602479623 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c00b9d0 con 0x7ffa8806fb30 2023-11-28T13:07:29.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.920+0000 7ffa877fe700 1 -- 172.21.15.153:0/1602479623 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00bd60 con 0x7ffa8806fb30 2023-11-28T13:07:29.925 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/1602479623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8806fb30 msgr2=0x7ffa88158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/1602479623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8806fb30 0x7ffa88158ce0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ffa8806da60 tx=0x7ffa7c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/1602479623 shutdown_connections 2023-11-28T13:07:29.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/1602479623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa880ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/1602479623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8806fb30 0x7ffa88158ce0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/1602479623 >> v2:172.21.15.177:3300/0 conn(0x7ffa8815d1f0 0x7ffa880b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/1602479623 >> 172.21.15.153:0/1602479623 conn(0x7ffa880cb670 msgr2=0x7ffa880cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:29.926 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/1602479623 shutdown_connections 2023-11-28T13:07:29.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.921+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/1602479623 wait complete. 2023-11-28T13:07:29.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.922+0000 7ffa8eb1b700 1 Processor -- start 2023-11-28T13:07:29.927 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.922+0000 7ffa8eb1b700 1 -- start start 2023-11-28T13:07:29.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8eb1b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa8806fb30 0x7ffa880c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.929 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8eb1b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa88157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8eb1b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8815d1f0 0x7ffa880c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8eb1b700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880b0530 con 0x7ffa8815d1f0 2023-11-28T13:07:29.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8eb1b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880c7cc0 con 0x7ffa8806fb30 2023-11-28T13:07:29.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8eb1b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880c49a0 con 0x7ffa880cdda0 2023-11-28T13:07:29.930 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8815d1f0 0x7ffa880c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8815d1f0 0x7ffa880c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43690/0 (socket says 172.21.15.153:43690) 2023-11-28T13:07:29.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 -- 172.21.15.153:0/3395331458 learned_addr learned my addr 172.21.15.153:0/3395331458 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:29.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8c8b7700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa8806fb30 0x7ffa880c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa87fff700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa88157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 -- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 msgr2=0x7ffa88157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa88157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.931 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 -- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa8806fb30 msgr2=0x7ffa880c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:29.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa8806fb30 0x7ffa880c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:29.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 -- 172.21.15.153:0/3395331458 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c009550 con 0x7ffa8815d1f0 2023-11-28T13:07:29.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.923+0000 7ffa8d0b8700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8815d1f0 0x7ffa880c4460 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000cef0 tx=0x7ffa8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.924+0000 7ffa857fa700 1 -- 172.21.15.153:0/3395331458 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000add0 con 0x7ffa8815d1f0 2023-11-28T13:07:29.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.924+0000 7ffa857fa700 1 -- 172.21.15.153:0/3395331458 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa80015440 con 0x7ffa8815d1f0 2023-11-28T13:07:29.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.924+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8806b8c0 con 0x7ffa8815d1f0 2023-11-28T13:07:29.932 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.924+0000 7ffa857fa700 1 -- 172.21.15.153:0/3395331458 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa800157f0 con 0x7ffa8815d1f0 2023-11-28T13:07:29.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.924+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa880cb3b0 con 0x7ffa8815d1f0 2023-11-28T13:07:29.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.925+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8806bba0 con 0x7ffa8815d1f0 2023-11-28T13:07:29.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.925+0000 7ffa857fa700 1 -- 172.21.15.153:0/3395331458 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffa8001c030 con 0x7ffa8815d1f0 2023-11-28T13:07:29.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.926+0000 7ffa857fa700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffa6c068320 0x7ffa6c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:29.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.926+0000 7ffa857fa700 1 -- 172.21.15.153:0/3395331458 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffa80085df0 con 0x7ffa8815d1f0 2023-11-28T13:07:29.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.926+0000 7ffa8c8b7700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffa6c068320 0x7ffa6c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:29.933 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.926+0000 7ffa8c8b7700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffa6c068320 0x7ffa6c06a7d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c000c00 tx=0x7ffa7c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:29.934 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:29.930+0000 7ffa857fa700 1 -- 172.21.15.153:0/3395331458 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffa800528c0 con 0x7ffa8815d1f0 2023-11-28T13:07:30.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.111+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa880745c0 con 0x7ffa6c068320 2023-11-28T13:07:30.112 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.112+0000 7ffa857fa700 1 -- 172.21.15.153:0/3395331458 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffa880745c0 con 0x7ffa6c068320 2023-11-28T13:07:30.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffa6c068320 msgr2=0x7ffa6c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffa6c068320 0x7ffa6c06a7d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c000c00 tx=0x7ffa7c008030 comp rx=0 tx=0).stop 2023-11-28T13:07:30.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8815d1f0 msgr2=0x7ffa880c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.115 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8815d1f0 0x7ffa880c4460 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000cef0 tx=0x7ffa8000a340 comp rx=0 tx=0).stop 2023-11-28T13:07:30.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 shutdown_connections 2023-11-28T13:07:30.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7ffa6c068320 0x7ffa6c06a7d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.116 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa8815d1f0 0x7ffa880c4460 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa8806fb30 0x7ffa880c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 --2- 172.21.15.153:0/3395331458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa880cdda0 0x7ffa88157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.117 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 >> 172.21.15.153:0/3395331458 conn(0x7ffa880cb670 msgr2=0x7ffa8806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 shutdown_connections 2023-11-28T13:07:30.118 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.114+0000 7ffa8eb1b700 1 -- 172.21.15.153:0/3395331458 wait complete. 2023-11-28T13:07:30.118 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T13:07:30.126 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T13:07:30.127 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:30.251 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.251+0000 7f7a52777700 1 Processor -- start 2023-11-28T13:07:30.252 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.251+0000 7f7a52777700 1 -- start start 2023-11-28T13:07:30.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.251+0000 7f7a52777700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7a4c06a980 0x7f7a4c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.251+0000 7f7a52777700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7a4c0af430 0x7f7a4c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.251+0000 7f7a52777700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0713c0 con 0x7f7a4c0af060 2023-11-28T13:07:30.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.251+0000 7f7a52777700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c06dee0 con 0x7f7a4c0af430 2023-11-28T13:07:30.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.251+0000 7f7a52777700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c0716a0 con 0x7f7a4c06a980 2023-11-28T13:07:30.254 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7a4c0af430 0x7f7a4c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f7a4c0af430 0x7f7a4c06d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43698/0 (socket says 172.21.15.153:43698) 2023-11-28T13:07:30.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 -- 172.21.15.153:0/3698430597 learned_addr learned my addr 172.21.15.153:0/3698430597 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:30.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a4b7fe700 1 --2- 172.21.15.153:0/3698430597 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c06a980 0x7f7a4c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 -- 172.21.15.153:0/3698430597 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c06a980 msgr2=0x7f7a4c070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:30.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 --2- 172.21.15.153:0/3698430597 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c06a980 0x7f7a4c070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 -- 172.21.15.153:0/3698430597 >> v1:172.21.15.177:6789/0 conn(0x7f7a4c0af060 legacy=0x7f7a4c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 -- 172.21.15.153:0/3698430597 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c071980 con 0x7f7a4c0af430 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a50d14700 1 --2- 172.21.15.153:0/3698430597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af430 0x7f7a4c06d9a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c06ad50 tx=0x7f7a4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283c7a0193eab581 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a4affd700 1 -- 172.21.15.153:0/3698430597 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400e070 con 0x7f7a4c0af430 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a4affd700 1 -- 172.21.15.153:0/3698430597 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4400b9c0 con 0x7f7a4c0af430 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.252+0000 7f7a4affd700 1 -- 172.21.15.153:0/3698430597 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400bd70 con 0x7f7a4c0af430 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 -- 172.21.15.153:0/3698430597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af430 msgr2=0x7f7a4c06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 --2- 172.21.15.153:0/3698430597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af430 0x7f7a4c06d9a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c06ad50 tx=0x7f7a4400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 -- 172.21.15.153:0/3698430597 shutdown_connections 2023-11-28T13:07:30.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 --2- 172.21.15.153:0/3698430597 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af430 0x7f7a4c06d9a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 --2- 172.21.15.153:0/3698430597 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c06a980 0x7f7a4c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 -- 172.21.15.153:0/3698430597 >> 172.21.15.153:0/3698430597 conn(0x7f7a4c06b380 msgr2=0x7f7a4c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 -- 172.21.15.153:0/3698430597 shutdown_connections 2023-11-28T13:07:30.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.253+0000 7f7a52777700 1 -- 172.21.15.153:0/3698430597 wait complete. 2023-11-28T13:07:30.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.254+0000 7f7a52777700 1 Processor -- start 2023-11-28T13:07:30.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.254+0000 7f7a52777700 1 -- start start 2023-11-28T13:07:30.260 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a52777700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4c06a980 0x7f7a4c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a52777700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af060 0x7f7a4c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a52777700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c0af430 0x7f7a4c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a52777700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c9f70 con 0x7f7a4c0af060 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a52777700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c9780 con 0x7f7a4c06a980 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a52777700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c15e430 con 0x7f7a4c0af430 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af060 0x7f7a4c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af060 0x7f7a4c15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43702/0 (socket says 172.21.15.153:43702) 2023-11-28T13:07:30.261 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 -- 172.21.15.153:0/3425962531 learned_addr learned my addr 172.21.15.153:0/3425962531 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a50d14700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c0af430 0x7f7a4c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4bfff700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4c06a980 0x7f7a4c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 -- 172.21.15.153:0/3425962531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c0af430 msgr2=0x7f7a4c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c0af430 0x7f7a4c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 -- 172.21.15.153:0/3425962531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4c06a980 msgr2=0x7f7a4c15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4c06a980 0x7f7a4c15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 -- 172.21.15.153:0/3425962531 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44009b30 con 0x7f7a4c0af060 2023-11-28T13:07:30.262 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4bfff700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4c06a980 0x7f7a4c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.255+0000 7f7a4b7fe700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af060 0x7f7a4c15a730 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00d7d0 tx=0x7f7a3c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.256+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0125f0 con 0x7f7a4c0af060 2023-11-28T13:07:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.256+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c012750 con 0x7f7a4c0af060 2023-11-28T13:07:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.256+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c071f40 con 0x7f7a4c0af060 2023-11-28T13:07:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.256+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c012ae0 con 0x7f7a4c0af060 2023-11-28T13:07:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.256+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c071c60 con 0x7f7a4c0af060 2023-11-28T13:07:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.257+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a4c0ca660 con 0x7f7a4c0af060 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.257+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a3c012c40 con 0x7f7a4c0af060 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.257+0000 7f7a48ff9700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a380682f0 0x7f7a3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.257+0000 7f7a4bfff700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a380682f0 0x7f7a3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.257+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7a3c085b50 con 0x7f7a4c0af060 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.258+0000 7f7a48ff9700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7f7a3806e760 0x7f7a38070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.258+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 --> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a38071240 con 0x7f7a3806e760 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.258+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a3c08a220 con 0x7f7a4c0af060 2023-11-28T13:07:30.264 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.258+0000 7f7a50d14700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7f7a3806e760 0x7f7a38070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.258+0000 7f7a4bfff700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a380682f0 0x7f7a3806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0c6c30 tx=0x7f7a40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.258+0000 7f7a50d14700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7f7a3806e760 0x7f7a38070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.265 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.259+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== osd.2 v2:172.21.15.153:6808/103562 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7a38071240 con 0x7f7a3806e760 2023-11-28T13:07:30.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 --> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7a4c15e5a0 con 0x7f7a3806e760 2023-11-28T13:07:30.273 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a48ff9700 1 -- 172.21.15.153:0/3425962531 <== osd.2 v2:172.21.15.153:6808/103562 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7a4c15e5a0 con 0x7f7a3806e760 2023-11-28T13:07:30.274 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:30.274 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:30.274 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:30.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7f7a3806e760 msgr2=0x7f7a38070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6808/103562,v1:172.21.15.153:6809/103562] conn(0x7f7a3806e760 0x7f7a38070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.275 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a380682f0 msgr2=0x7f7a3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f7a380682f0 0x7f7a3806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0c6c30 tx=0x7f7a40009040 comp rx=0 tx=0).stop 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af060 msgr2=0x7f7a4c15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 --2- 172.21.15.153:0/3425962531 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a4c0af060 0x7f7a4c15a730 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00d7d0 tx=0x7f7a3c00dae0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a50d14700 1 -- 172.21.15.153:0/3425962531 reap_dead start 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 shutdown_connections 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 >> 172.21.15.153:0/3425962531 conn(0x7f7a4c06b380 msgr2=0x7f7a4c0ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 shutdown_connections 2023-11-28T13:07:30.276 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.273+0000 7f7a52777700 1 -- 172.21.15.153:0/3425962531 wait complete. 2023-11-28T13:07:30.286 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T13:07:30.407 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.406+0000 7f0a7425d700 1 Processor -- start 2023-11-28T13:07:30.408 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.407+0000 7f0a7425d700 1 -- start start 2023-11-28T13:07:30.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.407+0000 7f0a7425d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0a6c0d01b0 0x7f0a6c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a7425d700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0b0230 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.410 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a7425d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0b37d0 con 0x7f0a6c0b16d0 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a7425d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c15cec0 con 0x7f0a6c0d01b0 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a717f8700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f0a6c0b20b0 0x7f0a6c0b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33642/0 (socket says 172.21.15.153:33642) 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a717f8700 1 -- 172.21.15.153:0/2904465889 learned_addr learned my addr 172.21.15.153:0/2904465889 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a71ff9700 1 --2- 172.21.15.153:0/2904465889 >> v2:172.21.15.177:3300/0 conn(0x7f0a6c0d01b0 0x7f0a6c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934372956 0 0) 0x7f0a6c0b0230 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a580035b0 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12628466 0 0) 0x7f0a6c0b37d0 con 0x7f0a6c0b16d0 2023-11-28T13:07:30.411 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c0b0230 con 0x7f0a6c0b16d0 2023-11-28T13:07:30.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a71ff9700 1 -- 172.21.15.153:0/2904465889 >> v1:172.21.15.177:6789/0 conn(0x7f0a6c0b16d0 legacy=0x7f0a6c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789025874 0 0) 0x7f0a580035b0 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a71ff9700 1 -- 172.21.15.153:0/2904465889 >> v1:172.21.15.153:6789/0 conn(0x7f0a6c0b20b0 legacy=0x7f0a6c0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a71ff9700 1 -- 172.21.15.153:0/2904465889 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0b4570 con 0x7f0a6c0d01b0 2023-11-28T13:07:30.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.408+0000 7f0a71ff9700 1 --2- 172.21.15.153:0/2904465889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0d01b0 0x7f0a6c0d0c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009840 tx=0x7f0a5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1781cb05b6691b80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68001670 con 0x7f0a6c0d01b0 2023-11-28T13:07:30.412 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c004030 con 0x7f0a6c0d01b0 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a70ff7700 1 -- 172.21.15.153:0/2904465889 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0043c0 con 0x7f0a6c0d01b0 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a7425d700 1 -- 172.21.15.153:0/2904465889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0d01b0 msgr2=0x7f0a6c0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a7425d700 1 --2- 172.21.15.153:0/2904465889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0d01b0 0x7f0a6c0d0c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009840 tx=0x7f0a5c009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a7425d700 1 -- 172.21.15.153:0/2904465889 shutdown_connections 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a7425d700 1 --2- 172.21.15.153:0/2904465889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0d01b0 0x7f0a6c0d0c20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a7425d700 1 -- 172.21.15.153:0/2904465889 >> 172.21.15.153:0/2904465889 conn(0x7f0a6c09c3f0 msgr2=0x7f0a6c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a7425d700 1 -- 172.21.15.153:0/2904465889 shutdown_connections 2023-11-28T13:07:30.413 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.409+0000 7f0a7425d700 1 -- 172.21.15.153:0/2904465889 wait complete. 2023-11-28T13:07:30.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.410+0000 7f0a7425d700 1 Processor -- start 2023-11-28T13:07:30.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a7425d700 1 -- start start 2023-11-28T13:07:30.416 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a7425d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0b16d0 0x7f0a6c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a7425d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c0b20b0 0x7f0a6c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a7425d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a6c0d01b0 0x7f0a6c0d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a7425d700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f31b0 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a7425d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f25a0 con 0x7f0a6c0b16d0 2023-11-28T13:07:30.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a7425d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c1f2880 con 0x7f0a6c0d01b0 2023-11-28T13:07:30.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a717f8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c0b20b0 0x7f0a6c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.417 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a717f8700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c0b20b0 0x7f0a6c0cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43718/0 (socket says 172.21.15.153:43718) 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a717f8700 1 -- 172.21.15.153:0/1230704330 learned_addr learned my addr 172.21.15.153:0/1230704330 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.411+0000 7f0a727fa700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a6c0d01b0 0x7f0a6c0d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a71ff9700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0b16d0 0x7f0a6c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a717f8700 1 -- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a6c0d01b0 msgr2=0x7f0a6c0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a717f8700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a6c0d01b0 0x7f0a6c0d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a717f8700 1 -- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0b16d0 msgr2=0x7f0a6c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a717f8700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0b16d0 0x7f0a6c158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.418 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a717f8700 1 -- 172.21.15.153:0/1230704330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0094a0 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a717f8700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c0b20b0 0x7f0a6c0cef30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800c750 tx=0x7f0a6800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a627fc700 1 -- 172.21.15.153:0/1230704330 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68007560 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a627fc700 1 -- 172.21.15.153:0/1230704330 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a680076c0 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a627fc700 1 -- 172.21.15.153:0/1230704330 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004a70 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c0afbe0 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.412+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c0b4850 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.413+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c0afec0 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.419 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.413+0000 7f0a627fc700 1 -- 172.21.15.153:0/1230704330 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0a68010070 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.414+0000 7f0a627fc700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0a500682f0 0x7f0a5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.414+0000 7f0a627fc700 1 -- 172.21.15.153:0/1230704330 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0a68084c30 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.414+0000 7f0a71ff9700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0a500682f0 0x7f0a5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.414+0000 7f0a71ff9700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0a500682f0 0x7f0a5006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00a010 tx=0x7f0a5c013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.420 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.418+0000 7f0a627fc700 1 -- 172.21.15.153:0/1230704330 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0a68051710 con 0x7f0a6c0b20b0 2023-11-28T13:07:30.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.603+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a6c068690 con 0x7f0a500682f0 2023-11-28T13:07:30.604 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.603+0000 7f0a627fc700 1 -- 172.21.15.153:0/1230704330 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0a6c068690 con 0x7f0a500682f0 2023-11-28T13:07:30.606 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0a500682f0 msgr2=0x7f0a5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0a500682f0 0x7f0a5006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00a010 tx=0x7f0a5c013040 comp rx=0 tx=0).stop 2023-11-28T13:07:30.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c0b20b0 msgr2=0x7f0a6c0cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c0b20b0 0x7f0a6c0cef30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800c750 tx=0x7f0a6800cb10 comp rx=0 tx=0).stop 2023-11-28T13:07:30.607 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 shutdown_connections 2023-11-28T13:07:30.608 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a6c0d01b0 0x7f0a6c0d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a6c0b16d0 0x7f0a6c158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f0a500682f0 0x7f0a5006a7a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 --2- 172.21.15.153:0/1230704330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a6c0b20b0 0x7f0a6c0cef30 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 >> 172.21.15.153:0/1230704330 conn(0x7f0a6c09c3f0 msgr2=0x7f0a6c06b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 shutdown_connections 2023-11-28T13:07:30.610 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.605+0000 7f0a7425d700 1 -- 172.21.15.153:0/1230704330 wait complete. 2023-11-28T13:07:30.610 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T13:07:30.618 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T13:07:30.618 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:30.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.741+0000 7f9252754700 1 Processor -- start 2023-11-28T13:07:30.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.741+0000 7f9252754700 1 -- start start 2023-11-28T13:07:30.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f9252754700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f924c15aba0 0x7f924c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.745 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f9252754700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0b6ae0 0x7f924c0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f9252754700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f924c0cfa20 0x7f924c0d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f9252754700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c15af70 con 0x7f924c15aba0 2023-11-28T13:07:30.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f9252754700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c0cc870 con 0x7f924c0cfa20 2023-11-28T13:07:30.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f9252754700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0cfec0 con 0x7f924c0b6ae0 2023-11-28T13:07:30.746 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f924bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f924c15aba0 0x7f924c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f924bfff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f924c15aba0 0x7f924c0b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43724/0 (socket says 172.21.15.153:43724) 2023-11-28T13:07:30.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.742+0000 7f924bfff700 1 -- 172.21.15.153:0/4218478638 learned_addr learned my addr 172.21.15.153:0/4218478638 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:30.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924b7fe700 1 --2- 172.21.15.153:0/4218478638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0b6ae0 0x7f924c0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f9250cf1700 1 --2- 172.21.15.153:0/4218478638 >> v2:172.21.15.177:3300/0 conn(0x7f924c0cfa20 0x7f924c0d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924bfff700 1 -- 172.21.15.153:0/4218478638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0b6ae0 msgr2=0x7f924c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:30.748 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924bfff700 1 --2- 172.21.15.153:0/4218478638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0b6ae0 0x7f924c0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924bfff700 1 -- 172.21.15.153:0/4218478638 >> v2:172.21.15.177:3300/0 conn(0x7f924c0cfa20 msgr2=0x7f924c0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:30.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924bfff700 1 --2- 172.21.15.153:0/4218478638 >> v2:172.21.15.177:3300/0 conn(0x7f924c0cfa20 0x7f924c0d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924bfff700 1 -- 172.21.15.153:0/4218478638 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0b4290 con 0x7f924c15aba0 2023-11-28T13:07:30.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924b7fe700 1 --2- 172.21.15.153:0/4218478638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0b6ae0 0x7f924c0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:30.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f9250cf1700 1 --2- 172.21.15.153:0/4218478638 >> v2:172.21.15.177:3300/0 conn(0x7f924c0cfa20 0x7f924c0d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:30.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924bfff700 1 --2- 172.21.15.153:0/4218478638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c0b3d50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f924c0ce7f0 tx=0x7f9234009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fccf60c532849db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924affd700 1 -- 172.21.15.153:0/4218478638 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923400b870 con 0x7f924c15aba0 2023-11-28T13:07:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924affd700 1 -- 172.21.15.153:0/4218478638 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923400b9d0 con 0x7f924c15aba0 2023-11-28T13:07:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f924affd700 1 -- 172.21.15.153:0/4218478638 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923400bd60 con 0x7f924c15aba0 2023-11-28T13:07:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f9252754700 1 -- 172.21.15.153:0/4218478638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 msgr2=0x7f924c0b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.743+0000 7f9252754700 1 --2- 172.21.15.153:0/4218478638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c0b3d50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f924c0ce7f0 tx=0x7f9234009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.744+0000 7f9252754700 1 -- 172.21.15.153:0/4218478638 shutdown_connections 2023-11-28T13:07:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.744+0000 7f9252754700 1 --2- 172.21.15.153:0/4218478638 >> v2:172.21.15.177:3300/0 conn(0x7f924c0cfa20 0x7f924c0d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.744+0000 7f9252754700 1 --2- 172.21.15.153:0/4218478638 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c0b3d50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.744+0000 7f9252754700 1 --2- 172.21.15.153:0/4218478638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0b6ae0 0x7f924c0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.744+0000 7f9252754700 1 -- 172.21.15.153:0/4218478638 >> 172.21.15.153:0/4218478638 conn(0x7f924c06df70 msgr2=0x7f924c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.744+0000 7f9252754700 1 -- 172.21.15.153:0/4218478638 shutdown_connections 2023-11-28T13:07:30.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.744+0000 7f9252754700 1 -- 172.21.15.153:0/4218478638 wait complete. 2023-11-28T13:07:30.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.745+0000 7f9252754700 1 Processor -- start 2023-11-28T13:07:30.751 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.745+0000 7f9252754700 1 -- start start 2023-11-28T13:07:30.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0cfa20 0x7f924c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.754 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f924c153ea0 0x7f924c0afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c15b0b0 con 0x7f924c15aba0 2023-11-28T13:07:30.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c153cb0 con 0x7f924c153ea0 2023-11-28T13:07:30.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c06c620 con 0x7f924c0cfa20 2023-11-28T13:07:30.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c1581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43732/0 (socket says 172.21.15.153:43732) 2023-11-28T13:07:30.755 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 -- 172.21.15.153:0/797985754 learned_addr learned my addr 172.21.15.153:0/797985754 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 -- 172.21.15.153:0/797985754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0cfa20 msgr2=0x7f924c153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f924c0cfa20 0x7f924c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9250cf1700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f924c153ea0 0x7f924c0afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 -- 172.21.15.153:0/797985754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f924c153ea0 msgr2=0x7f924c0afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f924c153ea0 0x7f924c0afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 -- 172.21.15.153:0/797985754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234009550 con 0x7f924c15aba0 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f924b7fe700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c1581e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f923c00d830 tx=0x7f923c00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.756 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c004030 con 0x7f924c15aba0 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c004190 con 0x7f924c15aba0 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c0ccd80 con 0x7f924c15aba0 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c004520 con 0x7f924c15aba0 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.746+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c0ccaa0 con 0x7f924c15aba0 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.747+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f924c0cd060 con 0x7f924c15aba0 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.748+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f923c01a020 con 0x7f924c15aba0 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.748+0000 7f9248ff9700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f922c0682f0 0x7f922c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.748+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f923c085e30 con 0x7f924c15aba0 2023-11-28T13:07:30.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.748+0000 7f924bfff700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f922c0682f0 0x7f922c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.748+0000 7f9248ff9700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7f922c06e760 0x7f922c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.748+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 --> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f922c071240 con 0x7f922c06e760 2023-11-28T13:07:30.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.748+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f923c0884e0 con 0x7f924c15aba0 2023-11-28T13:07:30.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.749+0000 7f9250cf1700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7f922c06e760 0x7f922c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.749+0000 7f924bfff700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f922c0682f0 0x7f922c06a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9234009c10 tx=0x7f9234010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.758 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.749+0000 7f9250cf1700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7f922c06e760 0x7f922c070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.759 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.749+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== osd.3 v2:172.21.15.153:6822/103565 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f922c071240 con 0x7f922c06e760 2023-11-28T13:07:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.763+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 --> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f924c068690 con 0x7f922c06e760 2023-11-28T13:07:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9248ff9700 1 -- 172.21.15.153:0/797985754 <== osd.3 v2:172.21.15.153:6822/103565 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f924c068690 con 0x7f922c06e760 2023-11-28T13:07:30.765 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:30.765 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:30.765 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7f922c06e760 msgr2=0x7f922c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6822/103565,v1:172.21.15.153:6824/103565] conn(0x7f922c06e760 0x7f922c070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f922c0682f0 msgr2=0x7f922c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f922c0682f0 0x7f922c06a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9234009c10 tx=0x7f9234010000 comp rx=0 tx=0).stop 2023-11-28T13:07:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 msgr2=0x7f924c1581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 --2- 172.21.15.153:0/797985754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f924c15aba0 0x7f924c1581e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f923c00d830 tx=0x7f923c00dbf0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9250cf1700 1 -- 172.21.15.153:0/797985754 reap_dead start 2023-11-28T13:07:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 shutdown_connections 2023-11-28T13:07:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 >> 172.21.15.153:0/797985754 conn(0x7f924c06df70 msgr2=0x7f924c0d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 shutdown_connections 2023-11-28T13:07:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.764+0000 7f9252754700 1 -- 172.21.15.153:0/797985754 wait complete. 2023-11-28T13:07:30.777 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T13:07:30.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.897+0000 7f4e7db51700 1 Processor -- start 2023-11-28T13:07:30.898 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.897+0000 7f4e7db51700 1 -- start start 2023-11-28T13:07:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e7db51700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e7806dee0 0x7f4e7806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e7db51700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4e78160540 0x7f4e78160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e7db51700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7815d060 con 0x7f4e78071040 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e7db51700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7815d340 con 0x7f4e7806dee0 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e7db51700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7815d620 con 0x7f4e78160540 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e7806dee0 0x7f4e7806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f4e7806dee0 0x7f4e7806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43742/0 (socket says 172.21.15.153:43742) 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 -- 172.21.15.153:0/36258769 learned_addr learned my addr 172.21.15.153:0/36258769 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e77fff700 1 --2- 172.21.15.153:0/36258769 >> v2:172.21.15.177:3300/0 conn(0x7f4e78160540 0x7f4e78160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 -- 172.21.15.153:0/36258769 >> v2:172.21.15.177:3300/0 conn(0x7f4e78160540 msgr2=0x7f4e78160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 --2- 172.21.15.153:0/36258769 >> v2:172.21.15.177:3300/0 conn(0x7f4e78160540 0x7f4e78160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 -- 172.21.15.153:0/36258769 >> v1:172.21.15.177:6789/0 conn(0x7f4e78071040 legacy=0x7f4e78071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 -- 172.21.15.153:0/36258769 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7815d900 con 0x7f4e7806dee0 2023-11-28T13:07:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e76ffd700 1 --2- 172.21.15.153:0/36258769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e7806e330 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4e780c3a10 tx=0x7f4e68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8548134f68b95617 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.898+0000 7f4e767fc700 1 -- 172.21.15.153:0/36258769 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011830 con 0x7f4e7806dee0 2023-11-28T13:07:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e767fc700 1 -- 172.21.15.153:0/36258769 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e68011990 con 0x7f4e7806dee0 2023-11-28T13:07:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e767fc700 1 -- 172.21.15.153:0/36258769 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011d20 con 0x7f4e7806dee0 2023-11-28T13:07:30.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 -- 172.21.15.153:0/36258769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 msgr2=0x7f4e7806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 --2- 172.21.15.153:0/36258769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e7806e330 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4e780c3a10 tx=0x7f4e68009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 -- 172.21.15.153:0/36258769 shutdown_connections 2023-11-28T13:07:30.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 --2- 172.21.15.153:0/36258769 >> v2:172.21.15.177:3300/0 conn(0x7f4e78160540 0x7f4e78160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 --2- 172.21.15.153:0/36258769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e7806e330 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.903 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 -- 172.21.15.153:0/36258769 >> 172.21.15.153:0/36258769 conn(0x7f4e780d0000 msgr2=0x7f4e780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 -- 172.21.15.153:0/36258769 shutdown_connections 2023-11-28T13:07:30.904 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.899+0000 7f4e7db51700 1 -- 172.21.15.153:0/36258769 wait complete. 2023-11-28T13:07:30.906 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.900+0000 7f4e7db51700 1 Processor -- start 2023-11-28T13:07:30.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e7db51700 1 -- start start 2023-11-28T13:07:30.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e7db51700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e780b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e7db51700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78071040 0x7f4e780b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e7db51700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e78160540 0x7f4e780b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.907 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e7db51700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780c43d0 con 0x7f4e7806dee0 2023-11-28T13:07:30.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e7db51700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780b4580 con 0x7f4e78160540 2023-11-28T13:07:30.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e7db51700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780c3aa0 con 0x7f4e78071040 2023-11-28T13:07:30.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e777fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e780b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.908 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e777fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e780b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43746/0 (socket says 172.21.15.153:43746) 2023-11-28T13:07:30.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e777fe700 1 -- 172.21.15.153:0/2789599199 learned_addr learned my addr 172.21.15.153:0/2789599199 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:30.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e76ffd700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78071040 0x7f4e780b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.901+0000 7f4e77fff700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e78160540 0x7f4e780b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.909 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e777fe700 1 -- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78071040 msgr2=0x7f4e780b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e777fe700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78071040 0x7f4e780b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e777fe700 1 -- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e78160540 msgr2=0x7f4e780b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e777fe700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e78160540 0x7f4e780b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.910 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e777fe700 1 -- 172.21.15.153:0/2789599199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68009550 con 0x7f4e7806dee0 2023-11-28T13:07:30.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e777fe700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e780b3ba0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c450 tx=0x7f4e6c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e5ffff700 1 -- 172.21.15.153:0/2789599199 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0052d0 con 0x7f4e7806dee0 2023-11-28T13:07:30.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e5ffff700 1 -- 172.21.15.153:0/2789599199 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c005430 con 0x7f4e7806dee0 2023-11-28T13:07:30.911 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7815ccd0 con 0x7f4e7806dee0 2023-11-28T13:07:30.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e5ffff700 1 -- 172.21.15.153:0/2789599199 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0038c0 con 0x7f4e7806dee0 2023-11-28T13:07:30.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.902+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7806e870 con 0x7f4e7806dee0 2023-11-28T13:07:30.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.903+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7806a270 con 0x7f4e7806dee0 2023-11-28T13:07:30.912 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.903+0000 7f4e5ffff700 1 -- 172.21.15.153:0/2789599199 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e6c019030 con 0x7f4e7806dee0 2023-11-28T13:07:30.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.904+0000 7f4e5ffff700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e640682f0 0x7f4e6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.904+0000 7f4e5ffff700 1 -- 172.21.15.153:0/2789599199 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4e6c084b30 con 0x7f4e7806dee0 2023-11-28T13:07:30.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.904+0000 7f4e76ffd700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e640682f0 0x7f4e6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.913 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.904+0000 7f4e76ffd700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e640682f0 0x7f4e6406a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e68014040 tx=0x7f4e68008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.914 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:30.908+0000 7f4e5ffff700 1 -- 172.21.15.153:0/2789599199 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e6c051690 con 0x7f4e7806dee0 2023-11-28T13:07:31.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.093+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e780c4510 con 0x7f4e640682f0 2023-11-28T13:07:31.094 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.093+0000 7f4e5ffff700 1 -- 172.21.15.153:0/2789599199 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4e68004040 con 0x7f4e640682f0 2023-11-28T13:07:31.096 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e640682f0 msgr2=0x7f4e6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e640682f0 0x7f4e6406a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e68014040 tx=0x7f4e68008030 comp rx=0 tx=0).stop 2023-11-28T13:07:31.097 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 msgr2=0x7f4e780b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.098 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e780b3ba0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c450 tx=0x7f4e6c00c760 comp rx=0 tx=0).stop 2023-11-28T13:07:31.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 shutdown_connections 2023-11-28T13:07:31.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4e640682f0 0x7f4e6406a7a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e7806dee0 0x7f4e780b3ba0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e78160540 0x7f4e780b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 --2- 172.21.15.153:0/2789599199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78071040 0x7f4e780b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.099 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 >> 172.21.15.153:0/2789599199 conn(0x7f4e780d0000 msgr2=0x7f4e7815ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.095+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 shutdown_connections 2023-11-28T13:07:31.100 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.096+0000 7f4e7db51700 1 -- 172.21.15.153:0/2789599199 wait complete. 2023-11-28T13:07:31.100 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T13:07:31.108 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T13:07:31.109 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:31.233 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.233+0000 7fc1a9d59700 1 Processor -- start 2023-11-28T13:07:31.234 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.233+0000 7fc1a9d59700 1 -- start start 2023-11-28T13:07:31.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a9d59700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a41596b0 0x7fc1a4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a9d59700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1a40cc260 0x7fc1a40cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.236 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a9d59700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a40cc7b0 con 0x7fc1a40d02a0 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a9d59700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a40d0720 con 0x7fc1a40cc260 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a9d59700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40cd610 con 0x7fc1a41596b0 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1a40cc260 0x7fc1a40cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7fc1a40cc260 0x7fc1a40cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43756/0 (socket says 172.21.15.153:43756) 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/4083802326 learned_addr learned my addr 172.21.15.153:0/4083802326 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a37fe700 1 --2- 172.21.15.153:0/4083802326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a41596b0 0x7fc1a4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/4083802326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a41596b0 msgr2=0x7fc1a4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:31.237 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 --2- 172.21.15.153:0/4083802326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a41596b0 0x7fc1a4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/4083802326 >> v1:172.21.15.177:6789/0 conn(0x7fc1a40d02a0 legacy=0x7fc1a415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/4083802326 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a4159a80 con 0x7fc1a40cc260 2023-11-28T13:07:31.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a2ffd700 1 --2- 172.21.15.153:0/4083802326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40cc260 0x7fc1a40cfd60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc1980098f0 tx=0x7fc198009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df581524a260053 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a27fc700 1 -- 172.21.15.153:0/4083802326 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800b8a0 con 0x7fc1a40cc260 2023-11-28T13:07:31.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a27fc700 1 -- 172.21.15.153:0/4083802326 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19800ba00 con 0x7fc1a40cc260 2023-11-28T13:07:31.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.234+0000 7fc1a27fc700 1 -- 172.21.15.153:0/4083802326 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800bd90 con 0x7fc1a40cc260 2023-11-28T13:07:31.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 -- 172.21.15.153:0/4083802326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40cc260 msgr2=0x7fc1a40cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 --2- 172.21.15.153:0/4083802326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40cc260 0x7fc1a40cfd60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc1980098f0 tx=0x7fc198009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:31.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 -- 172.21.15.153:0/4083802326 shutdown_connections 2023-11-28T13:07:31.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 --2- 172.21.15.153:0/4083802326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a41596b0 0x7fc1a4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 --2- 172.21.15.153:0/4083802326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40cc260 0x7fc1a40cfd60 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 -- 172.21.15.153:0/4083802326 >> 172.21.15.153:0/4083802326 conn(0x7fc1a406b380 msgr2=0x7fc1a406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 -- 172.21.15.153:0/4083802326 shutdown_connections 2023-11-28T13:07:31.239 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.235+0000 7fc1a9d59700 1 -- 172.21.15.153:0/4083802326 wait complete. 2023-11-28T13:07:31.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.236+0000 7fc1a9d59700 1 Processor -- start 2023-11-28T13:07:31.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.236+0000 7fc1a9d59700 1 -- start start 2023-11-28T13:07:31.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a9d59700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a40cc260 0x7fc1a40c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a9d59700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40d02a0 0x7fc1a40cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a9d59700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1a41596b0 0x7fc1a4146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a9d59700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c3e50 con 0x7fc1a40d02a0 2023-11-28T13:07:31.243 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a9d59700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c4130 con 0x7fc1a41596b0 2023-11-28T13:07:31.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a9d59700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c4410 con 0x7fc1a40cc260 2023-11-28T13:07:31.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40d02a0 0x7fc1a40cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40d02a0 0x7fc1a40cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43762/0 (socket says 172.21.15.153:43762) 2023-11-28T13:07:31.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/3880171116 learned_addr learned my addr 172.21.15.153:0/3880171116 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:31.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a40cc260 msgr2=0x7fc1a40c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1a40cc260 0x7fc1a40c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.244 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1a41596b0 msgr2=0x7fc1a4146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1a41596b0 0x7fc1a4146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.237+0000 7fc1a2ffd700 1 -- 172.21.15.153:0/3880171116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009550 con 0x7fc1a40d02a0 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.238+0000 7fc1a2ffd700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40d02a0 0x7fc1a40cde20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc198014040 tx=0x7fc198004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.238+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980036a0 con 0x7fc1a40d02a0 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.238+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19801e070 con 0x7fc1a40d02a0 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.238+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a40cca00 con 0x7fc1a40d02a0 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.238+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198003a40 con 0x7fc1a40d02a0 2023-11-28T13:07:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.238+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a406dfc0 con 0x7fc1a40d02a0 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.239+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1a40ccce0 con 0x7fc1a40d02a0 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.239+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc19800f050 con 0x7fc1a40d02a0 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.239+0000 7fc18bfff700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc1900682f0 0x7fc19006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.239+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc198020070 con 0x7fc1a40d02a0 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.239+0000 7fc1a37fe700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc1900682f0 0x7fc19006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.240+0000 7fc18bfff700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fc19006e760 0x7fc190070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.240+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 --> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc190071240 con 0x7fc19006e760 2023-11-28T13:07:31.246 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.240+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc198089420 con 0x7fc1a40d02a0 2023-11-28T13:07:31.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.240+0000 7fc1a3fff700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fc19006e760 0x7fc190070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.240+0000 7fc1a37fe700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc1900682f0 0x7fc19006a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc1940097f0 tx=0x7fc194008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.240+0000 7fc1a3fff700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fc19006e760 0x7fc190070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.247 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.241+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== osd.4 v2:172.21.15.177:6809/103772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc190071240 con 0x7fc19006e760 2023-11-28T13:07:31.255 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.255+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 --> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc1a414a2e0 con 0x7fc19006e760 2023-11-28T13:07:31.256 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.255+0000 7fc18bfff700 1 -- 172.21.15.153:0/3880171116 <== osd.4 v2:172.21.15.177:6809/103772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc1a414a2e0 con 0x7fc19006e760 2023-11-28T13:07:31.256 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:31.256 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:31.256 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:31.257 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.255+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fc19006e760 msgr2=0x7fc190070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.255+0000 7fc1a9d59700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.177:6809/103772,v1:172.21.15.177:6811/103772] conn(0x7fc19006e760 0x7fc190070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc1900682f0 msgr2=0x7fc19006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fc1900682f0 0x7fc19006a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc1940097f0 tx=0x7fc194008040 comp rx=0 tx=0).stop 2023-11-28T13:07:31.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40d02a0 msgr2=0x7fc1a40cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 --2- 172.21.15.153:0/3880171116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1a40d02a0 0x7fc1a40cde20 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc198014040 tx=0x7fc198004750 comp rx=0 tx=0).stop 2023-11-28T13:07:31.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a3fff700 1 -- 172.21.15.153:0/3880171116 reap_dead start 2023-11-28T13:07:31.258 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 shutdown_connections 2023-11-28T13:07:31.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 >> 172.21.15.153:0/3880171116 conn(0x7fc1a406b380 msgr2=0x7fc1a415b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 shutdown_connections 2023-11-28T13:07:31.259 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.256+0000 7fc1a9d59700 1 -- 172.21.15.153:0/3880171116 wait complete. 2023-11-28T13:07:31.268 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T13:07:31.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.389+0000 7f4af577f700 1 Processor -- start 2023-11-28T13:07:31.390 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.389+0000 7f4af577f700 1 -- start start 2023-11-28T13:07:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.389+0000 7f4af577f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 0x7f4af00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.389+0000 7f4af577f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4af00d2320 0x7f4af00cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.389+0000 7f4af577f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af00d1950 con 0x7f4af015c9b0 2023-11-28T13:07:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.389+0000 7f4af577f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00b5800 con 0x7f4af00d2320 2023-11-28T13:07:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.389+0000 7f4af577f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b5300 con 0x7f4af0161340 2023-11-28T13:07:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aeeffd700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f4af015c9b0 0x7f4af00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33702/0 (socket says 172.21.15.153:33702) 2023-11-28T13:07:31.394 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aeeffd700 1 -- 172.21.15.153:0/326915311 learned_addr learned my addr 172.21.15.153:0/326915311 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855463702 0 0) 0x7f4af00d1950 con 0x7f4af015c9b0 2023-11-28T13:07:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ad80035b0 con 0x7f4af015c9b0 2023-11-28T13:07:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aef7fe700 1 --2- 172.21.15.153:0/326915311 >> v2:172.21.15.177:3300/0 conn(0x7f4af00d2320 0x7f4af00cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aee7fc700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 0x7f4af00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2320146853 0 0) 0x7f4ad80035b0 con 0x7f4af015c9b0 2023-11-28T13:07:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 msgr2=0x7f4af00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:31.395 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 0x7f4af00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 >> v2:172.21.15.177:3300/0 conn(0x7f4af00d2320 msgr2=0x7f4af00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 --2- 172.21.15.153:0/326915311 >> v2:172.21.15.177:3300/0 conn(0x7f4af00d2320 0x7f4af00cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00b5520 con 0x7f4af015c9b0 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f4ae4001670 con 0x7f4af015c9b0 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 >> v1:172.21.15.153:6789/0 conn(0x7f4af015c9b0 legacy=0x7f4af00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 0x7f4ad8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ad8003fb0 0x7f4ad8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.396 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ad8004d90 0x7f4ad8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b5800 con 0x7f4ad8003fb0 2023-11-28T13:07:31.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b5300 con 0x7f4ad8004d90 2023-11-28T13:07:31.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad800ba60 con 0x7f4af0161340 2023-11-28T13:07:31.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aee7fc700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ad8003fb0 0x7f4ad8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aeeffd700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 0x7f4ad8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.390+0000 7f4aef7fe700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ad8004d90 0x7f4ad8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.397 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aee7fc700 1 -- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 msgr2=0x7f4ad8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:31.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aee7fc700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4af0161340 0x7f4ad8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aee7fc700 1 -- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ad8004d90 msgr2=0x7f4ad8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:31.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aee7fc700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ad8004d90 0x7f4ad8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aee7fc700 1 -- 172.21.15.153:0/326915311 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80035b0 con 0x7f4ad8003fb0 2023-11-28T13:07:31.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aee7fc700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ad8003fb0 0x7f4ad8004850 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009b60 tx=0x7f4ad8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c303b13542c6cb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000e6c0 con 0x7f4ad8003fb0 2023-11-28T13:07:31.398 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae000e820 con 0x7f4ad8003fb0 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aedffb700 1 -- 172.21.15.153:0/326915311 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000eb90 con 0x7f4ad8003fb0 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4af577f700 1 -- 172.21.15.153:0/326915311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ad8003fb0 msgr2=0x7f4ad8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4af577f700 1 --2- 172.21.15.153:0/326915311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ad8003fb0 0x7f4ad8004850 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009b60 tx=0x7f4ad8005880 comp rx=0 tx=0).stop 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4aef7fe700 1 -- 172.21.15.153:0/326915311 reap_dead start 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4af577f700 1 -- 172.21.15.153:0/326915311 shutdown_connections 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4af577f700 1 -- 172.21.15.153:0/326915311 >> 172.21.15.153:0/326915311 conn(0x7f4af0072600 msgr2=0x7f4af00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4af577f700 1 -- 172.21.15.153:0/326915311 shutdown_connections 2023-11-28T13:07:31.399 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.391+0000 7f4af577f700 1 -- 172.21.15.153:0/326915311 wait complete. 2023-11-28T13:07:31.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.393+0000 7f4af577f700 1 Processor -- start 2023-11-28T13:07:31.400 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.393+0000 7f4af577f700 1 -- start start 2023-11-28T13:07:31.402 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ad8003fb0 0x7f4af014a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af01e7210 0x7f4af00c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4af00c8590 0x7f4af00c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00c8f40 con 0x7f4af00c8590 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af014a990 con 0x7f4af01e7210 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00c74f0 con 0x7f4ad8003fb0 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4af00c8590 0x7f4af00c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4af00c8590 0x7f4af00c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43786/0 (socket says 172.21.15.153:43786) 2023-11-28T13:07:31.403 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 -- 172.21.15.153:0/1959807995 learned_addr learned my addr 172.21.15.153:0/1959807995 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:31.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aeeffd700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ad8003fb0 0x7f4af014a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aee7fc700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af01e7210 0x7f4af00c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 -- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ad8003fb0 msgr2=0x7f4af014a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ad8003fb0 0x7f4af014a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 -- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af01e7210 msgr2=0x7f4af00c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af01e7210 0x7f4af00c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.404 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 -- 172.21.15.153:0/1959807995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00d15e0 con 0x7f4af00c8590 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4aef7fe700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4af00c8590 0x7f4af00c8a00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4af008bfd0 tx=0x7f4ae8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4ad77fe700 1 -- 172.21.15.153:0/1959807995 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800e040 con 0x7f4af00c8590 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4ad77fe700 1 -- 172.21.15.153:0/1959807995 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae800f040 con 0x7f4af00c8590 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af00d1300 con 0x7f4af00c8590 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4ad77fe700 1 -- 172.21.15.153:0/1959807995 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800fc40 con 0x7f4af00c8590 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.394+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af015cd80 con 0x7f4af00c8590 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.395+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af00745c0 con 0x7f4af00c8590 2023-11-28T13:07:31.405 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.396+0000 7f4ad77fe700 1 -- 172.21.15.153:0/1959807995 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ae800fda0 con 0x7f4af00c8590 2023-11-28T13:07:31.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.396+0000 7f4ad77fe700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4adc0682f0 0x7f4adc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.396+0000 7f4ad77fe700 1 -- 172.21.15.153:0/1959807995 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ae8085f30 con 0x7f4af00c8590 2023-11-28T13:07:31.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.396+0000 7f4aeeffd700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4adc0682f0 0x7f4adc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.397+0000 7f4aeeffd700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4adc0682f0 0x7f4adc06a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4001cb0 tx=0x7f4ae4007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.406 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.400+0000 7f4ad77fe700 1 -- 172.21.15.153:0/1959807995 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4ae8052a90 con 0x7f4af00c8590 2023-11-28T13:07:31.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.585+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4af00d08a0 con 0x7f4adc0682f0 2023-11-28T13:07:31.586 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.585+0000 7f4ad77fe700 1 -- 172.21.15.153:0/1959807995 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4af00d08a0 con 0x7f4adc0682f0 2023-11-28T13:07:31.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4adc0682f0 msgr2=0x7f4adc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4adc0682f0 0x7f4adc06a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4001cb0 tx=0x7f4ae4007980 comp rx=0 tx=0).stop 2023-11-28T13:07:31.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4af00c8590 msgr2=0x7f4af00c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.588 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4af00c8590 0x7f4af00c8a00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4af008bfd0 tx=0x7f4ae8011ed0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 shutdown_connections 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f4adc0682f0 0x7f4adc06a7a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4af00c8590 0x7f4af00c8a00 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ad8003fb0 0x7f4af014a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 --2- 172.21.15.153:0/1959807995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af01e7210 0x7f4af00c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 >> 172.21.15.153:0/1959807995 conn(0x7f4af0072600 msgr2=0x7f4af015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 shutdown_connections 2023-11-28T13:07:31.589 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.587+0000 7f4af577f700 1 -- 172.21.15.153:0/1959807995 wait complete. 2023-11-28T13:07:31.590 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T13:07:31.600 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T13:07:31.600 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:31.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.724+0000 7f8b7302a700 1 Processor -- start 2023-11-28T13:07:31.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.724+0000 7f8b7302a700 1 -- start start 2023-11-28T13:07:31.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b7302a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c0af060 0x7f8b6c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b7302a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8b6c0af430 0x7f8b6c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b7302a700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8b6c06a270 0x7f8b6c06a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b7302a700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c06dee0 con 0x7f8b6c06a270 2023-11-28T13:07:31.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b7302a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c06ac10 con 0x7f8b6c0af430 2023-11-28T13:07:31.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b7302a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0713c0 con 0x7f8b6c0af060 2023-11-28T13:07:31.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8b6c06a270 0x7f8b6c06a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f8b6c06a270 0x7f8b6c06a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43794/0 (socket says 172.21.15.153:43794) 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 -- 172.21.15.153:0/1877834824 learned_addr learned my addr 172.21.15.153:0/1877834824 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 -- 172.21.15.153:0/1877834824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c0af060 msgr2=0x7f8b6c070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b6bfff700 1 --2- 172.21.15.153:0/1877834824 >> v2:172.21.15.177:3300/0 conn(0x7f8b6c0af430 0x7f8b6c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b70dc6700 1 --2- 172.21.15.153:0/1877834824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c0af060 0x7f8b6c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 --2- 172.21.15.153:0/1877834824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c0af060 0x7f8b6c070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 -- 172.21.15.153:0/1877834824 >> v2:172.21.15.177:3300/0 conn(0x7f8b6c0af430 msgr2=0x7f8b6c06d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 --2- 172.21.15.153:0/1877834824 >> v2:172.21.15.177:3300/0 conn(0x7f8b6c0af430 0x7f8b6c06d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 -- 172.21.15.153:0/1877834824 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c0716a0 con 0x7f8b6c06a270 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b6bfff700 1 --2- 172.21.15.153:0/1877834824 >> v2:172.21.15.177:3300/0 conn(0x7f8b6c0af430 0x7f8b6c06d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.725+0000 7f8b715c7700 1 --2- 172.21.15.153:0/1877834824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c06a270 0x7f8b6c06a6d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0d0e30 tx=0x7f8b6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3396166cb6de016 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b6b7fe700 1 -- 172.21.15.153:0/1877834824 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400e070 con 0x7f8b6c06a270 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b6b7fe700 1 -- 172.21.15.153:0/1877834824 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6400b9c0 con 0x7f8b6c06a270 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b6b7fe700 1 -- 172.21.15.153:0/1877834824 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400bd70 con 0x7f8b6c06a270 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 -- 172.21.15.153:0/1877834824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c06a270 msgr2=0x7f8b6c06a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 --2- 172.21.15.153:0/1877834824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c06a270 0x7f8b6c06a6d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0d0e30 tx=0x7f8b6400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:31.730 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 -- 172.21.15.153:0/1877834824 shutdown_connections 2023-11-28T13:07:31.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 --2- 172.21.15.153:0/1877834824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c0af060 0x7f8b6c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 --2- 172.21.15.153:0/1877834824 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c06a270 0x7f8b6c06a6d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 --2- 172.21.15.153:0/1877834824 >> v2:172.21.15.177:3300/0 conn(0x7f8b6c0af430 0x7f8b6c06d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 -- 172.21.15.153:0/1877834824 >> 172.21.15.153:0/1877834824 conn(0x7f8b6c06b380 msgr2=0x7f8b6c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 -- 172.21.15.153:0/1877834824 shutdown_connections 2023-11-28T13:07:31.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.726+0000 7f8b7302a700 1 -- 172.21.15.153:0/1877834824 wait complete. 2023-11-28T13:07:31.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.727+0000 7f8b7302a700 1 Processor -- start 2023-11-28T13:07:31.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.727+0000 7f8b7302a700 1 -- start start 2023-11-28T13:07:31.734 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b7302a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c06a270 0x7f8b6c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b7302a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6c0af060 0x7f8b6c0c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b7302a700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c0af430 0x7f8b6c0c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b7302a700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0b18e0 con 0x7f8b6c0af430 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b7302a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0c9e70 con 0x7f8b6c0af060 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b7302a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0c3570 con 0x7f8b6c06a270 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b715c7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c0af430 0x7f8b6c0c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b715c7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c0af430 0x7f8b6c0c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43800/0 (socket says 172.21.15.153:43800) 2023-11-28T13:07:31.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.728+0000 7f8b715c7700 1 -- 172.21.15.153:0/1945437594 learned_addr learned my addr 172.21.15.153:0/1945437594 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b70dc6700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c06a270 0x7f8b6c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b6bfff700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6c0af060 0x7f8b6c0c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b715c7700 1 -- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c06a270 msgr2=0x7f8b6c0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b715c7700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b6c06a270 0x7f8b6c0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b715c7700 1 -- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6c0af060 msgr2=0x7f8b6c0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b715c7700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b6c0af060 0x7f8b6c0c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b715c7700 1 -- 172.21.15.153:0/1945437594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64009b30 con 0x7f8b6c0af430 2023-11-28T13:07:31.736 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b715c7700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c0af430 0x7f8b6c0c5cd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400b370 tx=0x7f8b64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004250 con 0x7f8b6c0af430 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b640043f0 con 0x7f8b6c0af430 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c071c60 con 0x7f8b6c0af430 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640047a0 con 0x7f8b6c0af430 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.729+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c071980 con 0x7f8b6c0af430 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.730+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b6c071f40 con 0x7f8b6c0af430 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.730+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8b6401b430 con 0x7f8b6c0af430 2023-11-28T13:07:31.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b697fa700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8b50068320 0x7f8b5006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8b64086c50 con 0x7f8b6c0af430 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b70dc6700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8b50068320 0x7f8b5006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b697fa700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f8b5006e790 0x7f8b50070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 --> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b50071270 con 0x7f8b5006e790 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b6408c070 con 0x7f8b6c0af430 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b70dc6700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8b50068320 0x7f8b5006a7d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006fd0 tx=0x7f8b60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.731+0000 7f8b6bfff700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f8b5006e790 0x7f8b50070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.732+0000 7f8b6bfff700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f8b5006e790 0x7f8b50070ba0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.739 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.732+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== osd.5 v2:172.21.15.177:6800/103770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8b50071270 con 0x7f8b5006e790 2023-11-28T13:07:31.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.746+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 --> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8b6c068690 con 0x7f8b5006e790 2023-11-28T13:07:31.747 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b697fa700 1 -- 172.21.15.153:0/1945437594 <== osd.5 v2:172.21.15.177:6800/103770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8b6c068690 con 0x7f8b5006e790 2023-11-28T13:07:31.747 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:31.748 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:31.748 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:31.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f8b5006e790 msgr2=0x7f8b50070ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.177:6800/103770,v1:172.21.15.177:6801/103770] conn(0x7f8b5006e790 0x7f8b50070ba0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8b50068320 msgr2=0x7f8b5006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f8b50068320 0x7f8b5006a7d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b60006fd0 tx=0x7f8b60008040 comp rx=0 tx=0).stop 2023-11-28T13:07:31.749 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c0af430 msgr2=0x7f8b6c0c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 --2- 172.21.15.153:0/1945437594 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b6c0af430 0x7f8b6c0c5cd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400b370 tx=0x7f8b64004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b715c7700 1 -- 172.21.15.153:0/1945437594 reap_dead start 2023-11-28T13:07:31.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 shutdown_connections 2023-11-28T13:07:31.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 >> 172.21.15.153:0/1945437594 conn(0x7f8b6c06b380 msgr2=0x7f8b6c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 shutdown_connections 2023-11-28T13:07:31.750 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.747+0000 7f8b7302a700 1 -- 172.21.15.153:0/1945437594 wait complete. 2023-11-28T13:07:31.759 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T13:07:31.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.880+0000 7f5240ac3700 1 Processor -- start 2023-11-28T13:07:31.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.880+0000 7f5240ac3700 1 -- start start 2023-11-28T13:07:31.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f5240ac3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f523c06f850 0x7f523c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f5240ac3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c13ff00 con 0x7f523c0cd8c0 2023-11-28T13:07:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f5240ac3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c1404c0 con 0x7f523c140950 2023-11-28T13:07:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f5240ac3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523c140dc0 con 0x7f523c06f850 2023-11-28T13:07:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f523a59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f523c0cd8c0 0x7f523c0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33738/0 (socket says 172.21.15.153:33738) 2023-11-28T13:07:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f523a59c700 1 -- 172.21.15.153:0/105728668 learned_addr learned my addr 172.21.15.153:0/105728668 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f523ad9d700 1 --2- 172.21.15.153:0/105728668 >> v2:172.21.15.177:3300/0 conn(0x7f523c06f850 0x7f523c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f523959a700 1 -- 172.21.15.153:0/105728668 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741088309 0 0) 0x7f523c13ff00 con 0x7f523c0cd8c0 2023-11-28T13:07:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.881+0000 7f523959a700 1 -- 172.21.15.153:0/105728668 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52240035b0 con 0x7f523c0cd8c0 2023-11-28T13:07:31.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523ad9d700 1 -- 172.21.15.153:0/105728668 >> v1:172.21.15.177:6789/0 conn(0x7f523c140950 legacy=0x7f523c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523959a700 1 -- 172.21.15.153:0/105728668 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606097294 0 0) 0x7f523c1404c0 con 0x7f523c140950 2023-11-28T13:07:31.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523ad9d700 1 -- 172.21.15.153:0/105728668 >> v1:172.21.15.153:6789/0 conn(0x7f523c0cd8c0 legacy=0x7f523c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523ad9d700 1 -- 172.21.15.153:0/105728668 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c15aa80 con 0x7f523c06f850 2023-11-28T13:07:31.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523ad9d700 1 --2- 172.21.15.153:0/105728668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 0x7f523c06fcf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5234009d90 tx=0x7f523400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad749944ce255c0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523959a700 1 -- 172.21.15.153:0/105728668 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400e070 con 0x7f523c06f850 2023-11-28T13:07:31.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523959a700 1 -- 172.21.15.153:0/105728668 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523400bb20 con 0x7f523c06f850 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f523959a700 1 -- 172.21.15.153:0/105728668 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400bed0 con 0x7f523c06f850 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f5240ac3700 1 -- 172.21.15.153:0/105728668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 msgr2=0x7f523c06fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.882+0000 7f5240ac3700 1 --2- 172.21.15.153:0/105728668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 0x7f523c06fcf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5234009d90 tx=0x7f523400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.883+0000 7f5240ac3700 1 -- 172.21.15.153:0/105728668 shutdown_connections 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.883+0000 7f5240ac3700 1 --2- 172.21.15.153:0/105728668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 0x7f523c06fcf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.883+0000 7f5240ac3700 1 -- 172.21.15.153:0/105728668 >> 172.21.15.153:0/105728668 conn(0x7f523c06aed0 msgr2=0x7f523c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.883+0000 7f5240ac3700 1 -- 172.21.15.153:0/105728668 shutdown_connections 2023-11-28T13:07:31.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.883+0000 7f5240ac3700 1 -- 172.21.15.153:0/105728668 wait complete. 2023-11-28T13:07:31.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.884+0000 7f5240ac3700 1 Processor -- start 2023-11-28T13:07:31.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.884+0000 7f5240ac3700 1 -- start start 2023-11-28T13:07:31.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 0x7f523c1efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f523c0cd8c0 0x7f523c0b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523c140950 0x7f523c0b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c15ac70 con 0x7f523c0cd8c0 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c15af50 con 0x7f523c06f850 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c15b230 con 0x7f523c140950 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f523c0cd8c0 0x7f523c0b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f523c0cd8c0 0x7f523c0b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43814/0 (socket says 172.21.15.153:43814) 2023-11-28T13:07:31.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 -- 172.21.15.153:0/3588267354 learned_addr learned my addr 172.21.15.153:0/3588267354 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f523ad9d700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523c140950 0x7f523c0b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f523a59c700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 0x7f523c1efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 -- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523c140950 msgr2=0x7f523c0b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523c140950 0x7f523c0b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 -- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 msgr2=0x7f523c1efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 0x7f523c1efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 -- 172.21.15.153:0/3588267354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234009a70 con 0x7f523c0cd8c0 2023-11-28T13:07:31.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5239d9b700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f523c0cd8c0 0x7f523c0b24b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c520 tx=0x7f522c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f522affd700 1 -- 172.21.15.153:0/3588267354 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c0054e0 con 0x7f523c0cd8c0 2023-11-28T13:07:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f522affd700 1 -- 172.21.15.153:0/3588267354 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c003680 con 0x7f523c0cd8c0 2023-11-28T13:07:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c0b3310 con 0x7f523c0cd8c0 2023-11-28T13:07:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f522affd700 1 -- 172.21.15.153:0/3588267354 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c003a30 con 0x7f523c0cd8c0 2023-11-28T13:07:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.885+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c0b3040 con 0x7f523c0cd8c0 2023-11-28T13:07:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.886+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c0b35f0 con 0x7f523c0cd8c0 2023-11-28T13:07:31.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.887+0000 7f522affd700 1 -- 172.21.15.153:0/3588267354 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f522c019030 con 0x7f523c0cd8c0 2023-11-28T13:07:31.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.887+0000 7f522affd700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f521c0682c0 0x7f521c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:31.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.887+0000 7f522affd700 1 -- 172.21.15.153:0/3588267354 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f522c084cf0 con 0x7f523c0cd8c0 2023-11-28T13:07:31.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.887+0000 7f523a59c700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f521c0682c0 0x7f521c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:31.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.888+0000 7f523a59c700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f521c0682c0 0x7f521c06a770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5230001360 tx=0x7f5230009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:31.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:31.891+0000 7f522affd700 1 -- 172.21.15.153:0/3588267354 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f522c0517c0 con 0x7f523c0cd8c0 2023-11-28T13:07:32.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.074+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f523c068690 con 0x7f521c0682c0 2023-11-28T13:07:32.075 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.075+0000 7f522affd700 1 -- 172.21.15.153:0/3588267354 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f523c068690 con 0x7f521c0682c0 2023-11-28T13:07:32.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f521c0682c0 msgr2=0x7f521c06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f521c0682c0 0x7f521c06a770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5230001360 tx=0x7f5230009040 comp rx=0 tx=0).stop 2023-11-28T13:07:32.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f523c0cd8c0 msgr2=0x7f523c0b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f523c0cd8c0 0x7f523c0b24b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c520 tx=0x7f522c00c830 comp rx=0 tx=0).stop 2023-11-28T13:07:32.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 shutdown_connections 2023-11-28T13:07:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523c140950 0x7f523c0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523c06f850 0x7f523c1efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f521c0682c0 0x7f521c06a770 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 --2- 172.21.15.153:0/3588267354 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f523c0cd8c0 0x7f523c0b24b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 >> 172.21.15.153:0/3588267354 conn(0x7f523c06aed0 msgr2=0x7f523c0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 shutdown_connections 2023-11-28T13:07:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.077+0000 7f5240ac3700 1 -- 172.21.15.153:0/3588267354 wait complete. 2023-11-28T13:07:32.080 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T13:07:32.090 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T13:07:32.090 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:32.214 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.214+0000 7fa7609d4700 1 Processor -- start 2023-11-28T13:07:32.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.214+0000 7fa7609d4700 1 -- start start 2023-11-28T13:07:32.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa7609d4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa75c0b6ae0 0x7fa75c0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa7609d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 0x7fa75c0d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa7609d4700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c15af70 con 0x7fa75c15aba0 2023-11-28T13:07:32.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa7609d4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75c0cfec0 con 0x7fa75c0b6ae0 2023-11-28T13:07:32.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa7609d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0ccaa0 con 0x7fa75c0cfa20 2023-11-28T13:07:32.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75a59c700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7fa75c15aba0 0x7fa75c0b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33752/0 (socket says 172.21.15.153:33752) 2023-11-28T13:07:32.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75a59c700 1 -- 172.21.15.153:0/3669097246 learned_addr learned my addr 172.21.15.153:0/3669097246 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:32.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa759d9b700 1 --2- 172.21.15.153:0/3669097246 >> v2:172.21.15.177:3300/0 conn(0x7fa75c0b6ae0 0x7fa75c0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75ad9d700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 0x7fa75c0d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712215515 0 0) 0x7fa75c15af70 con 0x7fa75c15aba0 2023-11-28T13:07:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7440035b0 con 0x7fa75c15aba0 2023-11-28T13:07:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3649268005 0 0) 0x7fa7440035b0 con 0x7fa75c15aba0 2023-11-28T13:07:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 msgr2=0x7fa75c0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 0x7fa75c0d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 >> v2:172.21.15.177:3300/0 conn(0x7fa75c0b6ae0 msgr2=0x7fa75c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 --2- 172.21.15.153:0/3669097246 >> v2:172.21.15.177:3300/0 conn(0x7fa75c0b6ae0 0x7fa75c0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0cc870 con 0x7fa75c15aba0 2023-11-28T13:07:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7fa74c001670 con 0x7fa75c15aba0 2023-11-28T13:07:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 >> v1:172.21.15.153:6789/0 conn(0x7fa75c15aba0 legacy=0x7fa75c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 0x7fa744003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa744003f80 0x7fa744004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa744004d90 0x7fa744005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0cfec0 con 0x7fa744003f80 2023-11-28T13:07:32.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0ccaa0 con 0x7fa744004d90 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.215+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74400ba10 con 0x7fa75c0cfa20 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa759d9b700 1 --2- 172.21.15.153:0/3669097246 >> v2:172.21.15.177:3300/0 conn(0x7fa75c0b6ae0 0x7fa75c0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75a59c700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 0x7fa744003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75ad9d700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa744004d90 0x7fa744005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa759d9b700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa744003f80 0x7fa744004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75a59c700 1 -- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa744004d90 msgr2=0x7fa744005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75a59c700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa744004d90 0x7fa744005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.222 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75a59c700 1 -- 172.21.15.153:0/3669097246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa744003f80 msgr2=0x7fa744004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75a59c700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa744003f80 0x7fa744004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75a59c700 1 -- 172.21.15.153:0/3669097246 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7440035b0 con 0x7fa75c0cfa20 2023-11-28T13:07:32.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa759d9b700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa744003f80 0x7fa744004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:32.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75ad9d700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa744004d90 0x7fa744005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:32.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75a59c700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 0x7fa744003a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00c320 tx=0x7fa744005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e927a82330844469 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00ce60 con 0x7fa75c0cfa20 2023-11-28T13:07:32.223 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c005960 con 0x7fa75c0cfa20 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.216+0000 7fa75959a700 1 -- 172.21.15.153:0/3669097246 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c005cf0 con 0x7fa75c0cfa20 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.217+0000 7fa7609d4700 1 -- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 msgr2=0x7fa744003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.217+0000 7fa7609d4700 1 --2- 172.21.15.153:0/3669097246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c0cfa20 0x7fa744003a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00c320 tx=0x7fa744005880 comp rx=0 tx=0).stop 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.217+0000 7fa75ad9d700 1 -- 172.21.15.153:0/3669097246 reap_dead start 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.217+0000 7fa7609d4700 1 -- 172.21.15.153:0/3669097246 shutdown_connections 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.217+0000 7fa7609d4700 1 -- 172.21.15.153:0/3669097246 >> 172.21.15.153:0/3669097246 conn(0x7fa75c06df70 msgr2=0x7fa75c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.217+0000 7fa7609d4700 1 -- 172.21.15.153:0/3669097246 shutdown_connections 2023-11-28T13:07:32.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.217+0000 7fa7609d4700 1 -- 172.21.15.153:0/3669097246 wait complete. 2023-11-28T13:07:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.218+0000 7fa7609d4700 1 Processor -- start 2023-11-28T13:07:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.218+0000 7fa7609d4700 1 -- start start 2023-11-28T13:07:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa7609d4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa75c0cfa20 0x7fa75c152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa7609d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c159440 0x7fa75c0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa7609d4700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa75c06b380 0x7fa75c06b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa7609d4700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0c5ea0 con 0x7fa75c06b380 2023-11-28T13:07:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa7609d4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1529d0 con 0x7fa75c0cfa20 2023-11-28T13:07:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa7609d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c06bd40 con 0x7fa75c159440 2023-11-28T13:07:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa75c06b380 0x7fa75c06b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa75c06b380 0x7fa75c06b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43836/0 (socket says 172.21.15.153:43836) 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 -- 172.21.15.153:0/252838757 learned_addr learned my addr 172.21.15.153:0/252838757 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75a59c700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa75c0cfa20 0x7fa75c152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa759d9b700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c159440 0x7fa75c0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 -- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c159440 msgr2=0x7fa75c0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa75c159440 0x7fa75c0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 -- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa75c0cfa20 msgr2=0x7fa75c152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa75c0cfa20 0x7fa75c152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.219+0000 7fa75ad9d700 1 -- 172.21.15.153:0/252838757 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0cd060 con 0x7fa75c06b380 2023-11-28T13:07:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.220+0000 7fa75a59c700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa75c0cfa20 0x7fa75c152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.220+0000 7fa75ad9d700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa75c06b380 0x7fa75c06b800 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0b6940 tx=0x7fa754015680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.220+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754017040 con 0x7fa75c06b380 2023-11-28T13:07:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.220+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa754015e50 con 0x7fa75c06b380 2023-11-28T13:07:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.220+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c0ccd80 con 0x7fa75c06b380 2023-11-28T13:07:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.220+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754003c60 con 0x7fa75c06b380 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.220+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c0cd340 con 0x7fa75c06b380 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.221+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa75c068690 con 0x7fa75c06b380 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.221+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa754024410 con 0x7fa75c06b380 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa74affd700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa73c0682f0 0x7fa73c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa754089bd0 con 0x7fa75c06b380 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa75a59c700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa73c0682f0 0x7fa73c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa74affd700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7fa73c06e760 0x7fa73c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.231 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 --> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa73c071240 con 0x7fa73c06e760 2023-11-28T13:07:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa75408e2a0 con 0x7fa75c06b380 2023-11-28T13:07:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa75a59c700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa73c0682f0 0x7fa73c06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa74c000c00 tx=0x7fa74c00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.222+0000 7fa759d9b700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7fa73c06e760 0x7fa73c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.223+0000 7fa759d9b700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7fa73c06e760 0x7fa73c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.223+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== osd.6 v2:172.21.15.177:6824/103787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa73c071240 con 0x7fa73c06e760 2023-11-28T13:07:32.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.237+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 --> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa75c0c5fe0 con 0x7fa73c06e760 2023-11-28T13:07:32.238 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa74affd700 1 -- 172.21.15.153:0/252838757 <== osd.6 v2:172.21.15.177:6824/103787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa75c0c5fe0 con 0x7fa73c06e760 2023-11-28T13:07:32.239 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:32.239 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:32.239 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:32.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7fa73c06e760 msgr2=0x7fa73c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.177:6824/103787,v1:172.21.15.177:6825/103787] conn(0x7fa73c06e760 0x7fa73c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.240 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa73c0682f0 msgr2=0x7fa73c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7fa73c0682f0 0x7fa73c06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa74c000c00 tx=0x7fa74c00a000 comp rx=0 tx=0).stop 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa75c06b380 msgr2=0x7fa75c06b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 --2- 172.21.15.153:0/252838757 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa75c06b380 0x7fa75c06b800 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0b6940 tx=0x7fa754015680 comp rx=0 tx=0).stop 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa75ad9d700 1 -- 172.21.15.153:0/252838757 reap_dead start 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 shutdown_connections 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 >> 172.21.15.153:0/252838757 conn(0x7fa75c06df70 msgr2=0x7fa75c06cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 shutdown_connections 2023-11-28T13:07:32.241 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.238+0000 7fa7609d4700 1 -- 172.21.15.153:0/252838757 wait complete. 2023-11-28T13:07:32.251 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T13:07:32.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.371+0000 7f2ffefce700 1 Processor -- start 2023-11-28T13:07:32.372 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.371+0000 7f2ffefce700 1 -- start start 2023-11-28T13:07:32.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ffefce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8071040 0x7f2ff8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ffefce700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ff806dee0 0x7f2ff806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ffefce700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2ff806cdc0 0x7f2ff806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ffefce700 1 -- --> v2:172.21.15.153:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff8071950 con 0x7f2ff806dee0 2023-11-28T13:07:32.375 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ffefce700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff815d060 con 0x7f2ff806cdc0 2023-11-28T13:07:32.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ffefce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff815d340 con 0x7f2ff8071040 2023-11-28T13:07:32.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ff7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ff806dee0 0x7f2ff806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ff7fff700 1 --2- >> v2:172.21.15.153:3300/0 conn(0x7f2ff806dee0 0x7f2ff806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43844/0 (socket says 172.21.15.153:43844) 2023-11-28T13:07:32.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.372+0000 7f2ff7fff700 1 -- 172.21.15.153:0/3378743804 learned_addr learned my addr 172.21.15.153:0/3378743804 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:32.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ffd56b700 1 --2- 172.21.15.153:0/3378743804 >> v2:172.21.15.177:3300/0 conn(0x7f2ff806cdc0 0x7f2ff806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff7fff700 1 -- 172.21.15.153:0/3378743804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8071040 msgr2=0x7f2ff8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.376 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ffcd6a700 1 --2- 172.21.15.153:0/3378743804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8071040 0x7f2ff8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff7fff700 1 --2- 172.21.15.153:0/3378743804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8071040 0x7f2ff8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff7fff700 1 -- 172.21.15.153:0/3378743804 >> v2:172.21.15.177:3300/0 conn(0x7f2ff806cdc0 msgr2=0x7f2ff806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff7fff700 1 --2- 172.21.15.153:0/3378743804 >> v2:172.21.15.177:3300/0 conn(0x7f2ff806cdc0 0x7f2ff806d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff7fff700 1 -- 172.21.15.153:0/3378743804 --> v2:172.21.15.153:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff815d620 con 0x7f2ff806dee0 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff7fff700 1 --2- 172.21.15.153:0/3378743804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff806dee0 0x7f2ff806e350 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8071d40 tx=0x7f2fe4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567865da9aed9ff4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff77fe700 1 -- 172.21.15.153:0/3378743804 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400b870 con 0x7f2ff806dee0 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff77fe700 1 -- 172.21.15.153:0/3378743804 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe400b9d0 con 0x7f2ff806dee0 2023-11-28T13:07:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ff77fe700 1 -- 172.21.15.153:0/3378743804 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400bd60 con 0x7f2ff806dee0 2023-11-28T13:07:32.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ffefce700 1 -- 172.21.15.153:0/3378743804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff806dee0 msgr2=0x7f2ff806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.373+0000 7f2ffefce700 1 --2- 172.21.15.153:0/3378743804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff806dee0 0x7f2ff806e350 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8071d40 tx=0x7f2fe4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.374+0000 7f2ffefce700 1 -- 172.21.15.153:0/3378743804 shutdown_connections 2023-11-28T13:07:32.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.374+0000 7f2ffefce700 1 --2- 172.21.15.153:0/3378743804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8071040 0x7f2ff8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.374+0000 7f2ffefce700 1 --2- 172.21.15.153:0/3378743804 >> v2:172.21.15.177:3300/0 conn(0x7f2ff806cdc0 0x7f2ff806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.374+0000 7f2ffefce700 1 --2- 172.21.15.153:0/3378743804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff806dee0 0x7f2ff806e350 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.378 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.374+0000 7f2ffefce700 1 -- 172.21.15.153:0/3378743804 >> 172.21.15.153:0/3378743804 conn(0x7f2ff80d0000 msgr2=0x7f2ff80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.374+0000 7f2ffefce700 1 -- 172.21.15.153:0/3378743804 shutdown_connections 2023-11-28T13:07:32.379 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.374+0000 7f2ffefce700 1 -- 172.21.15.153:0/3378743804 wait complete. 2023-11-28T13:07:32.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.375+0000 7f2ffefce700 1 Processor -- start 2023-11-28T13:07:32.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.375+0000 7f2ffefce700 1 -- start start 2023-11-28T13:07:32.381 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffefce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff806cdc0 0x7f2ff8159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffefce700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff806dee0 0x7f2ff8150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffefce700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff8071040 0x7f2ff80ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffefce700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c55b0 con 0x7f2ff8071040 2023-11-28T13:07:32.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffefce700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c5890 con 0x7f2ff806dee0 2023-11-28T13:07:32.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffefce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c5b70 con 0x7f2ff806cdc0 2023-11-28T13:07:32.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff8071040 0x7f2ff80ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.382 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff8071040 0x7f2ff80ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43852/0 (socket says 172.21.15.153:43852) 2023-11-28T13:07:32.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 -- 172.21.15.153:0/792585181 learned_addr learned my addr 172.21.15.153:0/792585181 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:32.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ff7fff700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff806dee0 0x7f2ff8150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffcd6a700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff806cdc0 0x7f2ff8159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 -- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff806cdc0 msgr2=0x7f2ff8159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff806cdc0 0x7f2ff8159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 -- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff806dee0 msgr2=0x7f2ff8150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.383 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff806dee0 0x7f2ff8150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 -- 172.21.15.153:0/792585181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4009550 con 0x7f2ff8071040 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ff7fff700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff806dee0 0x7f2ff8150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffcd6a700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff806cdc0 0x7f2ff8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.376+0000 7f2ffd56b700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff8071040 0x7f2ff80ade40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000cb80 tx=0x7f2ff000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.377+0000 7f2ff57fa700 1 -- 172.21.15.153:0/792585181 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000a8e0 con 0x7f2ff8071040 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.377+0000 7f2ff57fa700 1 -- 172.21.15.153:0/792585181 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff000aa40 con 0x7f2ff8071040 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.377+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff806e890 con 0x7f2ff8071040 2023-11-28T13:07:32.384 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.377+0000 7f2ff57fa700 1 -- 172.21.15.153:0/792585181 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000ae30 con 0x7f2ff8071040 2023-11-28T13:07:32.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.377+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff815d900 con 0x7f2ff8071040 2023-11-28T13:07:32.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.378+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff815ccd0 con 0x7f2ff8071040 2023-11-28T13:07:32.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.378+0000 7f2ff57fa700 1 -- 172.21.15.153:0/792585181 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ff0021070 con 0x7f2ff8071040 2023-11-28T13:07:32.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.378+0000 7f2ff57fa700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.378+0000 7f2ffcd6a700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.379+0000 7f2ff57fa700 1 -- 172.21.15.153:0/792585181 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2ff0085a10 con 0x7f2ff8071040 2023-11-28T13:07:32.385 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.379+0000 7f2ffcd6a700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00a9b0 tx=0x7f2fec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.383+0000 7f2ff57fa700 1 -- 172.21.15.153:0/792585181 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ff00524a0 con 0x7f2ff8071040 2023-11-28T13:07:32.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.571+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ff80720a0 con 0x7f2fdc0682f0 2023-11-28T13:07:32.572 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.571+0000 7f2ff57fa700 1 -- 172.21.15.153:0/792585181 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2ff80720a0 con 0x7f2fdc0682f0 2023-11-28T13:07:32.574 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2fdc0682f0 msgr2=0x7f2fdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00a9b0 tx=0x7f2fec008040 comp rx=0 tx=0).stop 2023-11-28T13:07:32.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff8071040 msgr2=0x7f2ff80ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff8071040 0x7f2ff80ade40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000cb80 tx=0x7f2ff000cf40 comp rx=0 tx=0).stop 2023-11-28T13:07:32.575 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 shutdown_connections 2023-11-28T13:07:32.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2ff8071040 0x7f2ff80ade40 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff806cdc0 0x7f2ff8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 --2- 172.21.15.153:0/792585181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff806dee0 0x7f2ff8150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.576 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.573+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 >> 172.21.15.153:0/792585181 conn(0x7f2ff80d0000 msgr2=0x7f2ff806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.574+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 shutdown_connections 2023-11-28T13:07:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.574+0000 7f2ffefce700 1 -- 172.21.15.153:0/792585181 wait complete. 2023-11-28T13:07:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T13:07:32.586 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T13:07:32.587 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.711+0000 7f91c84b3700 1 Processor -- start 2023-11-28T13:07:32.712 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.711+0000 7f91c84b3700 1 -- start start 2023-11-28T13:07:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.712+0000 7f91c84b3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f91c00b4b80 0x7f91c006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.718 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.712+0000 7f91c84b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 0x7f91c006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.712+0000 7f91c84b3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c006a270 con 0x7f91c015a880 2023-11-28T13:07:32.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.712+0000 7f91c84b3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c00b5040 con 0x7f91c00b4b80 2023-11-28T13:07:32.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.712+0000 7f91c84b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00b5320 con 0x7f91c015ac50 2023-11-28T13:07:32.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.712+0000 7f91c624f700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f91c015a880 0x7f91c00b4470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33788/0 (socket says 172.21.15.153:33788) 2023-11-28T13:07:32.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.712+0000 7f91c624f700 1 -- 172.21.15.153:0/749872153 learned_addr learned my addr 172.21.15.153:0/749872153 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:32.719 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c6a50700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 0x7f91c006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c5a4e700 1 --2- 172.21.15.153:0/749872153 >> v2:172.21.15.177:3300/0 conn(0x7f91c00b4b80 0x7f91c006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492424161 0 0) 0x7f91c006a270 con 0x7f91c015a880 2023-11-28T13:07:32.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ac0035b0 con 0x7f91c015a880 2023-11-28T13:07:32.720 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988525962 0 0) 0x7f91ac0035b0 con 0x7f91c015a880 2023-11-28T13:07:32.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 msgr2=0x7f91c006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 0x7f91c006cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 >> v2:172.21.15.177:3300/0 conn(0x7f91c00b4b80 msgr2=0x7f91c006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 --2- 172.21.15.153:0/749872153 >> v2:172.21.15.177:3300/0 conn(0x7f91c00b4b80 0x7f91c006c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00d0020 con 0x7f91c015a880 2023-11-28T13:07:32.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f91b0001670 con 0x7f91c015a880 2023-11-28T13:07:32.721 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 >> v1:172.21.15.153:6789/0 conn(0x7f91c015a880 legacy=0x7f91c00b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 0x7f91ac003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac003f80 0x7f91ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91ac004d90 0x7f91ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00b5040 con 0x7f91ac004d90 2023-11-28T13:07:32.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00b5320 con 0x7f91ac003f80 2023-11-28T13:07:32.722 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac00ba40 con 0x7f91c015ac50 2023-11-28T13:07:32.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c6a50700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91ac004d90 0x7f91ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c5a4e700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac003f80 0x7f91ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c624f700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 0x7f91ac003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c6a50700 1 -- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 msgr2=0x7f91ac003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c6a50700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91c015ac50 0x7f91ac003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c6a50700 1 -- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac003f80 msgr2=0x7f91ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.723 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c6a50700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91ac003f80 0x7f91ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.713+0000 7f91c6a50700 1 -- 172.21.15.153:0/749872153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0035b0 con 0x7f91ac004d90 2023-11-28T13:07:32.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c6a50700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91ac004d90 0x7f91ac005200 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f91c00c6550 tx=0x7f91ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc24966d5b90f674 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b800eb70 con 0x7f91ac004d90 2023-11-28T13:07:32.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b800f040 con 0x7f91ac004d90 2023-11-28T13:07:32.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c524d700 1 -- 172.21.15.153:0/749872153 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8014480 con 0x7f91ac004d90 2023-11-28T13:07:32.724 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c84b3700 1 -- 172.21.15.153:0/749872153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91ac004d90 msgr2=0x7f91ac005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c84b3700 1 --2- 172.21.15.153:0/749872153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91ac004d90 0x7f91ac005200 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f91c00c6550 tx=0x7f91ac005880 comp rx=0 tx=0).stop 2023-11-28T13:07:32.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c6a50700 1 -- 172.21.15.153:0/749872153 reap_dead start 2023-11-28T13:07:32.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c84b3700 1 -- 172.21.15.153:0/749872153 shutdown_connections 2023-11-28T13:07:32.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c84b3700 1 -- 172.21.15.153:0/749872153 >> 172.21.15.153:0/749872153 conn(0x7f91c00b3080 msgr2=0x7f91c00cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c84b3700 1 -- 172.21.15.153:0/749872153 shutdown_connections 2023-11-28T13:07:32.725 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.714+0000 7f91c84b3700 1 -- 172.21.15.153:0/749872153 wait complete. 2023-11-28T13:07:32.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.715+0000 7f91c84b3700 1 Processor -- start 2023-11-28T13:07:32.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c84b3700 1 -- start start 2023-11-28T13:07:32.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c84b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91ac004d90 0x7f91c01554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c84b3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91c00ad730 0x7f91c00c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c84b3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91c00c8850 0x7f91c01f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c84b3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00c8ca0 con 0x7f91c00c8850 2023-11-28T13:07:32.726 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c84b3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c01559f0 con 0x7f91c00ad730 2023-11-28T13:07:32.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c84b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c01551d0 con 0x7f91ac004d90 2023-11-28T13:07:32.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c6a50700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91c00c8850 0x7f91c01f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c6a50700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91c00c8850 0x7f91c01f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43872/0 (socket says 172.21.15.153:43872) 2023-11-28T13:07:32.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c6a50700 1 -- 172.21.15.153:0/2708916637 learned_addr learned my addr 172.21.15.153:0/2708916637 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:32.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c624f700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91ac004d90 0x7f91c01554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.716+0000 7f91c5a4e700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91c00ad730 0x7f91c00c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.727 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c6a50700 1 -- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91ac004d90 msgr2=0x7f91c01554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c6a50700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91ac004d90 0x7f91c01554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c6a50700 1 -- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91c00ad730 msgr2=0x7f91c00c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c6a50700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91c00ad730 0x7f91c00c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c6a50700 1 -- 172.21.15.153:0/2708916637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00aec90 con 0x7f91c00c8850 2023-11-28T13:07:32.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c6a50700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91c00c8850 0x7f91c01f3bd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f91ac004770 tx=0x7f91b8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8007000 con 0x7f91c00c8850 2023-11-28T13:07:32.728 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b8014c90 con 0x7f91c00c8850 2023-11-28T13:07:32.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8003a20 con 0x7f91c00c8850 2023-11-28T13:07:32.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c00d0300 con 0x7f91c00c8850 2023-11-28T13:07:32.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.717+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c0065ef0 con 0x7f91c00c8850 2023-11-28T13:07:32.729 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.718+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91c00c9840 con 0x7f91c00c8850 2023-11-28T13:07:32.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.718+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91b802c070 con 0x7f91c00c8850 2023-11-28T13:07:32.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91b6ffd700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91a40682f0 0x7f91a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f91b8086bd0 con 0x7f91c00c8850 2023-11-28T13:07:32.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91c624f700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91a40682f0 0x7f91a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.731 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91b6ffd700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f91a406e760 0x7f91a4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 --> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91a4071240 con 0x7f91a406e760 2023-11-28T13:07:32.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91b808b2a0 con 0x7f91c00c8850 2023-11-28T13:07:32.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91c5a4e700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f91a406e760 0x7f91a4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.719+0000 7f91c624f700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91a40682f0 0x7f91a406a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f91c006db00 tx=0x7f91b0007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.720+0000 7f91c5a4e700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f91a406e760 0x7f91a4070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.732 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.720+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== osd.7 v2:172.21.15.177:6808/103771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f91a4071240 con 0x7f91a406e760 2023-11-28T13:07:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.734+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 --> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f91c00c8de0 con 0x7f91a406e760 2023-11-28T13:07:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91b6ffd700 1 -- 172.21.15.153:0/2708916637 <== osd.7 v2:172.21.15.177:6808/103771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f91c00c8de0 con 0x7f91a406e760 2023-11-28T13:07:32.736 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2023-11-28T13:07:32.736 INFO:teuthology.orchestra.run.smithi153.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:32.736 INFO:teuthology.orchestra.run.smithi153.stdout:} 2023-11-28T13:07:32.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f91a406e760 msgr2=0x7f91a4070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.177:6808/103771,v1:172.21.15.177:6810/103771] conn(0x7f91a406e760 0x7f91a4070b70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91a40682f0 msgr2=0x7f91a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f91a40682f0 0x7f91a406a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f91c006db00 tx=0x7f91b0007980 comp rx=0 tx=0).stop 2023-11-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91c00c8850 msgr2=0x7f91c01f3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 --2- 172.21.15.153:0/2708916637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91c00c8850 0x7f91c01f3bd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f91ac004770 tx=0x7f91b8004b40 comp rx=0 tx=0).stop 2023-11-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c6a50700 1 -- 172.21.15.153:0/2708916637 reap_dead start 2023-11-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 shutdown_connections 2023-11-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 >> 172.21.15.153:0/2708916637 conn(0x7f91c00b3080 msgr2=0x7f91c00acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 shutdown_connections 2023-11-28T13:07:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.735+0000 7f91c84b3700 1 -- 172.21.15.153:0/2708916637 wait complete. 2023-11-28T13:07:32.747 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T13:07:32.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.868+0000 7f550a71e700 1 Processor -- start 2023-11-28T13:07:32.869 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.868+0000 7f550a71e700 1 -- start start 2023-11-28T13:07:32.873 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.869+0000 7f550a71e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 0x7f5504167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.869+0000 7f550a71e700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550415d060 con 0x7f5504071040 2023-11-28T13:07:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.869+0000 7f550a71e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550415d340 con 0x7f550406dee0 2023-11-28T13:07:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.869+0000 7f550a71e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5504161550 con 0x7f55041658c0 2023-11-28T13:07:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.869+0000 7f5503fff700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f5504071040 0x7f5504071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33812/0 (socket says 172.21.15.153:33812) 2023-11-28T13:07:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.869+0000 7f5503fff700 1 -- 172.21.15.153:0/513569453 learned_addr learned my addr 172.21.15.153:0/513569453 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291716932 0 0) 0x7f550415d060 con 0x7f5504071040 2023-11-28T13:07:32.874 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5508cbb700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 0x7f5504167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ec0035b0 con 0x7f5504071040 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2671154295 0 0) 0x7f54ec0035b0 con 0x7f5504071040 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 msgr2=0x7f5504167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 0x7f5504167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 >> v1:172.21.15.177:6789/0 conn(0x7f550406dee0 legacy=0x7f550406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550415d620 con 0x7f5504071040 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f54f8001670 con 0x7f5504071040 2023-11-28T13:07:32.875 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 >> v1:172.21.15.153:6789/0 conn(0x7f5504071040 legacy=0x7f5504071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54ec003a30 0x7f54ec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 0x7f54ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec004d90 0x7f54ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504161550 con 0x7f54ec004d90 2023-11-28T13:07:32.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec00bbd0 con 0x7f54ec003a30 2023-11-28T13:07:32.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec00bd90 con 0x7f55041658c0 2023-11-28T13:07:32.878 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5508cbb700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec004d90 0x7f54ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5503fff700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54ec003a30 0x7f54ec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f55037fe700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 0x7f54ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5508cbb700 1 -- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 msgr2=0x7f54ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5508cbb700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55041658c0 0x7f54ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5508cbb700 1 -- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54ec003a30 msgr2=0x7f54ec003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5508cbb700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54ec003a30 0x7f54ec003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.870+0000 7f5508cbb700 1 -- 172.21.15.153:0/513569453 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0035b0 con 0x7f54ec004d90 2023-11-28T13:07:32.879 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f5508cbb700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec004d90 0x7f54ec005200 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f55040c3a30 tx=0x7f54ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae3ef93297c7ad8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00eba0 con 0x7f54ec004d90 2023-11-28T13:07:32.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc00f040 con 0x7f54ec004d90 2023-11-28T13:07:32.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f5502ffd700 1 -- 172.21.15.153:0/513569453 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc014480 con 0x7f54ec004d90 2023-11-28T13:07:32.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f550a71e700 1 -- 172.21.15.153:0/513569453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec004d90 msgr2=0x7f54ec005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f550a71e700 1 --2- 172.21.15.153:0/513569453 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec004d90 0x7f54ec005200 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f55040c3a30 tx=0x7f54ec005880 comp rx=0 tx=0).stop 2023-11-28T13:07:32.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f5508cbb700 1 -- 172.21.15.153:0/513569453 reap_dead start 2023-11-28T13:07:32.880 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f550a71e700 1 -- 172.21.15.153:0/513569453 shutdown_connections 2023-11-28T13:07:32.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f550a71e700 1 -- 172.21.15.153:0/513569453 >> 172.21.15.153:0/513569453 conn(0x7f55040d0000 msgr2=0x7f55040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:32.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f550a71e700 1 -- 172.21.15.153:0/513569453 shutdown_connections 2023-11-28T13:07:32.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.871+0000 7f550a71e700 1 -- 172.21.15.153:0/513569453 wait complete. 2023-11-28T13:07:32.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.872+0000 7f550a71e700 1 Processor -- start 2023-11-28T13:07:32.881 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.873+0000 7f550a71e700 1 -- start start 2023-11-28T13:07:32.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.873+0000 7f550a71e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55041f5600 0x7f55040b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.873+0000 7f550a71e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55040b11a0 0x7f55040afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.873+0000 7f550a71e700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55040b5090 0x7f55040b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.873+0000 7f550a71e700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040b0d10 con 0x7f55040b5090 2023-11-28T13:07:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.873+0000 7f550a71e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55041563f0 con 0x7f55041f5600 2023-11-28T13:07:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.873+0000 7f550a71e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55041f59d0 con 0x7f55040b11a0 2023-11-28T13:07:32.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55040b5090 0x7f55040b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55040b5090 0x7f55040b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43894/0 (socket says 172.21.15.153:43894) 2023-11-28T13:07:32.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 -- 172.21.15.153:0/961082161 learned_addr learned my addr 172.21.15.153:0/961082161 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:32.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f55037fe700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55040b11a0 0x7f55040afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5503fff700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55041f5600 0x7f55040b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 -- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55040b11a0 msgr2=0x7f55040afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55040b11a0 0x7f55040afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 -- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55041f5600 msgr2=0x7f55040b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55041f5600 0x7f55040b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 -- 172.21.15.153:0/961082161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550415ccd0 con 0x7f55040b5090 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5508cbb700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55040b5090 0x7f55040b5530 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f54fc002230 tx=0x7f54fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5500ff9700 1 -- 172.21.15.153:0/961082161 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc007000 con 0x7f55040b5090 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5500ff9700 1 -- 172.21.15.153:0/961082161 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc003c40 con 0x7f55040b5090 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550415d900 con 0x7f55040b5090 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f5500ff9700 1 -- 172.21.15.153:0/961082161 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc009410 con 0x7f55040b5090 2023-11-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.874+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550406a270 con 0x7f55040b5090 2023-11-28T13:07:32.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.875+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55040c6e90 con 0x7f55040b5090 2023-11-28T13:07:32.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.875+0000 7f5500ff9700 1 -- 172.21.15.153:0/961082161 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f54fc0149c0 con 0x7f55040b5090 2023-11-28T13:07:32.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.876+0000 7f5500ff9700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f54f00682f0 0x7f54f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:32.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.876+0000 7f5500ff9700 1 -- 172.21.15.153:0/961082161 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f54fc087370 con 0x7f55040b5090 2023-11-28T13:07:32.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.876+0000 7f5503fff700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f54f00682f0 0x7f54f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:32.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.876+0000 7f5503fff700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f54f00682f0 0x7f54f006a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f550415b4a0 tx=0x7f54f8000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:32.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:32.880+0000 7f5500ff9700 1 -- 172.21.15.153:0/961082161 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f54fc053f90 con 0x7f55040b5090 2023-11-28T13:07:33.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.062+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f55040720a0 con 0x7f54f00682f0 2023-11-28T13:07:33.063 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.062+0000 7f5500ff9700 1 -- 172.21.15.153:0/961082161 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55040720a0 con 0x7f54f00682f0 2023-11-28T13:07:33.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f54f00682f0 msgr2=0x7f54f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.065 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f54f00682f0 0x7f54f006a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f550415b4a0 tx=0x7f54f8000bc0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55040b5090 msgr2=0x7f55040b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55040b5090 0x7f55040b5530 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f54fc002230 tx=0x7f54fc009750 comp rx=0 tx=0).stop 2023-11-28T13:07:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 shutdown_connections 2023-11-28T13:07:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f54f00682f0 0x7f54f006a7a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55040b5090 0x7f55040b5530 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55041f5600 0x7f55040b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.066 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 --2- 172.21.15.153:0/961082161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55040b11a0 0x7f55040afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 >> 172.21.15.153:0/961082161 conn(0x7f55040d0000 msgr2=0x7f550406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:33.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 shutdown_connections 2023-11-28T13:07:33.067 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.064+0000 7f550a71e700 1 -- 172.21.15.153:0/961082161 wait complete. 2023-11-28T13:07:33.067 INFO:teuthology.orchestra.run.smithi153.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T13:07:33.077 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.196+0000 7f2d7b91c700 1 Processor -- start 2023-11-28T13:07:33.197 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.196+0000 7f2d7b91c700 1 -- start start 2023-11-28T13:07:33.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d7b91c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 0x7f2d74071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.200 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d7b91c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2d74071580 0x7f2d7415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d7b91c700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d740719e0 con 0x7f2d74154e70 2023-11-28T13:07:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d7b91c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740ce050 con 0x7f2d74071580 2023-11-28T13:07:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d7b91c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74155240 con 0x7f2d740cdbd0 2023-11-28T13:07:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d796b8700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f2d74154e70 0x7f2d740cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33830/0 (socket says 172.21.15.153:33830) 2023-11-28T13:07:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d796b8700 1 -- 172.21.15.153:0/2741080132 learned_addr learned my addr 172.21.15.153:0/2741080132 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277638970 0 0) 0x7f2d740719e0 con 0x7f2d74154e70 2023-11-28T13:07:33.201 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d600035b0 con 0x7f2d74154e70 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d79eb9700 1 --2- 172.21.15.153:0/2741080132 >> v2:172.21.15.177:3300/0 conn(0x7f2d74071580 0x7f2d7415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d78eb7700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 0x7f2d74071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92664215 0 0) 0x7f2d600035b0 con 0x7f2d74154e70 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 msgr2=0x7f2d74071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 0x7f2d74071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 >> v2:172.21.15.177:3300/0 conn(0x7f2d74071580 msgr2=0x7f2d7415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 --2- 172.21.15.153:0/2741080132 >> v2:172.21.15.177:3300/0 conn(0x7f2d74071580 0x7f2d7415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.202 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7415b100 con 0x7f2d74154e70 2023-11-28T13:07:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f2d70001670 con 0x7f2d74154e70 2023-11-28T13:07:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 >> v1:172.21.15.153:6789/0 conn(0x7f2d74154e70 legacy=0x7f2d740cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d60003a00 0x7f2d60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.197+0000 7f2d6bfff700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 0x7f2d60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d6bfff700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d60004d90 0x7f2d60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74155240 con 0x7f2d60004d90 2023-11-28T13:07:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6000ba10 con 0x7f2d60003a00 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6000bb80 con 0x7f2d740cdbd0 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 --2- 172.21.15.153:0/2741080132 >> v2:172.21.15.177:3300/0 conn(0x7f2d74071580 0x7f2d7415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d60004d90 0x7f2d60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d78eb7700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 0x7f2d60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d796b8700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d60003a00 0x7f2d60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 -- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 msgr2=0x7f2d60004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740cdbd0 0x7f2d60004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.204 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 -- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d60003a00 msgr2=0x7f2d60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:33.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d60003a00 0x7f2d60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 -- 172.21.15.153:0/2741080132 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d600035b0 con 0x7f2d60004d90 2023-11-28T13:07:33.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d79eb9700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d60004d90 0x7f2d60005200 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0073e0 tx=0x7f2d60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fba5d8149cc8df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:33.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007ea0 con 0x7f2d60004d90 2023-11-28T13:07:33.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c014400 con 0x7f2d60004d90 2023-11-28T13:07:33.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.198+0000 7f2d6bfff700 1 -- 172.21.15.153:0/2741080132 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c014770 con 0x7f2d60004d90 2023-11-28T13:07:33.205 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.199+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2741080132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d60004d90 msgr2=0x7f2d60005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.199+0000 7f2d7b91c700 1 --2- 172.21.15.153:0/2741080132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d60004d90 0x7f2d60005200 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0073e0 tx=0x7f2d60005880 comp rx=0 tx=0).stop 2023-11-28T13:07:33.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.199+0000 7f2d79eb9700 1 -- 172.21.15.153:0/2741080132 reap_dead start 2023-11-28T13:07:33.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.199+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2741080132 shutdown_connections 2023-11-28T13:07:33.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.199+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2741080132 >> 172.21.15.153:0/2741080132 conn(0x7f2d740d0000 msgr2=0x7f2d740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:33.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.199+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2741080132 shutdown_connections 2023-11-28T13:07:33.206 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.199+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2741080132 wait complete. 2023-11-28T13:07:33.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.200+0000 7f2d7b91c700 1 Processor -- start 2023-11-28T13:07:33.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.201+0000 7f2d7b91c700 1 -- start start 2023-11-28T13:07:33.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.201+0000 7f2d7b91c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d741541b0 0x7f2d7415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.201+0000 7f2d7b91c700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74154580 0x7f2d740c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.209 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.201+0000 7f2d7b91c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740c4530 0x7f2d740c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.201+0000 7f2d7b91c700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740b3c20 con 0x7f2d74154580 2023-11-28T13:07:33.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.201+0000 7f2d7b91c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740b3f00 con 0x7f2d741541b0 2023-11-28T13:07:33.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.201+0000 7f2d7b91c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740b41e0 con 0x7f2d740c4530 2023-11-28T13:07:33.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74154580 0x7f2d740c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74154580 0x7f2d740c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43912/0 (socket says 172.21.15.153:43912) 2023-11-28T13:07:33.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 -- 172.21.15.153:0/2127309467 learned_addr learned my addr 172.21.15.153:0/2127309467 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:33.210 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d79eb9700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740c4530 0x7f2d740c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d796b8700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d741541b0 0x7f2d7415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 -- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740c4530 msgr2=0x7f2d740c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740c4530 0x7f2d740c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 -- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d741541b0 msgr2=0x7f2d7415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d741541b0 0x7f2d7415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 -- 172.21.15.153:0/2127309467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7415b6c0 con 0x7f2d74154580 2023-11-28T13:07:33.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d78eb7700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74154580 0x7f2d740c3ff0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2d64011aa0 tx=0x7f2d64011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:33.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d69ffb700 1 -- 172.21.15.153:0/2127309467 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64011ed0 con 0x7f2d74154580 2023-11-28T13:07:33.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d69ffb700 1 -- 172.21.15.153:0/2127309467 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400f940 con 0x7f2d74154580 2023-11-28T13:07:33.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7415b3e0 con 0x7f2d74154580 2023-11-28T13:07:33.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d69ffb700 1 -- 172.21.15.153:0/2127309467 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400fcf0 con 0x7f2d74154580 2023-11-28T13:07:33.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.202+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7415b9a0 con 0x7f2d74154580 2023-11-28T13:07:33.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.203+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d74072260 con 0x7f2d74154580 2023-11-28T13:07:33.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.204+0000 7f2d69ffb700 1 -- 172.21.15.153:0/2127309467 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d64014440 con 0x7f2d74154580 2023-11-28T13:07:33.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.204+0000 7f2d69ffb700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2d580682f0 0x7f2d5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:33.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.204+0000 7f2d69ffb700 1 -- 172.21.15.153:0/2127309467 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2d640860d0 con 0x7f2d74154580 2023-11-28T13:07:33.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.204+0000 7f2d796b8700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2d580682f0 0x7f2d5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:33.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.205+0000 7f2d796b8700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2d580682f0 0x7f2d5806a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c00 tx=0x7f2d70007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:33.213 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.208+0000 7f2d69ffb700 1 -- 172.21.15.153:0/2127309467 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d64052c90 con 0x7f2d74154580 2023-11-28T13:07:33.386 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.385+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d7404e620 con 0x7f2d580682f0 2023-11-28T13:07:33.389 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.388+0000 7f2d69ffb700 1 -- 172.21.15.153:0/2127309467 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73208 (secure 0 0 0) 0x7f2d7404e620 con 0x7f2d580682f0 2023-11-28T13:07:33.390 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:33.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2d580682f0 msgr2=0x7f2d5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2d580682f0 0x7f2d5806a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c00 tx=0x7f2d70007980 comp rx=0 tx=0).stop 2023-11-28T13:07:33.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74154580 msgr2=0x7f2d740c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:33.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74154580 0x7f2d740c3ff0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2d64011aa0 tx=0x7f2d64011db0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.391 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 shutdown_connections 2023-11-28T13:07:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d740c4530 0x7f2d740c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d741541b0 0x7f2d7415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f2d580682f0 0x7f2d5806a7a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 --2- 172.21.15.153:0/2127309467 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74154580 0x7f2d740c3ff0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 >> 172.21.15.153:0/2127309467 conn(0x7f2d740d0000 msgr2=0x7f2d740ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 shutdown_connections 2023-11-28T13:07:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:33.390+0000 7f2d7b91c700 1 -- 172.21.15.153:0/2127309467 wait complete. 2023-11-28T13:07:33.393 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:07:33.403 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-11-28T13:07:32.254565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":84612,"kb_used_data":7556,"kb_used_omap":0,"kb_used_meta":76992,"kb_avail":754890108,"statfs":{"total":773094113280,"available":773007470592,"internally_reserved":0,"allocated":7737344,"data_stored":4064695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78839808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.545476"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:24.624425+0000","last_change":"2023-11-28T13:00:12.497209+0000","last_active":"2023-11-28T13:07:24.624425+0000","last_peered":"2023-11-28T13:07:24.624425+0000","last_clean":"2023-11-28T13:07:24.624425+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:07:24.624425+0000","last_undegraded":"2023-11-28T13:07:24.624425+0000","last_fullsized":"2023-11-28T13:07:24.624425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:23.712828+0000","last_change":"2023-11-28T13:00:12.204831+0000","last_active":"2023-11-28T13:07:23.712828+0000","last_peered":"2023-11-28T13:07:23.712828+0000","last_clean":"2023-11-28T13:07:23.712828+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:07:23.712828+0000","last_undegraded":"2023-11-28T13:07:23.712828+0000","last_fullsized":"2023-11-28T13:07:23.712828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:24.623913+0000","last_change":"2023-11-28T13:00:12.497101+0000","last_active":"2023-11-28T13:07:24.623913+0000","last_peered":"2023-11-28T13:07:24.623913+0000","last_clean":"2023-11-28T13:07:24.623913+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:07:24.623913+0000","last_undegraded":"2023-11-28T13:07:24.623913+0000","last_fullsized":"2023-11-28T13:07:24.623913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:22.971005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:23.712472+0000","last_change":"2023-11-28T13:00:12.204505+0000","last_active":"2023-11-28T13:07:23.712472+0000","last_peered":"2023-11-28T13:07:23.712472+0000","last_clean":"2023-11-28T13:07:23.712472+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:07:23.712472+0000","last_undegraded":"2023-11-28T13:07:23.712472+0000","last_fullsized":"2023-11-28T13:07:23.712472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:53:07.197959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:24.625926+0000","last_change":"2023-11-28T13:00:05.935941+0000","last_active":"2023-11-28T13:07:24.625926+0000","last_peered":"2023-11-28T13:07:24.625926+0000","last_clean":"2023-11-28T13:07:24.625926+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:07:24.625926+0000","last_undegraded":"2023-11-28T13:07:24.625926+0000","last_fullsized":"2023-11-28T13:07:24.625926+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:04.852268+0000","last_clean_scrub_stamp":"2023-11-28T13:00:04.852268+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:24.651651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:24.625832+0000","last_change":"2023-11-28T13:00:12.210924+0000","last_active":"2023-11-28T13:07:24.625832+0000","last_peered":"2023-11-28T13:07:24.625832+0000","last_clean":"2023-11-28T13:07:24.625832+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:07:24.625832+0000","last_undegraded":"2023-11-28T13:07:24.625832+0000","last_fullsized":"2023-11-28T13:07:24.625832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:57.373488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:24.627515+0000","last_change":"2023-11-28T13:00:12.497225+0000","last_active":"2023-11-28T13:07:24.627515+0000","last_peered":"2023-11-28T13:07:24.627515+0000","last_clean":"2023-11-28T13:07:24.627515+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:07:24.627515+0000","last_undegraded":"2023-11-28T13:07:24.627515+0000","last_fullsized":"2023-11-28T13:07:24.627515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:40:09.492407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:24.624005+0000","last_change":"2023-11-28T13:00:12.207557+0000","last_active":"2023-11-28T13:07:24.624005+0000","last_peered":"2023-11-28T13:07:24.624005+0000","last_clean":"2023-11-28T13:07:24.624005+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:07:24.624005+0000","last_undegraded":"2023-11-28T13:07:24.624005+0000","last_fullsized":"2023-11-28T13:07:24.624005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:44.363271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T13:07:23.715257+0000","last_change":"2023-11-28T13:00:12.211272+0000","last_active":"2023-11-28T13:07:23.715257+0000","last_peered":"2023-11-28T13:07:23.715257+0000","last_clean":"2023-11-28T13:07:23.715257+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:07:23.715257+0000","last_undegraded":"2023-11-28T13:07:23.715257+0000","last_fullsized":"2023-11-28T13:07:23.715257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:11.186253+0000","last_clean_scrub_stamp":"2023-11-28T13:00:11.186253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:18.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509534,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9960,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361880,"statfs":{"total":96636764160,"available":96626565120,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.372,"15min":0.371},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":0.497,"5min":0.57699999999999996,"15min":1.2669999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.57399999999999995},"min":{"1min":0.222,"5min":0.222,"15min":0.216},"max":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":8.218},"last":0.35299999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.26300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.623,"5min":0.63600000000000001,"15min":1.8280000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.41199999999999998},"min":{"1min":0.29699999999999999,"5min":0.218,"15min":0.214},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":1.355},"last":0.34000000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.76400000000000001},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":13.384},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.26500000000000001},"max":{"1min":0.60999999999999999,"5min":0.65200000000000002,"15min":5.226},"last":0.42199999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.35999999999999999,"15min":0.35699999999999998},"max":{"1min":0.81799999999999995,"5min":0.94499999999999995,"15min":8.1890000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.73399999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78900000000000003,"5min":0.93999999999999995,"15min":5.3609999999999998},"last":0.57899999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":1.3029999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.80400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.878,"5min":0.96899999999999997,"15min":8.1579999999999995},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":5.3209999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.3320000000000001},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.83099999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":8.1029999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.76000000000000001},"min":{"1min":0.50800000000000001,"5min":0.40300000000000002,"15min":0.38900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":5.3849999999999998},"last":0.55100000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9640,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362200,"statfs":{"total":96636764160,"available":96626892800,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35199999999999998,"15min":0.33000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":0.51700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.187},"max":{"1min":0.48599999999999999,"5min":0.61599999999999999,"15min":1.0169999999999999},"last":0.504}]},{"osd":1,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.224,"15min":0.20499999999999999},"max":{"1min":0.52600000000000002,"5min":0.62,"15min":0.95999999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.221},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":1.1459999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.39200000000000002,"15min":0.36599999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.63400000000000001,"15min":1.1659999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.57599999999999996,"5min":0.70299999999999996,"15min":1.1919999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.641},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.91100000000000003,"15min":1.056},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.89800000000000002,"15min":1.6120000000000001},"last":0.998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.34499999999999997},"max":{"1min":0.72599999999999998,"5min":0.82899999999999996,"15min":1.774},"last":0.79400000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.626,"15min":0.58199999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84599999999999997,"15min":1.242},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.92900000000000005,"15min":1.6539999999999999},"last":0.754}]},{"osd":7,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.876,"15min":1.6919999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.89300000000000002,"15min":1.2130000000000001},"last":0.86899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542238,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":9896,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361944,"statfs":{"total":96636764160,"available":96626630656,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.35999999999999999},"max":{"1min":0.75800000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.56499999999999995},"min":{"1min":0.442,"5min":0.41299999999999998,"15min":0.33600000000000002},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.442}]},{"osd":2,"last update":"Tue Nov 28 13:07:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":1.2270000000000001},"last":0.379},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.52700000000000002},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.745,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45100000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:07:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.86899999999999999,"15min":1.2110000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.55800000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.39000000000000001},"max":{"1min":0.77200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.76900000000000002,"15min":0.81200000000000006},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51000000000000001,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.36299999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.52900000000000003},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.247},"max":{"1min":0.69099999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.43099999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":0.83599999999999997,"15min":0.88700000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.42599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14952,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356888,"statfs":{"total":96636764160,"available":96621453312,"internally_reserved":0,"allocated":1409024,"data_stored":950190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.33800000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.214},"max":{"1min":0.52800000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.74199999999999999},"min":{"1min":0.34000000000000002,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":0.56200000000000006,"5min":0.61599999999999999,"15min":14.957000000000001},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.747},"min":{"1min":0.36499999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53200000000000003,"5min":0.60499999999999998,"15min":15.032999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.754},"min":{"1min":0.40699999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51900000000000002,"5min":0.59499999999999997,"15min":14.997999999999999},"last":0.54500000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41099999999999998,"15min":0.84699999999999998},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.21199999999999999},"max":{"1min":0.49399999999999999,"5min":0.56999999999999995,"15min":19.045999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.41099999999999998},"min":{"1min":0.40300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60699999999999998,"15min":0.67200000000000004},"min":{"1min":0.53700000000000003,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.1799999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":1.073},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.35299999999999998},"max":{"1min":0.754,"5min":0.89700000000000002,"15min":18.995000000000001},"last":0.878}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61299999999999999,"15min":1.0600000000000001},"min":{"1min":0.55200000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.93700000000000006,"15min":19.02},"last":0.747},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.628,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":4.2119999999999997},"last":0.86199999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63600000000000001,"15min":0.70199999999999996},"min":{"1min":0.57399999999999995,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":4.2359999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63500000000000001,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.92500000000000004,"15min":4.1150000000000002},"last":0.75800000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.70199999999999996},"min":{"1min":0.57999999999999996,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.876,"15min":4.1539999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64500000000000002,"15min":1.085},"min":{"1min":0.63300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":18.957999999999998},"last":0.89900000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542238,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11052,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360788,"statfs":{"total":96636764160,"available":96625446912,"internally_reserved":0,"allocated":1413120,"data_stored":950959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.78400000000000003,"15min":1.1699999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.81599999999999995,"15min":1.415},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.80600000000000005,"15min":1.867},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57299999999999995,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":0.70399999999999996,"5min":0.79700000000000004,"15min":4.7830000000000004},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.84499999999999997,"15min":1.9550000000000001},"last":0.621},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34000000000000002},"max":{"1min":0.68400000000000005,"5min":0.81499999999999995,"15min":1.9510000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.88600000000000001,"15min":1.6859999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73799999999999999,"5min":0.89900000000000002,"15min":4.8440000000000003},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.33300000000000002,"15min":0.28499999999999998},"max":{"1min":0.69999999999999996,"5min":0.873,"15min":2.6030000000000002},"last":0.435},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":1.528},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.311},"max":{"1min":0.64700000000000002,"5min":0.94699999999999995,"15min":1.25},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71599999999999997,"5min":0.93600000000000005,"15min":2.5329999999999999},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":0.65700000000000003,"5min":0.96599999999999997,"15min":1.972},"last":0.504},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.98899999999999999,"15min":2.508},"last":0.63500000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9768,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362072,"statfs":{"total":96636764160,"available":96626761728,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68999999999999995,"5min":0.82799999999999996,"15min":1.419},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.68100000000000005,"5min":0.78500000000000003,"15min":0.80000000000000004},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.343,"15min":1.343},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69499999999999995}]},{"osd":2,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70599999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.40799999999999997,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":3.6160000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.81100000000000005,"15min":1.4490000000000001},"last":0.626},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.74399999999999999,"5min":1.228,"15min":1.228},"last":0.59999999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46100000000000002,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.72399999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.38100000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":0.85299999999999998,"15min":0.89600000000000002},"last":0.74099999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9576,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362264,"statfs":{"total":96636764160,"available":96626958336,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.36199999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.75,"15min":0.75},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":1.0249999999999999},"last":0.432}]},{"osd":2,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.54800000000000004},"min":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.38},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49299999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.49099999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.76500000000000001,"5min":0.77700000000000002,"15min":0.79500000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78000000000000003},"last":0.501}]},{"osd":4,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.33100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.83399999999999996},"last":0.41699999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.40000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.44800000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52400000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509534,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9768,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362072,"statfs":{"total":96636764160,"available":96626761728,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.46300000000000002,"5min":0.60999999999999999,"15min":0.97099999999999997},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39300000000000002,"15min":0.35899999999999999},"min":{"1min":0.25800000000000001,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":0.55400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.30199999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.375,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52900000000000003,"5min":0.57499999999999996,"15min":0.72899999999999998},"last":0.224},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.41099999999999998,"15min":0.378},"min":{"1min":0.27500000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54500000000000004,"5min":0.60099999999999998,"15min":0.64300000000000002},"last":0.33400000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26500000000000001},"max":{"1min":0.67000000000000004,"5min":0.69199999999999995,"15min":0.752},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42699999999999999,"15min":0.40400000000000003},"min":{"1min":0.29399999999999998,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.875},"last":0.378}]},{"osd":4,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.58799999999999997},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.36299999999999999},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":1.044},"last":0.47299999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.47699999999999998,"5min":0.433,"15min":0.28999999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92600000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.36799999999999999},"max":{"1min":0.82199999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.46500000000000002,"15min":0.31900000000000001},"max":{"1min":0.83299999999999996,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.38900000000000001},"max":{"1min":0.90000000000000002,"5min":0.93899999999999995,"15min":1.089},"last":0.42199999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66700000000000004,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":1.202},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.63600000000000001},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":0.95799999999999996,"15min":1.02},"last":0.46700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:33.408 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.408 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.408 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.409 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.409 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T13:00:04.852268+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=4, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.409 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.409 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.410 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.410 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-28T13:00:11.186253+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=28, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:33.410 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T13:07:47.198 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:47.202 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:53.411 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:53.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.543+0000 7f53e557f700 1 Processor -- start 2023-11-28T13:07:53.545 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.543+0000 7f53e557f700 1 -- start start 2023-11-28T13:07:53.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53e557f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f53e00cd7c0 0x7f53e00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53e557f700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00d1950 con 0x7f53e0161340 2023-11-28T13:07:53.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53e557f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00ce140 con 0x7f53e015c9b0 2023-11-28T13:07:53.549 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53e557f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00b5800 con 0x7f53e00cd7c0 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53de7fc700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f53e0161340 0x7f53e00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33852/0 (socket says 172.21.15.153:33852) 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53de7fc700 1 -- 172.21.15.153:0/741368934 learned_addr learned my addr 172.21.15.153:0/741368934 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53df7fe700 1 --2- 172.21.15.153:0/741368934 >> v2:172.21.15.177:3300/0 conn(0x7f53e00cd7c0 0x7f53e00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.544+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947130747 0 0) 0x7f53e00d1950 con 0x7f53e0161340 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c80035b0 con 0x7f53e0161340 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1598271045 0 0) 0x7f53c80035b0 con 0x7f53e0161340 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 >> v2:172.21.15.177:3300/0 conn(0x7f53e00cd7c0 msgr2=0x7f53e00cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.550 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 --2- 172.21.15.153:0/741368934 >> v2:172.21.15.177:3300/0 conn(0x7f53e00cd7c0 0x7f53e00cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 >> v1:172.21.15.177:6789/0 conn(0x7f53e015c9b0 legacy=0x7f53e00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00d24f0 con 0x7f53e0161340 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f53d0001670 con 0x7f53e0161340 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 >> v1:172.21.15.153:6789/0 conn(0x7f53e0161340 legacy=0x7f53e00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c8004850 0x7f53c8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c8005200 0x7f53c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b5800 con 0x7f53c8003a00 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c800c110 con 0x7f53c8004850 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c800c2d0 con 0x7f53c8005200 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53deffd700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53de7fc700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c8004850 0x7f53c8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53df7fe700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c8005200 0x7f53c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53deffd700 1 -- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c8005200 msgr2=0x7f53c800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53deffd700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c8005200 0x7f53c800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53deffd700 1 -- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c8004850 msgr2=0x7f53c8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53deffd700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53c8004850 0x7f53c8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53deffd700 1 -- 172.21.15.153:0/741368934 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80035b0 con 0x7f53c8003a00 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53df7fe700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53c8005200 0x7f53c800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53df7fe700 1 -- 172.21.15.153:0/741368934 reap_dead start 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.545+0000 7f53deffd700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53c8003e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f53d40017c0 tx=0x7f53c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ddfca93e7995e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d40052d0 con 0x7f53c8003a00 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d4005430 con 0x7f53c8003a00 2023-11-28T13:07:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53ddffb700 1 -- 172.21.15.153:0/741368934 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d40038c0 con 0x7f53c8003a00 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53e557f700 1 -- 172.21.15.153:0/741368934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 msgr2=0x7f53c8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53e557f700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53c8003e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f53d40017c0 tx=0x7f53c8005880 comp rx=0 tx=0).stop 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53e557f700 1 -- 172.21.15.153:0/741368934 shutdown_connections 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53e557f700 1 --2- 172.21.15.153:0/741368934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53c8003e10 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53e557f700 1 -- 172.21.15.153:0/741368934 >> 172.21.15.153:0/741368934 conn(0x7f53e0072600 msgr2=0x7f53e00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53e557f700 1 -- 172.21.15.153:0/741368934 shutdown_connections 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.546+0000 7f53e557f700 1 -- 172.21.15.153:0/741368934 wait complete. 2023-11-28T13:07:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 Processor -- start 2023-11-28T13:07:53.555 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 -- start start 2023-11-28T13:07:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53e00ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cd330 0x7f53e015e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e01f1950 0x7f53e01f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c78e0 con 0x7f53c8003a00 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00cb9d0 con 0x7f53e00cd330 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53e557f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c7d00 con 0x7f53e01f1950 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53deffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53e00ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53deffd700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53e00ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43930/0 (socket says 172.21.15.153:43930) 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.548+0000 7f53deffd700 1 -- 172.21.15.153:0/2872511882 learned_addr learned my addr 172.21.15.153:0/2872511882 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:53.559 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53de7fc700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cd330 0x7f53e015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53df7fe700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e01f1950 0x7f53e01f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53deffd700 1 -- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e01f1950 msgr2=0x7f53e01f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53deffd700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e01f1950 0x7f53e01f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53deffd700 1 -- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cd330 msgr2=0x7f53e015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53deffd700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cd330 0x7f53e015e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53deffd700 1 -- 172.21.15.153:0/2872511882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00d1300 con 0x7f53c8003a00 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53deffd700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53e00ca240 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f53c800c0e0 tx=0x7f53d4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.560 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53c77fe700 1 -- 172.21.15.153:0/2872511882 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4001290 con 0x7f53c8003a00 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53c77fe700 1 -- 172.21.15.153:0/2872511882 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d401d480 con 0x7f53c8003a00 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e00b5520 con 0x7f53c8003a00 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53c77fe700 1 -- 172.21.15.153:0/2872511882 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d401d830 con 0x7f53c8003a00 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.549+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e00d15e0 con 0x7f53c8003a00 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.550+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e00745c0 con 0x7f53c8003a00 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.550+0000 7f53c77fe700 1 -- 172.21.15.153:0/2872511882 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f53d402f070 con 0x7f53c8003a00 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.551+0000 7f53c77fe700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f53cc0682f0 0x7f53cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.561 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.551+0000 7f53c77fe700 1 -- 172.21.15.153:0/2872511882 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f53d40866c0 con 0x7f53c8003a00 2023-11-28T13:07:53.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.551+0000 7f53de7fc700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f53cc0682f0 0x7f53cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.551+0000 7f53de7fc700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f53cc0682f0 0x7f53cc06a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f53e00cd740 tx=0x7f53d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.562 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.555+0000 7f53c77fe700 1 -- 172.21.15.153:0/2872511882 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f53d4053220 con 0x7f53c8003a00 2023-11-28T13:07:53.737 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.735+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 --> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53e00c79c0 con 0x7f53cc0682f0 2023-11-28T13:07:53.739 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.738+0000 7f53c77fe700 1 -- 172.21.15.153:0/2872511882 <== mgr.4107 v2:172.21.15.153:6832/103291 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73285 (secure 0 0 0) 0x7f53e00c79c0 con 0x7f53cc0682f0 2023-11-28T13:07:53.741 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:53.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f53cc0682f0 msgr2=0x7f53cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f53cc0682f0 0x7f53cc06a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f53e00cd740 tx=0x7f53d0009040 comp rx=0 tx=0).stop 2023-11-28T13:07:53.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 msgr2=0x7f53e00ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.742 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53e00ca240 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f53c800c0e0 tx=0x7f53d4001120 comp rx=0 tx=0).stop 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 shutdown_connections 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f53cc0682f0 0x7f53cc06a7a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f53c8003a00 0x7f53e00ca240 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e01f1950 0x7f53e01f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 --2- 172.21.15.153:0/2872511882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cd330 0x7f53e015e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 >> 172.21.15.153:0/2872511882 conn(0x7f53e0072600 msgr2=0x7f53e006b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 shutdown_connections 2023-11-28T13:07:53.743 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.740+0000 7f53e557f700 1 -- 172.21.15.153:0/2872511882 wait complete. 2023-11-28T13:07:53.744 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2023-11-28T13:07:53.754 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-11-28T13:07:52.258934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":84676,"kb_used_data":7556,"kb_used_omap":0,"kb_used_meta":77056,"kb_avail":754890044,"statfs":{"total":773094113280,"available":773007405056,"internally_reserved":0,"allocated":7737344,"data_stored":4064695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78905344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002634"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:30.481440+0000","last_change":"2023-11-28T13:07:30.481420+0000","last_active":"2023-11-28T13:07:30.481440+0000","last_peered":"2023-11-28T13:07:30.481440+0000","last_clean":"2023-11-28T13:07:30.481440+0000","last_became_active":"2023-11-28T13:00:12.496905+0000","last_became_peered":"2023-11-28T13:00:12.496905+0000","last_unstale":"2023-11-28T13:07:30.481440+0000","last_undegraded":"2023-11-28T13:07:30.481440+0000","last_fullsized":"2023-11-28T13:07:30.481440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:30.481344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:30.481344+0000","last_clean_scrub_stamp":"2023-11-28T13:07:30.481344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:17:49.461994+0000","scrub_duration":0.0034609829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":69,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:33.564741+0000","last_change":"2023-11-28T13:07:33.564710+0000","last_active":"2023-11-28T13:07:33.564741+0000","last_peered":"2023-11-28T13:07:33.564741+0000","last_clean":"2023-11-28T13:07:33.564741+0000","last_became_active":"2023-11-28T13:00:12.204447+0000","last_became_peered":"2023-11-28T13:00:12.204447+0000","last_unstale":"2023-11-28T13:07:33.564741+0000","last_undegraded":"2023-11-28T13:07:33.564741+0000","last_fullsized":"2023-11-28T13:07:33.564741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:33.564601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:33.564601+0000","last_clean_scrub_stamp":"2023-11-28T13:07:33.564601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:12.436841+0000","scrub_duration":0.0036082480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:32.058505+0000","last_change":"2023-11-28T13:07:32.058482+0000","last_active":"2023-11-28T13:07:32.058505+0000","last_peered":"2023-11-28T13:07:32.058505+0000","last_clean":"2023-11-28T13:07:32.058505+0000","last_became_active":"2023-11-28T13:00:12.496795+0000","last_became_peered":"2023-11-28T13:00:12.496795+0000","last_unstale":"2023-11-28T13:07:32.058505+0000","last_undegraded":"2023-11-28T13:07:32.058505+0000","last_fullsized":"2023-11-28T13:07:32.058505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:32.058394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:32.058394+0000","last_clean_scrub_stamp":"2023-11-28T13:07:32.058394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:26:20.303443+0000","scrub_duration":0.0034722590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":69,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:32.530995+0000","last_change":"2023-11-28T13:07:32.530959+0000","last_active":"2023-11-28T13:07:32.530995+0000","last_peered":"2023-11-28T13:07:32.530995+0000","last_clean":"2023-11-28T13:07:32.530995+0000","last_became_active":"2023-11-28T13:00:12.204164+0000","last_became_peered":"2023-11-28T13:00:12.204164+0000","last_unstale":"2023-11-28T13:07:32.530995+0000","last_undegraded":"2023-11-28T13:07:32.530995+0000","last_fullsized":"2023-11-28T13:07:32.530995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:32.530844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:32.530844+0000","last_clean_scrub_stamp":"2023-11-28T13:07:32.530844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:50.333993+0000","scrub_duration":0.0037194480000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"68'119","reported_seq":222,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:34.802043+0000","last_change":"2023-11-28T13:07:34.802000+0000","last_active":"2023-11-28T13:07:34.802043+0000","last_peered":"2023-11-28T13:07:34.802043+0000","last_clean":"2023-11-28T13:07:34.802043+0000","last_became_active":"2023-11-28T13:00:05.934963+0000","last_became_peered":"2023-11-28T13:00:05.934963+0000","last_unstale":"2023-11-28T13:07:34.802043+0000","last_undegraded":"2023-11-28T13:07:34.802043+0000","last_fullsized":"2023-11-28T13:07:34.802043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'119","last_scrub_stamp":"2023-11-28T13:07:34.801894+0000","last_deep_scrub":"68'119","last_deep_scrub_stamp":"2023-11-28T13:07:34.801894+0000","last_clean_scrub_stamp":"2023-11-28T13:07:34.801894+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:46.989768+0000","scrub_duration":0.0072597720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":67,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:41.924090+0000","last_change":"2023-11-28T13:07:41.924046+0000","last_active":"2023-11-28T13:07:41.924090+0000","last_peered":"2023-11-28T13:07:41.924090+0000","last_clean":"2023-11-28T13:07:41.924090+0000","last_became_active":"2023-11-28T13:00:12.210638+0000","last_became_peered":"2023-11-28T13:00:12.210638+0000","last_unstale":"2023-11-28T13:07:41.924090+0000","last_undegraded":"2023-11-28T13:07:41.924090+0000","last_fullsized":"2023-11-28T13:07:41.924090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:41.923937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:41.923937+0000","last_clean_scrub_stamp":"2023-11-28T13:07:41.923937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:42.221508+0000","scrub_duration":0.0040871470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":67,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:36.869014+0000","last_change":"2023-11-28T13:07:36.868944+0000","last_active":"2023-11-28T13:07:36.869014+0000","last_peered":"2023-11-28T13:07:36.869014+0000","last_clean":"2023-11-28T13:07:36.869014+0000","last_became_active":"2023-11-28T13:00:12.496918+0000","last_became_peered":"2023-11-28T13:00:12.496918+0000","last_unstale":"2023-11-28T13:07:36.869014+0000","last_undegraded":"2023-11-28T13:07:36.869014+0000","last_fullsized":"2023-11-28T13:07:36.869014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:36.868715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:36.868715+0000","last_clean_scrub_stamp":"2023-11-28T13:07:36.868715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:38.781417+0000","scrub_duration":0.0039740899999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:33.513159+0000","last_change":"2023-11-28T13:07:33.513135+0000","last_active":"2023-11-28T13:07:33.513159+0000","last_peered":"2023-11-28T13:07:33.513159+0000","last_clean":"2023-11-28T13:07:33.513159+0000","last_became_active":"2023-11-28T13:00:12.207218+0000","last_became_peered":"2023-11-28T13:00:12.207218+0000","last_unstale":"2023-11-28T13:07:33.513159+0000","last_undegraded":"2023-11-28T13:07:33.513159+0000","last_fullsized":"2023-11-28T13:07:33.513159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:33.513053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:33.513053+0000","last_clean_scrub_stamp":"2023-11-28T13:07:33.513053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:02.186266+0000","scrub_duration":0.0038105669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:07:37.037130+0000","last_change":"2023-11-28T13:07:37.037095+0000","last_active":"2023-11-28T13:07:37.037130+0000","last_peered":"2023-11-28T13:07:37.037130+0000","last_clean":"2023-11-28T13:07:37.037130+0000","last_became_active":"2023-11-28T13:00:12.210999+0000","last_became_peered":"2023-11-28T13:00:12.210999+0000","last_unstale":"2023-11-28T13:07:37.037130+0000","last_undegraded":"2023-11-28T13:07:37.037130+0000","last_fullsized":"2023-11-28T13:07:37.037130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:37.036991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:37.036991+0000","last_clean_scrub_stamp":"2023-11-28T13:07:37.036991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:29:38.580736+0000","scrub_duration":0.0035335029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9960,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361880,"statfs":{"total":96636764160,"available":96626565120,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.372,"15min":0.371},"min":{"1min":0.25,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":0.497,"5min":0.57699999999999996,"15min":1.2669999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.57399999999999995},"min":{"1min":0.222,"5min":0.222,"15min":0.216},"max":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":8.218},"last":0.36299999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39800000000000002},"min":{"1min":0.26300000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.623,"5min":0.63600000000000001,"15min":1.8280000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.41399999999999998,"15min":0.41199999999999998},"min":{"1min":0.29699999999999999,"5min":0.218,"15min":0.214},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":1.355},"last":0.504}]},{"osd":2,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.76400000000000001},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55000000000000004,"5min":0.60199999999999998,"15min":13.384},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.28000000000000003,"15min":0.26500000000000001},"max":{"1min":0.60999999999999999,"5min":0.65200000000000002,"15min":5.226},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.35999999999999999,"15min":0.35699999999999998},"max":{"1min":0.81799999999999995,"5min":0.94499999999999995,"15min":8.1890000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63300000000000001,"15min":0.73399999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78900000000000003,"5min":0.93999999999999995,"15min":5.3609999999999998},"last":0.66400000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":1.3029999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.80400000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38200000000000001},"max":{"1min":0.878,"5min":0.96899999999999997,"15min":8.1579999999999995},"last":0.56299999999999994}]},{"osd":6,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.74299999999999999},"min":{"1min":0.42399999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.97699999999999998,"15min":5.3209999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.3320000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.83099999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83499999999999996,"5min":0.93000000000000005,"15min":8.1029999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.76000000000000001},"min":{"1min":0.50800000000000001,"5min":0.40300000000000002,"15min":0.38900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":5.3849999999999998},"last":0.70599999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509537,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9640,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362200,"statfs":{"total":96636764160,"available":96626892800,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35199999999999998,"15min":0.33000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":0.51700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.35199999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.187},"max":{"1min":0.48599999999999999,"5min":0.61599999999999999,"15min":1.0169999999999999},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.224,"15min":0.20499999999999999},"max":{"1min":0.52600000000000002,"5min":0.62,"15min":0.95999999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39200000000000002},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.221},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":1.1459999999999999},"last":0.378}]},{"osd":3,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.39200000000000002,"15min":0.36599999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.23499999999999999},"max":{"1min":0.55400000000000005,"5min":0.63400000000000001,"15min":1.1659999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.33900000000000002,"5min":0.27500000000000002,"15min":0.23799999999999999},"max":{"1min":0.57599999999999996,"5min":0.70299999999999996,"15min":1.1919999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.623,"15min":0.58799999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.41999999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.641},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.434,"5min":0.39400000000000002,"15min":0.36799999999999999},"max":{"1min":0.76500000000000001,"5min":0.91100000000000003,"15min":1.056},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.623,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.89800000000000002,"15min":1.6120000000000001},"last":0.748},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60899999999999999,"15min":0.58399999999999996},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.34499999999999997},"max":{"1min":0.72599999999999998,"5min":0.82899999999999996,"15min":1.774},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.626,"15min":0.58199999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.83799999999999997,"5min":0.84599999999999997,"15min":1.242},"last":0.625},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.627,"15min":0.61599999999999999},"min":{"1min":0.44600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.92900000000000005,"15min":1.6539999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40600000000000003},"max":{"1min":0.753,"5min":0.876,"15min":1.6919999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.874,"5min":0.89300000000000002,"15min":1.2130000000000001},"last":0.66700000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":9896,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361944,"statfs":{"total":96636764160,"available":96626630656,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.34699999999999998},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59699999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.35999999999999999},"max":{"1min":0.75800000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.56499999999999995},"min":{"1min":0.442,"5min":0.41299999999999998,"15min":0.33600000000000002},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.628}]},{"osd":2,"last update":"Tue Nov 28 13:07:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":1.2270000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.52700000000000002},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.745,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56599999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":0.86899999999999999,"15min":1.2110000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.55800000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.39000000000000001},"max":{"1min":0.77200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.76900000000000002,"15min":0.81200000000000006},"last":0.439},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51000000000000001,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.31},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40300000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.499,"15min":0.52900000000000003},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.247},"max":{"1min":0.69099999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54000000000000004,"15min":0.55000000000000004},"min":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52400000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56599999999999995,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":0.83599999999999997,"15min":0.88700000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.505}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14952,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":13568,"kb_avail":94356888,"statfs":{"total":96636764160,"available":96621453312,"internally_reserved":0,"allocated":1409024,"data_stored":950190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13893632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.33800000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.214},"max":{"1min":0.52800000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.39000000000000001,"15min":0.74199999999999999},"min":{"1min":0.34000000000000002,"5min":0.22,"15min":0.21199999999999999},"max":{"1min":0.56200000000000006,"5min":0.61599999999999999,"15min":14.957000000000001},"last":0.42599999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.747},"min":{"1min":0.36499999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53200000000000003,"5min":0.60499999999999998,"15min":15.032999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.754},"min":{"1min":0.40699999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.51900000000000002,"5min":0.59499999999999997,"15min":14.997999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41099999999999998,"15min":0.84699999999999998},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.21199999999999999},"max":{"1min":0.49399999999999999,"5min":0.56999999999999995,"15min":19.045999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.41099999999999998},"min":{"1min":0.40300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60699999999999998,"15min":0.67200000000000004},"min":{"1min":0.53700000000000003,"5min":0.379,"15min":0.35099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.1799999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":1.073},"min":{"1min":0.51300000000000001,"5min":0.40100000000000002,"15min":0.35299999999999998},"max":{"1min":0.754,"5min":0.89700000000000002,"15min":18.995000000000001},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61299999999999999,"15min":1.0600000000000001},"min":{"1min":0.55200000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83999999999999997,"5min":0.93700000000000006,"15min":19.02},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.628,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":4.2119999999999997},"last":0.70299999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63600000000000001,"15min":0.70199999999999996},"min":{"1min":0.57399999999999995,"5min":0.377,"15min":0.34699999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":4.2359999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63500000000000001,"15min":0.69799999999999995},"min":{"1min":0.56299999999999994,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":0.92500000000000004,"15min":4.1150000000000002},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.70199999999999996},"min":{"1min":0.57999999999999996,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.876,"15min":4.1539999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64500000000000002,"15min":1.085},"min":{"1min":0.63300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":18.957999999999998},"last":0.68300000000000005}]}]},{"osd":7,"up_from":14,"seq":60129542242,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11052,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360788,"statfs":{"total":96636764160,"available":96625446912,"internally_reserved":0,"allocated":1413120,"data_stored":950959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.26900000000000002},"max":{"1min":0.68700000000000006,"5min":0.78400000000000003,"15min":1.1699999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":0.81599999999999995,"15min":1.415},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.32000000000000001},"max":{"1min":0.72199999999999998,"5min":0.80600000000000005,"15min":1.867},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57299999999999995,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":0.70399999999999996,"5min":0.79700000000000004,"15min":4.7830000000000004},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.84499999999999997,"15min":1.9550000000000001},"last":0.62},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.34000000000000002},"max":{"1min":0.68400000000000005,"5min":0.81499999999999995,"15min":1.9510000000000001},"last":0.67900000000000005}]},{"osd":3,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.35899999999999999},"max":{"1min":0.73299999999999998,"5min":0.88600000000000001,"15min":1.6859999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.64500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73799999999999999,"5min":0.89900000000000002,"15min":4.8440000000000003},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50800000000000001,"15min":0.51500000000000001},"min":{"1min":0.36699999999999999,"5min":0.33300000000000002,"15min":0.28499999999999998},"max":{"1min":0.69999999999999996,"5min":0.873,"15min":2.6030000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.36199999999999999,"15min":0.34699999999999998},"max":{"1min":0.68100000000000005,"5min":0.91600000000000004,"15min":1.528},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.311},"max":{"1min":0.64700000000000002,"5min":0.94699999999999995,"15min":1.25},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71599999999999997,"5min":0.93600000000000005,"15min":2.5329999999999999},"last":0.57099999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.626},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":0.65700000000000003,"5min":0.96599999999999997,"15min":1.972},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.70699999999999996,"5min":0.98899999999999999,"15min":2.508},"last":0.45600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542242,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9768,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362072,"statfs":{"total":96636764160,"available":96626761728,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68999999999999995,"5min":0.82799999999999996,"15min":1.419},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.68100000000000005,"5min":0.78500000000000003,"15min":0.80000000000000004},"last":0.56599999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.29799999999999999},"max":{"1min":0.66600000000000004,"5min":1.343,"15min":1.343},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.34799999999999998},"max":{"1min":0.71599999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56699999999999995,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70599999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.66300000000000003},"min":{"1min":0.44700000000000001,"5min":0.40799999999999997,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":3.6160000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.81100000000000005,"15min":1.4490000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.74399999999999999,"5min":1.228,"15min":1.228},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46100000000000002,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72199999999999998,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.53800000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.54000000000000004},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.748,"5min":0.85299999999999998,"15min":0.89600000000000002},"last":0.53400000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542241,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9576,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362264,"statfs":{"total":96636764160,"available":96626958336,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.41999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77100000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65600000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.75,"15min":0.75},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":0.746,"15min":1.0249999999999999},"last":0.72399999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47599999999999998,"5min":0.39000000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.753},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.54800000000000004},"min":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.38},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.62}]},{"osd":3,"last update":"Tue Nov 28 13:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.49099999999999999,"5min":0.36899999999999999,"15min":0.33900000000000002},"max":{"1min":0.76500000000000001,"5min":0.77700000000000002,"15min":0.79500000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.55700000000000005},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78000000000000003},"last":0.76700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.33100000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.80500000000000005,"15min":0.81699999999999995},"last":0.495},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.83399999999999996},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.55700000000000005},"min":{"1min":0.40000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.92600000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.35799999999999998},"max":{"1min":0.77200000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.63200000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509538,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9832,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362008,"statfs":{"total":96636764160,"available":96626696192,"internally_reserved":0,"allocated":819200,"data_stored":360591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.46300000000000002,"5min":0.60999999999999999,"15min":0.97099999999999997},"last":0.249},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.39300000000000002,"15min":0.35899999999999999},"min":{"1min":0.25800000000000001,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":0.55400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.375,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52900000000000003,"5min":0.57499999999999996,"15min":0.72899999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.41099999999999998,"15min":0.378},"min":{"1min":0.27500000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54500000000000004,"5min":0.60099999999999998,"15min":0.64300000000000002},"last":0.309}]},{"osd":3,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45400000000000001,"15min":0.41499999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26500000000000001},"max":{"1min":0.67000000000000004,"5min":0.69199999999999995,"15min":0.752},"last":0.442},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42699999999999999,"15min":0.40400000000000003},"min":{"1min":0.29399999999999998,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.875},"last":0.41399999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.58799999999999997},"min":{"1min":0.46999999999999997,"5min":0.442,"15min":0.36299999999999999},"max":{"1min":0.871,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.47099999999999997,"5min":0.41399999999999998,"15min":0.36799999999999999},"max":{"1min":0.85599999999999998,"5min":0.90600000000000003,"15min":1.044},"last":0.58999999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.61299999999999999},"min":{"1min":0.47699999999999998,"5min":0.433,"15min":0.28999999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92600000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.36799999999999999},"max":{"1min":0.82199999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.621}]},{"osd":6,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64600000000000002,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.46500000000000002,"15min":0.31900000000000001},"max":{"1min":0.83299999999999996,"5min":0.94299999999999995,"15min":1.0669999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.38900000000000001},"max":{"1min":0.90000000000000002,"5min":0.93899999999999995,"15min":1.089},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:07:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66700000000000004,"15min":0.63600000000000001},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.42299999999999999},"max":{"1min":0.87,"5min":0.95799999999999996,"15min":1.202},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65900000000000003,"15min":0.63600000000000001},"min":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.42999999999999999},"max":{"1min":0.83499999999999996,"5min":0.95799999999999996,"15min":1.02},"last":0.63700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:53.756 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T13:07:53.876 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.875+0000 7f9cfc4c5700 1 Processor -- start 2023-11-28T13:07:53.877 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.875+0000 7f9cfc4c5700 1 -- start start 2023-11-28T13:07:53.882 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.875+0000 7f9cfc4c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 0x7f9cf4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.875+0000 7f9cfc4c5700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf4071950 con 0x7f9cf4160540 2023-11-28T13:07:53.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cfc4c5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf415d060 con 0x7f9cf406dee0 2023-11-28T13:07:53.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cfc4c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf415d340 con 0x7f9cf4071040 2023-11-28T13:07:53.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cfaa62700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9cf4160540 0x7f9cf41609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33872/0 (socket says 172.21.15.153:33872) 2023-11-28T13:07:53.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cfaa62700 1 -- 172.21.15.153:0/1844294026 learned_addr learned my addr 172.21.15.153:0/1844294026 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:53.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cfa261700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 0x7f9cf4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.883 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970066353 0 0) 0x7f9cf4071950 con 0x7f9cf4160540 2023-11-28T13:07:53.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce00035b0 con 0x7f9cf4160540 2023-11-28T13:07:53.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3536210957 0 0) 0x7f9ce00035b0 con 0x7f9cf4160540 2023-11-28T13:07:53.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 msgr2=0x7f9cf4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 0x7f9cf4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 >> v1:172.21.15.177:6789/0 conn(0x7f9cf406dee0 legacy=0x7f9cf406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf415d620 con 0x7f9cf4160540 2023-11-28T13:07:53.884 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f9cec0019f0 con 0x7f9cf4160540 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 >> v1:172.21.15.153:6789/0 conn(0x7f9cf4160540 legacy=0x7f9cf41609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce0003a30 0x7f9ce0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ce0004850 0x7f9ce0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 0x7f9ce0005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf415d340 con 0x7f9ce0003a30 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce000bb40 con 0x7f9ce0004850 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce000bd00 con 0x7f9cf4071040 2023-11-28T13:07:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cfa261700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce0003a30 0x7f9ce0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cf9a60700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ce0004850 0x7f9ce0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.876+0000 7f9cfaa62700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 0x7f9ce0005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfa261700 1 -- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 msgr2=0x7f9ce0005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfa261700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 0x7f9ce0005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfa261700 1 -- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ce0004850 msgr2=0x7f9ce0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfa261700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ce0004850 0x7f9ce0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfa261700 1 -- 172.21.15.153:0/1844294026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00035b0 con 0x7f9ce0003a30 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfaa62700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cf4071040 0x7f9ce0005240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cf9a60700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ce0004850 0x7f9ce0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfa261700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce0003a30 0x7f9ce0003e00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009b60 tx=0x7f9ce0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e373aab7f35c51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000e6c0 con 0x7f9ce0003a30 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000e820 con 0x7f9ce0003a30 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cf925f700 1 -- 172.21.15.153:0/1844294026 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000eb90 con 0x7f9ce0003a30 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/1844294026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce0003a30 msgr2=0x7f9ce0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfc4c5700 1 --2- 172.21.15.153:0/1844294026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ce0003a30 0x7f9ce0003e00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009b60 tx=0x7f9ce0005880 comp rx=0 tx=0).stop 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfaa62700 1 -- 172.21.15.153:0/1844294026 reap_dead start 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/1844294026 shutdown_connections 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.877+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/1844294026 >> 172.21.15.153:0/1844294026 conn(0x7f9cf40d0000 msgr2=0x7f9cf40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.878+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/1844294026 shutdown_connections 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.878+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/1844294026 wait complete. 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 Processor -- start 2023-11-28T13:07:53.888 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 -- start start 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ce0003a30 0x7f9cf4159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cf40c59f0 0x7f9cf41f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf41f20d0 0x7f9cf4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40c5e60 con 0x7f9cf41f20d0 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4159690 con 0x7f9cf40c59f0 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.879+0000 7f9cfc4c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40c4c30 con 0x7f9ce0003a30 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf41f20d0 0x7f9cf4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf41f20d0 0x7f9cf4159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43952/0 (socket says 172.21.15.153:43952) 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 -- 172.21.15.153:0/3111419769 learned_addr learned my addr 172.21.15.153:0/3111419769 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cf9a60700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cf40c59f0 0x7f9cf41f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfa261700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ce0003a30 0x7f9cf4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 -- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ce0003a30 msgr2=0x7f9cf4159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ce0003a30 0x7f9cf4159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 -- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cf40c59f0 msgr2=0x7f9cf41f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cf40c59f0 0x7f9cf41f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 -- 172.21.15.153:0/3111419769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf415ccd0 con 0x7f9cf41f20d0 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cf9a60700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cf40c59f0 0x7f9cf41f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfaa62700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf41f20d0 0x7f9cf4159a10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9cec001460 tx=0x7f9cec00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00ec70 con 0x7f9cf41f20d0 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec00edf0 con 0x7f9cf41f20d0 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf415d900 con 0x7f9cf41f20d0 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec013660 con 0x7f9cf41f20d0 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.880+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf406a270 con 0x7f9cf41f20d0 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.881+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf40720a0 con 0x7f9cf41f20d0 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.882+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9cec0137c0 con 0x7f9cf41f20d0 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.882+0000 7f9ceaffd700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9cd8068320 0x7f9cd806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.882+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9cec086030 con 0x7f9cf41f20d0 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.882+0000 7f9cfa261700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9cd8068320 0x7f9cd806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.882+0000 7f9cfa261700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9cd8068320 0x7f9cd806a7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00090b0 tx=0x7f9cf0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.894 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:53.886+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9cec052b90 con 0x7f9cf41f20d0 2023-11-28T13:07:54.070 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.069+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f9cf40cd680 con 0x7f9cf41f20d0 2023-11-28T13:07:54.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.073+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9cec052b90 con 0x7f9cf41f20d0 2023-11-28T13:07:54.074 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.073+0000 7f9ceaffd700 1 -- 172.21.15.153:0/3111419769 <== mon.0 v2:172.21.15.153:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9cec0561b0 con 0x7f9cf41f20d0 2023-11-28T13:07:54.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9cd8068320 msgr2=0x7f9cd806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9cd8068320 0x7f9cd806a7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00090b0 tx=0x7f9cf0008030 comp rx=0 tx=0).stop 2023-11-28T13:07:54.078 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf41f20d0 msgr2=0x7f9cf4159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf41f20d0 0x7f9cf4159a10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9cec001460 tx=0x7f9cec00e440 comp rx=0 tx=0).stop 2023-11-28T13:07:54.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 shutdown_connections 2023-11-28T13:07:54.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9cd8068320 0x7f9cd806a7d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf41f20d0 0x7f9cf4159a10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.079 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ce0003a30 0x7f9cf4159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 --2- 172.21.15.153:0/3111419769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9cf40c59f0 0x7f9cf41f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 >> 172.21.15.153:0/3111419769 conn(0x7f9cf40d0000 msgr2=0x7f9cf40cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 shutdown_connections 2023-11-28T13:07:54.080 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.076+0000 7f9cfc4c5700 1 -- 172.21.15.153:0/3111419769 wait complete. 2023-11-28T13:07:54.090 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:07:54.090 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:54.211 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.210+0000 7f9edeee3700 1 Processor -- start 2023-11-28T13:07:54.212 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.210+0000 7f9edeee3700 1 -- start start 2023-11-28T13:07:54.215 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.210+0000 7f9edeee3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9ed8071040 0x7f9ed8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.210+0000 7f9edeee3700 1 -- --> v1:172.21.15.153:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed8071950 con 0x7f9ed8160540 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.210+0000 7f9edeee3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed815d060 con 0x7f9ed806dee0 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.210+0000 7f9edeee3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed815d340 con 0x7f9ed8071040 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9edd480700 1 --1- >> v1:172.21.15.153:6789/0 conn(0x7f9ed8160540 0x7f9ed8160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.153:6789/0 says I am v1:172.21.15.153:33888/0 (socket says 172.21.15.153:33888) 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9edd480700 1 -- 172.21.15.153:0/1655128433 learned_addr learned my addr 172.21.15.153:0/1655128433 (peer_addr_for_me v1:172.21.15.153:0/0) 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9edcc7f700 1 --2- 172.21.15.153:0/1655128433 >> v2:172.21.15.177:3300/0 conn(0x7f9ed8071040 0x7f9ed8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 <== mon.0 v1:172.21.15.153:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859583477 0 0) 0x7f9ed8071950 con 0x7f9ed8160540 2023-11-28T13:07:54.216 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 --> v1:172.21.15.153:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec80035b0 con 0x7f9ed8160540 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668774549 0 0) 0x7f9ed815d060 con 0x7f9ed806dee0 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed8071950 con 0x7f9ed806dee0 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 <== mon.0 v1:172.21.15.153:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2601895882 0 0) 0x7f9ec80035b0 con 0x7f9ed8160540 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 >> v2:172.21.15.177:3300/0 conn(0x7f9ed8071040 msgr2=0x7f9ed8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 --2- 172.21.15.153:0/1655128433 >> v2:172.21.15.177:3300/0 conn(0x7f9ed8071040 0x7f9ed8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 >> v1:172.21.15.177:6789/0 conn(0x7f9ed806dee0 legacy=0x7f9ed806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 --> v1:172.21.15.153:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed815d620 con 0x7f9ed8160540 2023-11-28T13:07:54.217 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 <== mon.0 v1:172.21.15.153:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2721871308 0 0) 0x7f9ed00019f0 con 0x7f9ed8160540 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 >> v1:172.21.15.153:6789/0 conn(0x7f9ed8160540 legacy=0x7f9ed8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8003be0 0x7f9ec8003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ec8004850 0x7f9ec8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ec80052c0 0x7f9ec800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed815d340 con 0x7f9ec8004850 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec800c200 con 0x7f9ec80052c0 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec800c3c0 con 0x7f9ec8003be0 2023-11-28T13:07:54.218 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9ed7fff700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ec8004850 0x7f9ec8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.211+0000 7f9edcc7f700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8003be0 0x7f9ec8003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edd480700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ec80052c0 0x7f9ec800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed7fff700 1 -- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8003be0 msgr2=0x7f9ec8003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed7fff700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8003be0 0x7f9ec8003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed7fff700 1 -- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ec80052c0 msgr2=0x7f9ec800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed7fff700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ec80052c0 0x7f9ec800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed7fff700 1 -- 172.21.15.153:0/1655128433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80035b0 con 0x7f9ec8004850 2023-11-28T13:07:54.219 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edd480700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ec80052c0 0x7f9ec800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edd480700 1 -- 172.21.15.153:0/1655128433 reap_dead start 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed7fff700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ec8004850 0x7f9ec8004cc0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4001980 tx=0x7f9ec8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d49953adebaef9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4001470 con 0x7f9ec8004850 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9ec40052f0 con 0x7f9ec8004850 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9ed77fe700 1 -- 172.21.15.153:0/1655128433 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4003680 con 0x7f9ec8004850 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edeee3700 1 -- 172.21.15.153:0/1655128433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ec8004850 msgr2=0x7f9ec8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edeee3700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ec8004850 0x7f9ec8004cc0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4001980 tx=0x7f9ec8005880 comp rx=0 tx=0).stop 2023-11-28T13:07:54.220 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edeee3700 1 -- 172.21.15.153:0/1655128433 shutdown_connections 2023-11-28T13:07:54.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edeee3700 1 --2- 172.21.15.153:0/1655128433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ec8004850 0x7f9ec8004cc0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.212+0000 7f9edeee3700 1 -- 172.21.15.153:0/1655128433 >> 172.21.15.153:0/1655128433 conn(0x7f9ed80d0000 msgr2=0x7f9ed80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.213+0000 7f9edeee3700 1 -- 172.21.15.153:0/1655128433 shutdown_connections 2023-11-28T13:07:54.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.213+0000 7f9edeee3700 1 -- 172.21.15.153:0/1655128433 wait complete. 2023-11-28T13:07:54.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.214+0000 7f9edeee3700 1 Processor -- start 2023-11-28T13:07:54.221 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.214+0000 7f9edeee3700 1 -- start start 2023-11-28T13:07:54.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8004850 0x7f9ed814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ed80add50 0x7f9ed80ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b09b0 0x7f9ed80b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80b1750 con 0x7f9ed80add50 2023-11-28T13:07:54.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80ae700 con 0x7f9ed80b09b0 2023-11-28T13:07:54.224 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80c4550 con 0x7f9ec8004850 2023-11-28T13:07:54.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ed80add50 0x7f9ed80ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ed80add50 0x7f9ed80ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43968/0 (socket says 172.21.15.153:43968) 2023-11-28T13:07:54.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 -- 172.21.15.153:0/2294062892 learned_addr learned my addr 172.21.15.153:0/2294062892 (peer_addr_for_me v2:172.21.15.153:0/0) 2023-11-28T13:07:54.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edcc7f700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8004850 0x7f9ed814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edd480700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b09b0 0x7f9ed80b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 -- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8004850 msgr2=0x7f9ed814b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.225 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8004850 0x7f9ed814b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 -- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b09b0 msgr2=0x7f9ed80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b09b0 0x7f9ed80b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 -- 172.21.15.153:0/2294062892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed815ccd0 con 0x7f9ed80add50 2023-11-28T13:07:54.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edcc7f700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8004850 0x7f9ed814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:54.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edd480700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b09b0 0x7f9ed80b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:54.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed7fff700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ed80add50 0x7f9ed80ae1c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c1d0 tx=0x7f9ec4001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.226 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed57fa700 1 -- 172.21.15.153:0/2294062892 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4006a70 con 0x7f9ed80add50 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed57fa700 1 -- 172.21.15.153:0/2294062892 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9ec4006bf0 con 0x7f9ed80add50 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed815d900 con 0x7f9ed80add50 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9ed57fa700 1 -- 172.21.15.153:0/2294062892 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007040 con 0x7f9ed80add50 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.215+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed806a270 con 0x7f9ed80add50 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.216+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed80c46e0 con 0x7f9ed80add50 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.217+0000 7f9ed57fa700 1 -- 172.21.15.153:0/2294062892 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ec402d070 con 0x7f9ed80add50 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.217+0000 7f9ed57fa700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ebc0682f0 0x7f9ebc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.227 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.217+0000 7f9edcc7f700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ebc0682f0 0x7f9ebc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.217+0000 7f9ed57fa700 1 -- 172.21.15.153:0/2294062892 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9ec4086300 con 0x7f9ed80add50 2023-11-28T13:07:54.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.218+0000 7f9edcc7f700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ebc0682f0 0x7f9ebc06a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009b10 tx=0x7f9ecc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.228 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.222+0000 7f9ed57fa700 1 -- 172.21.15.153:0/2294062892 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ec4052e60 con 0x7f9ed80add50 2023-11-28T13:07:54.448 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.446+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ed80716f0 con 0x7f9ed80add50 2023-11-28T13:07:54.449 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.447+0000 7f9ed57fa700 1 -- 172.21.15.153:0/2294062892 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f9ec4021de0 con 0x7f9ed80add50 2023-11-28T13:07:54.449 INFO:teuthology.orchestra.run.smithi153.stdout: 2023-11-28T13:07:54.449 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/677557627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":677557627},{"type":"v1","addr":"172.21.15.177:6835","nonce":677557627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5641,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1906792978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1906792978},{"type":"v1","addr":"172.21.15.177:6834","nonce":1906792978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6836/1664896724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6834","nonce":1664896724},{"type":"v1","addr":"172.21.15.153:6836","nonce":1664896724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6699,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6837/730034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6835","nonce":730034079},{"type":"v1","addr":"172.21.15.153:6837","nonce":730034079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2023-11-28T13:07:54.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ebc0682f0 msgr2=0x7f9ebc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ebc0682f0 0x7f9ebc06a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009b10 tx=0x7f9ecc008040 comp rx=0 tx=0).stop 2023-11-28T13:07:54.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ed80add50 msgr2=0x7f9ed80ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ed80add50 0x7f9ed80ae1c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c1d0 tx=0x7f9ec4001310 comp rx=0 tx=0).stop 2023-11-28T13:07:54.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 shutdown_connections 2023-11-28T13:07:54.451 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed80b09b0 0x7f9ed80b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ec8004850 0x7f9ed814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:6832/103291,v1:172.21.15.153:6833/103291] conn(0x7f9ebc0682f0 0x7f9ebc06a7a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 --2- 172.21.15.153:0/2294062892 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9ed80add50 0x7f9ed80ae1c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 >> 172.21.15.153:0/2294062892 conn(0x7f9ed80d0000 msgr2=0x7f9ed80ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 shutdown_connections 2023-11-28T13:07:54.452 INFO:teuthology.orchestra.run.smithi153.stderr:2023-11-28T13:07:54.449+0000 7f9edeee3700 1 -- 172.21.15.153:0/2294062892 wait complete. 2023-11-28T13:07:54.453 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 34 2023-11-28T13:07:54.463 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T13:07:54.463 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T13:07:54.463 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:07:54.463 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.471 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:07:54.472 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:07:54.472 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.520 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:07:54.520 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:07:54.520 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.531 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:07:54.531 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:07:54.531 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.582 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:07:54.582 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T13:07:54.583 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T13:07:54.583 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.615 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T13:07:54.615 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T13:07:54.615 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.694 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T13:07:54.695 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T13:07:54.695 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.765 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T13:07:54.765 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T13:07:54.765 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.845 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T13:07:54.845 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T13:07:54.845 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.886 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T13:07:54.887 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T13:07:54.887 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.936 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T13:07:54.936 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T13:07:54.936 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:54.974 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T13:07:54.974 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T13:07:54.974 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:55.056 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T13:07:55.057 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T13:07:55.057 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T13:07:55.058 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:55.083 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T13:07:55.083 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T13:07:55.083 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:55.122 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T13:07:55.122 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T13:07:55.123 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T13:07:55.123 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:55.134 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T13:07:55.134 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T13:07:55.134 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:55.173 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T13:07:55.174 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T13:07:55.174 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:07:55.224 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T13:07:55.224 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T13:07:55.225 DEBUG:teuthology.orchestra.run.smithi153:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T13:07:55.257 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:07:55.258 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T13:07:55.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:07:55.532 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T13:07:55.630 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:07:55.631 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T13:07:55.737 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:07:55.737 DEBUG:teuthology.orchestra.run.smithi153:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T13:07:55.817 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T13:07:55.818 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T13:07:56.052 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T13:07:56.052 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T13:07:56.143 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T13:07:56.144 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T13:07:56.229 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T13:07:56.229 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T13:07:56.313 INFO:tasks.ceph:Archiving mon data... 2023-11-28T13:07:56.315 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/data/mon.a.tgz 2023-11-28T13:07:56.315 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2023-11-28T13:07:56.333 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.sjIoQLJqA0 2023-11-28T13:07:56.333 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.sjIoQLJqA0 2023-11-28T13:07:56.493 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.sjIoQLJqA0 2023-11-28T13:07:56.579 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.sjIoQLJqA0 is 300KB 2023-11-28T13:07:56.647 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.sjIoQLJqA0 2023-11-28T13:07:56.663 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/data/mon.b.tgz 2023-11-28T13:07:56.664 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-11-28T13:07:56.681 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.0NkInx6eqj 2023-11-28T13:07:56.682 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0NkInx6eqj 2023-11-28T13:07:56.813 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.0NkInx6eqj 2023-11-28T13:07:56.898 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.0NkInx6eqj is 305KB 2023-11-28T13:07:56.965 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.0NkInx6eqj 2023-11-28T13:07:56.981 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/data/mon.c.tgz 2023-11-28T13:07:56.981 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-11-28T13:07:57.036 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.gWi4pH2Z6l 2023-11-28T13:07:57.037 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gWi4pH2Z6l 2023-11-28T13:07:57.160 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.gWi4pH2Z6l 2023-11-28T13:07:57.245 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.gWi4pH2Z6l is 305KB 2023-11-28T13:07:57.340 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.gWi4pH2Z6l 2023-11-28T13:07:57.356 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T13:07:57.356 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T13:07:57.359 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T13:07:57.421 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T13:07:57.422 DEBUG:teuthology.orchestra.run.smithi153:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T13:07:57.426 DEBUG:teuthology.orchestra.run.smithi177:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T13:07:57.458 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T13:07:57.494 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T13:07:57.500 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T13:07:57.502 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/remote/smithi153/crash 2023-11-28T13:07:57.503 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T13:07:57.531 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/remote/smithi177/crash 2023-11-28T13:07:57.532 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T13:07:57.565 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T13:07:57.566 DEBUG:teuthology.orchestra.run.smithi153:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:57.576 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:57.662 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:57.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:57.715 INFO:tasks.ceph:Compressing logs... 2023-11-28T13:07:57.715 DEBUG:teuthology.orchestra.run.smithi153:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:07:57.732 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:07:59.689 INFO:tasks.ceph:Archiving logs... 2023-11-28T13:07:59.689 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/remote/smithi153/log 2023-11-28T13:07:59.691 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T13:08:00.297 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/remote/smithi177/log 2023-11-28T13:08:00.298 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T13:08:00.555 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T13:08:00.564 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T13:08:00.565 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T13:08:00.567 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T13:08:00.651 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T13:08:00.661 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T13:08:00.661 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:08:00.663 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:08:00.677 INFO:teuthology.orchestra.run.smithi153.stderr:bash: ntpq: command not found 2023-11-28T13:08:00.680 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-11-28T13:08:00.681 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:08:00.682 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2023-11-28T13:08:00.682 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +918us[ +918us] +/- 98ms 2023-11-28T13:08:00.682 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -364us[ -340us] +/- 66ms 2023-11-28T13:08:00.682 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 -289us[ -266us] +/- 81ms 2023-11-28T13:08:00.682 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:08:00.684 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:08:00.684 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-11-28T13:08:00.685 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +922us[ +922us] +/- 98ms 2023-11-28T13:08:00.685 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -351us[ -320us] +/- 66ms 2023-11-28T13:08:00.685 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -267us[ -267us] +/- 81ms 2023-11-28T13:08:00.685 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:08:00.686 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T13:08:00.696 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T13:08:00.696 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T13:08:00.705 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T13:08:00.823 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T13:08:00.917 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T13:08:00.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:00.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T13:08:00.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:00.982 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T13:08:00.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T13:08:01.000 INFO:teuthology.task.internal:Duration was 1251.103448 seconds 2023-11-28T13:08:01.001 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T13:08:01.010 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T13:08:01.010 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T13:08:01.013 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T13:08:01.057 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T13:08:01.068 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T13:08:01.389 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T13:08:01.390 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2023-11-28T13:08:01.390 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T13:08:01.414 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T13:08:01.415 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T13:08:01.438 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T13:08:01.439 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:08:01.457 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:08:01.518 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T13:08:01.519 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T13:08:01.542 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T13:08:01.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T13:08:01.644 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T13:08:01.644 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T13:08:01.671 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T13:08:01.698 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T13:08:01.708 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:01.713 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:01.737 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2023-11-28T13:08:01.764 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2023-11-28T13:08:01.777 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:01.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:01.808 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:01.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:01.833 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T13:08:01.843 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T13:08:01.844 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/remote/smithi153 2023-11-28T13:08:01.844 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T13:08:01.889 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972/remote/smithi177 2023-11-28T13:08:01.889 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T13:08:01.926 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T13:08:01.927 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T13:08:01.929 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T13:08:01.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T13:08:01.990 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T13:08:01.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T13:08:02.000 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T13:08:02.000 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T13:08:02.002 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T13:08:02.016 INFO:teuthology.orchestra.run.smithi153.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 13:08 /home/ubuntu/cephtest 2023-11-28T13:08:02.033 INFO:teuthology.orchestra.run.smithi177.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 13:08 /home/ubuntu/cephtest 2023-11-28T13:08:02.034 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T13:08:02.047 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T13:08:02.067 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T13:08:02.094 DEBUG:teuthology.nuke:shortname: smithi153 2023-11-28T13:08:02.094 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T13:08:02.117 DEBUG:teuthology.nuke:shortname: smithi177 2023-11-28T13:08:02.117 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T13:08:02.121 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:36.653404', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T13:08:02.140 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:36.654995', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T13:08:02.174 INFO:teuthology.orchestra.console.smithi153:Power off 2023-11-28T13:08:02.175 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T13:08:02.200 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2023-11-28T13:08:02.201 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:02.215 INFO:teuthology.orchestra.console.smithi177:Power off 2023-11-28T13:08:02.216 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T13:08:02.227 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2023-11-28T13:08:02.239 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2023-11-28T13:08:02.239 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:02.264 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-11-28T13:08:06.228 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:06.265 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:06.341 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2023-11-28T13:08:06.379 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-11-28T13:08:10.346 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:10.380 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:10.458 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2023-11-28T13:08:10.491 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-11-28T13:08:10.559 INFO:teuthology.orchestra.console.smithi153:Power off completed 2023-11-28T13:08:10.592 INFO:teuthology.orchestra.console.smithi177:Power off completed 2023-11-28T13:08:10.697 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1251.103447675705 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ddd754f5f1d143ffb59c580558af7612 status: fail success: false 2023-11-28T13:08:10.698 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T13:08:10.777 INFO:teuthology.run:FAIL